fix: validate Kimi PP8 DFlash PD on GSM8K C1 and C8
This commit is contained in:
Zhiyi Hong
parent
0e33209ae5
commit
3fbfb84e12
@ -1,5 +1,7 @@
|
||||
# sskj — 多平台大模型推理性能基准测试项目
|
||||
|
||||
**更新（2026-08-31 18:22:43 CST）**：完成 Kimi-K3 PP8 + DFlash 八节点 PD 部署验证。P 601-604 为 TP4/PP8/EP4，D 605-608 为 TP32/PP1/EP4，均使用 FlashInfer MXFP4、BF16 KV、8K Chunk。固定 GSM8K 64 题 C1/C8 共 128/128 请求成功，答案检查全部通过且无输出截断；按 verify 步数加权的接受长度为 6.4211/6.5158，整体 Output TPS 为 33.38/117.56。修复评测容器的数据集路径，保留原失败与续跑证据；结果、完整服务日志及资源清理记录已归档。历史 5-shot 与前 5 题重叠，本轮作为部署回归而非独立准确率评测。详见 [实现与结果](experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md)。
|
||||
|
||||
**更新（2026-08-31 17:50:51 CST）**：修复 Kimi 混合注意力缓存池在 PD 初始化时缺少 `end_layer` 的兼容问题，按完整注意力层偏移和本地层数确定传输范围，保留普通池与 layer-shard 路径。新增回归后 24 项 CPU 测试通过，八节点修复镜像摘要一致；P 组已完成预热和 Mooncake 注册，健康检查返回 200，D 组正在验证启动。当前 Run 为 `pd-dflash-kvbounds1-20260831-1740`，尚无 GSM8K 验收结果。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
|
||||
|
||||
**更新（2026-08-31 17:05:00 CST）**：新增 Kimi-K3 PP8 + DFlash PD 适配验证入口。基于 SGLang PR #33863 固定源码，接通 PP 分段 hidden 投影、P 侧 prompt draft KV 生成、D 侧输入生命周期与 TP4→TP32 的 draft GQA KV 传输，保留 Kimi SM120 FlashInfer MXFP4 接入。修复 P 普通预热误带 DFlash verify metadata 的启动问题，23 项 CPU 回归通过；修复镜像已同步 601–608，服务级验证正在进行，尚无 GSM8K 结果。配置为 P TP4/PP8/EP4、D TP32/PP1/EP4、BF16 KV、8K Chunk，计划固定 64 题 C1/C8。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
|
||||
|
||||
@ -1,18 +1,18 @@
|
||||
# Kimi-K3：从 PP + DSpark 迁移到 PP + DFlash
|
||||
|
||||
更新：2026-08-31；PP、传输与 MoE 集成首版。
|
||||
更新：2026-08-31；PP8 + PD + DFlash 集成与 GSM8K 64 题 C1/C8 验证完成。
|
||||
|
||||
## 1. 决策与范围
|
||||
|
||||
**优先推进 DFlash 的 PP + PD 适配。** #33863 的分段投影设计能复用于 DFlash，但需要补齐 DFlash worker、调度入口和异构 TP 下的 draft KV 传输。
|
||||
|
||||
当前已实现 DFlash PP worker、Kimi 跨 stage capture、PD 输入衔接、异构 TP 的 draft KV 传输，以及 Kimi FlashInfer 布局/SiTU 接入。P 端两次启动分别暴露普通预热误带 verify metadata、混合池缺少 `end_layer` 的问题，均已针对实际调用点修复，24 项 CPU 测试通过。SM120 编译及 Kimi SiTU GPU 集成回归已通过。Run `pd-dflash-kvbounds1-20260831-1740` 的 P 组健康检查已返回 200，D 组正在启动；尚无跨组请求和 GSM8K 结果。详见 [实现进度与命令](evidence/kimi_k3_pd_dflash/README.md)。
|
||||
已实现 DFlash PP worker、Kimi 跨 stage capture、PD 输入衔接、异构 TP 的 draft KV 传输及 Kimi FlashInfer 布局/SiTU 接入。24 项 CPU 回归及 SM120 GPU 集成回归通过。Run `pd-dflash-kvbounds1-20260831-1740` 已完成跨组 smoke 与 GSM8K 64 题 C1/C8：两轮各 64/64 请求和答案检查通过，均无输出截断；按验证步数加权的接受长度为 6.4211/6.5158，整体 Output TPS 为 33.38/117.56。八节点实验容器已清理、GPU 已释放。详见 [实现、结果与实际命令](../experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md)。
|
||||
|
||||
暂停 EAGLE3 baseline。KV cache 保持 BF16，chunk 保持 8192。FlashInfer 使用官方 0.6.18 加已合并 #4460 的显式 backport；原版 0.6.18 wheel 尚未包含所需 CUTLASS SiTU 接口。
|
||||
|
||||
目标拓扑：P 使用 601–604 的 TP4/PP8/EP4 与已适配的 FlashInfer MXFP4；D 使用 605–608 的 TP32/PP1，接入 DFlash。这里适配的是 **P 侧 PP 生成 draft 上下文**，D 侧投机执行仍为 PP1。
|
||||
已验证拓扑：P 使用 601–604 的 TP4/PP8/EP4；D 使用 605–608 的 TP32/PP1/EP4，两组均为 FlashInfer MXFP4。这里适配的是 **P 侧 PP 生成 draft 上下文**，D 侧投机执行仍为 PP1。
|
||||
|
||||
审计固定版本：
|
||||
前期源码审计快照（PR 状态为该次审计记录，不表示实时状态）：
|
||||
|
||||
| 对象 | 版本 |
|
||||
|---|---|
|
||||
@ -91,14 +91,14 @@ PR 当前 [Mooncake flat 传输分支](https://github.com/sgl-project/sglang/blo
|
||||
|
||||
具体用例：D buffer 起始地址设为 2000000，目标 page ID=2。正确目标地址为 `2000000 + 2×16384 = 2032768`，当前函数生成 `2065536`，复制 32768 bytes 而不是单 head 页的 16384 bytes。
|
||||
|
||||
这是特定布局的地址规划复现，尚未启动完整服务复现。迁移时必须加入每条目的布局校验，并实现 GQA head-aware 传输；不能直接复用 flat copy。还需确认 D 接收到全部 target/draft 组件后才进入首轮 draft。
|
||||
这是原 PR 的特定布局地址规划复现，未在 GPU 上故意执行越界复制。当前实现已补齐每条目布局校验及 GQA head-aware 传输，CPU 测试覆盖全部 P-TP4→D-TP32 head 映射；修复版本的完整服务已通过跨组请求和 C1/C8 评测。原 flat-copy 失败证据保留用于说明修复动机。
|
||||
|
||||
## 5. 实现与验收顺序
|
||||
## 5. 验收与结果口径
|
||||
|
||||
1. **局部数学与协议验证。** 检查各 stage capture 层覆盖、feature 顺序、空 capture stage、单次 RMSNorm；比较拼接投影与分段投影误差。测试 TP4→TP32 的全部 8 个 draft KV head、K/V、6 层、页索引以及失败清理。源/目标布局不兼容时在传输前报错。
|
||||
2. **最小 PD 请求。** 保留 P TP4/PP8、D TP32/PP1，先 C1。核对真实 buffer metadata、draft KV 到达情况、首轮 draft logits，以及成功/失败后的资源释放。
|
||||
3. **GSM8K 固定 64 题。** 相同 prompt/template、采样设置，依次测试 C1/C8；保留准确率、完成率、接受长度、接受直方图、Output TPS、总耗时与显存状态。当前客户端非流式，不把总耗时换算成 TTFT/ITL。输入不裁剪，输出上限 512，报告截断，不新增 16K synthetic 测试。历史 5-shot 与前 64 题有重叠，本轮沿用以便部署回归，不能作为独立无泄漏的模型准确率评估。
|
||||
4. **验收后决定继续投入。** DFlash 正确性与部署通过且有实际 Decode 收益，继续 PD 调优；若关键适配无法通过，或接受行为同样异常，切换到下面的 D-only 诊断。
|
||||
1. **局部数学与协议验证已通过。** 覆盖各 stage capture、feature 顺序、空 capture stage、单次 RMSNorm、BF16/FP32 分段投影与 TP4→TP32 传输规划。CPU 方法级回归不等于完整模型 logits 等价验证。
|
||||
2. **最小 PD 请求已通过。** P TP4/PP8、D TP32/PP1，经 Router 返回 128 token，服务日志有实际 RDMA ready ACK，D Decode 使用 CUDA Graph。
|
||||
3. **GSM8K 固定 64 题 C1/C8 已完成。** 两轮各 64 个响应均成功，独立解析最终答案也全部通过；总耗时 195.42/55.55 秒，无 512-token 截断。完整逐题输出、直方图、服务日志和资源清理证据见结果目录。客户端非流式，不把总耗时换算成 TTFT/ITL。
|
||||
4. **统计口径已复核。** 接口的每题等权接受长度为 6.9433/7.0070；直方图按 verify 步数加权为 6.4211/6.5158。两者权重和首尾 token 处理不同，不混用。5-shot 与前 5 题重叠，排除后剩余 59 题也均通过；这仍是历史口径部署回归，不是独立无泄漏的模型能力评测。本轮不重跑 no-spec，因此不宣称投机加速倍数。
|
||||
|
||||
真实实验代码继续维护在 601 的 `/data/hzy/sskj` 工作区，不改同事部署目录，不再创建分支。本轮核实当前检出名为 `hzy-kimi3-pd-pp8-standard`，保持现状；独立 SGLang 源码位于 `/data/hzy/src/sglang-kimi-pp-dflash-33863`。
|
||||
|
||||
|
||||
32
docs/evidence/kimi_k3_pp_dflash_pr33863_transfer_plan.json
Normal file
32
docs/evidence/kimi_k3_pp_dflash_pr33863_transfer_plan.json
Normal file
@ -0,0 +1,32 @@
|
||||
{
|
||||
"upstream_commit": "6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5",
|
||||
"scope": "Extracted upstream functions, synthetic pointer metadata, no GPU or network I/O",
|
||||
"source_sha256": {
|
||||
"transfer": "2f5a73b256e040966e181a5695fb4fa97e9fc7fb5c4760832e180b0683820cea",
|
||||
"pairing": "47bc97bfb649d6564c870113d276a1043230271f2d8924e8d41744cea12c971a"
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"case": "equal_tp_control",
|
||||
"source_page_stride_bytes": 16384,
|
||||
"destination_page_stride_bytes": 16384,
|
||||
"actual_source_address": 1016384,
|
||||
"actual_destination_address": 2032768,
|
||||
"expected_destination_address": 2032768,
|
||||
"actual_copy_bytes": 16384,
|
||||
"one_destination_head_page_bytes": 16384,
|
||||
"layout_matches": true
|
||||
},
|
||||
{
|
||||
"case": "P_TP4_to_D_TP32_DFlash",
|
||||
"source_page_stride_bytes": 32768,
|
||||
"destination_page_stride_bytes": 16384,
|
||||
"actual_source_address": 1032768,
|
||||
"actual_destination_address": 2065536,
|
||||
"expected_destination_address": 2032768,
|
||||
"actual_copy_bytes": 32768,
|
||||
"one_destination_head_page_bytes": 16384,
|
||||
"layout_matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -1,109 +1,146 @@
|
||||
# Kimi-K3 PP + DFlash 适配进度
|
||||
# Kimi-K3 PP8 + DFlash PD：实现与验证
|
||||
|
||||
2026-08-31：PP worker、PD 输入衔接、异构 TP 传输和 Kimi FlashInfer 接入已完成首版；SM120 编译及 Kimi SiTU GPU 集成回归通过。两项启动兼容修复及 24 项 CPU 回归已完成。Run `pd-dflash-kvbounds1-20260831-1740` 的 P 组已通过预热、Mooncake 初始化和健康检查（HTTP 200），D 组正在启动；尚无跨组请求或 GSM8K 结果。
|
||||
更新：2026-08-31。**八节点部署与固定 GSM8K 64 题 C1/C8 验证完成，128/128 请求成功，答案检查均通过，无输出截断。** 本次完成的是用户后续指定的 DFlash 路线，未恢复 EAGLE3/no-spec 实验。
|
||||
|
||||
## 源码和环境
|
||||
## 1. 目标与实现
|
||||
|
||||
- 基于官方 PR #33863 的 `6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5`。
|
||||
- 上游 Python 目录的 4619 个文件均与 Git blob SHA1 校验一致。
|
||||
- `pp_dflash_integration.patch` 包含完整 SGLang 增量；`pp_dflash_files.json` 记录文件 SHA256。`mixed_kv_transfer.patch` 保留传输层首版记录。
|
||||
- 601 独立源码路径：`/data/hzy/src/sglang-kimi-pp-dflash-33863`。
|
||||
- 601 实验路径：`/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation`。
|
||||
- 不修改原 EAGLE3/DSpark 部署；不创建新分支。
|
||||
保留 P 侧 PP8 的部署方案，让 P 生成真实 DFlash prompt KV，经 Mooncake 传给 D，D 接着执行投机解码。
|
||||
|
||||
## 本轮实现
|
||||
```text
|
||||
P 601-604：TP4 / PP8 / EP4
|
||||
各 stage 采集 target hidden，计算局部投影
|
||||
→ PP stage 间累加投影结果
|
||||
→ 最后 stage 统一 RMSNorm，生成 6 层 draft KV
|
||||
→ 传输 target MLA KV、KDA 状态及 draft KV
|
||||
D 605-608：TP32 / PP1 / EP4
|
||||
接收上下文 → DFlash 提议 → target 验证 → 提交 token
|
||||
```
|
||||
|
||||
| 部分 | 实现 |
|
||||
基于 SGLang PR #33863 固定提交 `6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5`。分段投影复用该 PR 的设计；新增 DFlash worker、Kimi capture、PD 调度衔接和异构 TP 传输，保留已验证的 Kimi SM120 FlashInfer MXFP4 接入。
|
||||
|
||||
| 模块（相对于 `python/sglang/srt/`） | 本次改动 |
|
||||
|---|---|
|
||||
| Kimi hidden 采集 | 接入 DFlash checkpoint 的 capture 层；PP 边界由下一 stage 使用实际 residual 聚合权重采集 |
|
||||
| 分段投影 | 各 stage 计算对应权重列的局部投影，通过 PP proxy 累加；末 stage 统一 RMSNorm 并写 6 层 draft KV |
|
||||
| P 侧 draft 执行 | 非末 stage 只保留最小 KV pool；P 不执行 draft decode CUDA Graph，D 的原有初始化路径保留 |
|
||||
| Scheduler 与 PD | 接通 DFlash proxy、next draft input、FutureMap 发布与 idle 生命周期；仅放开 P 侧 PP |
|
||||
| Mooncake 传输 | 注册每 entry 的 dtype/head/page stride/容量；target MLA 整页传输，draft GQA 按 head 交集逐 token 切片；写入前完成全部边界检查 |
|
||||
| FlashInfer | 迁移已验证的 Kimi SM120 布局与 SiTU 参数适配，同时保留新上游 SwigluStep 行为 |
|
||||
| `models/kimi_k3.py`、`speculative/dflash_pp.py` | 按 checkpoint 采集指定层；处理 PP 边界 residual；分段投影后只做一次归一化 |
|
||||
| `speculative/dflash_worker_v2.py` | 接收、传递 PP 上下文；最后 stage 写 draft KV；非末 stage 保留最小 KV pool |
|
||||
| `managers/scheduler_pp_mixin.py`、`speculative/spec_info.py` | 接通 DFlash proxy、PD 输入、FutureMap 与 idle 生命周期 |
|
||||
| `disaggregation/common/kv_entry_layout.py`、`mooncake/conn.py` 等 | target MLA 整页传输；draft GQA 按 head 交集逐 token 切片；独立计算源/目标 stride，在提交前检查边界 |
|
||||
| `model_executor/runner/base_runner.py`、`disaggregation/prefill.py` | 修复 P 普通预热误带 verify metadata，以及混合 KV 池传输层范围计算 |
|
||||
| `arg_groups/` | 仅放开已实现的 P 侧 PP + DFlash，D 侧保持 PP1 |
|
||||
|
||||
目标范围：Kimi hybrid MLA + 普通 NHD DFlash draft KV、Mooncake、CP=DCP=1，无 staging/unified KV/HiSparse。P TP4/PP8/EP4；D TP32/PP1；BF16 KV；chunk=8192。
|
||||
完整增量见 [pp_dflash_integration.patch](pp_dflash_integration.patch)，19 个文件校验值见 [pp_dflash_files.json](pp_dflash_files.json)。独立源码位于 601 的 `/data/hzy/src/sglang-kimi-pp-dflash-33863`，不覆盖同事部署源码。
|
||||
|
||||
## FlashInfer 依赖
|
||||
## 2. 实际配置
|
||||
|
||||
官方 `#4460` 已于 2026-08-21 合并，提交为 `b460bc00cb373541102d2155aec35bd626e522ce`。但 `v0.6.18` 属于另一条发布分支，实际下载的官方 wheel 没有 CUTLASS `situ_beta/situ_linear_beta` API。
|
||||
| 项目 | 本次值 |
|
||||
|---|---|
|
||||
| GPU | 8 节点 × 8 张 RTX 6000D；每卡 85651 MiB，SM120，PCIe |
|
||||
| P / D 并行 | TP4/PP8/EP4；TP32/PP1/EP4 |
|
||||
| MoE / A2A | `flashinfer_mxfp4` / `none`，两组一致 |
|
||||
| Target / Draft KV | 均为 `bfloat16` |
|
||||
| Chunk / Page | 8192 / 64 |
|
||||
| 最大活跃请求 / Decode Graph BS | 8 / 8；D 日志确认实际 Decode 使用 CUDA Graph |
|
||||
| P / D 显存比例 | 0.88 / 0.86 |
|
||||
| P / D Mamba memory ratio | 0.36 / 0.21 |
|
||||
| Prefix Cache | `--disable-radix-cache`；128 个响应的 cached tokens 均为 0 |
|
||||
| RDMA | Mooncake，`mlx5_0,mlx5_1,mlx5_2,mlx5_3`，容器挂载 `/dev/infiniband` |
|
||||
| Draft | `/data/hf_models/Kimi-K3-DFlash`，6 层，hidden 7168，32 Q heads / 8 KV heads，head dim 128 |
|
||||
| Draft 上下文 | Sliding window 4096，RoPE theta 2000000，capture `[19,37,54,66,78,90]` |
|
||||
| Draft block | `--speculative-num-draft-tokens 16`；每个 verify step 统计 15 个 draft 候选位置 |
|
||||
|
||||
因此本镜像使用 **官方 0.6.18 wheel + 未改写的 #4460 合并补丁**。Dockerfile 对 2 个 Python 和 5 个 C++ 文件先执行 `git apply --check`，再应用上游补丁。它不是未经修改的官方 0.6.18，也不是旧的私人 SiTU kernel。
|
||||
当前镜像：`local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1`。Linux/amd64 manifest 为 `e74c9b1e7fb6a9509db32576d2c9196b42ef329118909eeebd620cb9db2f91e9`；image config 为 `26584138eb1e097183ce167cbffe4198be0611e0cfde7422cbe692100d865982`。八节点镜像身份已核对。
|
||||
|
||||
移除镜像内旧版 `flashinfer-cubin`、`flashinfer-jit-cache`，保留版本校验；SM120 CUTLASS 通过官方 JIT 路径编译。[安装说明](https://docs.flashinfer.ai/installation.html)、[#4460](https://github.com/flashinfer-ai/flashinfer/pull/4460)。
|
||||
FlashInfer 使用**官方 0.6.18 wheel + 已合并 #4460 的原样 backport**（`b460bc00cb373541102d2155aec35bd626e522ce`）。该 wheel 所属发布分支尚无 CUTLASS SiTU API，不能描述为“未经修改的 0.6.18 直接支持”。移除旧 cubin/JIT-cache 包，保留版本检查；编译缓存挂载到 `/data/hzy/cache/kimi-dflash-fi0618-situ4460`。最后两项启动修复仅替换 Python 层，不改变 FlashInfer kernel。
|
||||
|
||||
## CPU 验证
|
||||
[Dockerfile](Dockerfile) 的完整重建输入保留在 601 本实验 `results/`：`sglang-kimi-pp-dflash-33863-integrated.tar.gz`（SHA256 `226065165bbb5f85ce516b847ff6da7e5aa823f12e69b87558e67c91996e3ba0`）和 `official_flashinfer_0618/flashinfer_python-0.6.18-py3-none-any.whl`（SHA256 `d5d26edb48f8def0bf28b2c94415aef7f0754e2cd9e3cb3d6e7081262d068aad`）。源码归档含固定上游 Python 树、补丁后的代码、3 个测试文件及 `SOURCE_MANIFEST.json`；GPU 测试和服务验证状态以外部结果日志为准，不回写构建时的历史 manifest。
|
||||
|
||||
```bash
|
||||
# 独立传输测试，无需 GPU 或 torch
|
||||
cd /data/hzy/src/sglang-kimi-pp-dflash-33863
|
||||
python3 test/registered/disaggregation/test_mixed_kv_entry_layout.py
|
||||
## 3. 测试结果
|
||||
|
||||
# 全部 CPU 回归需要带 torch 的容器；镜像构建时自动执行
|
||||
python3 -m unittest discover -s test/registered/disaggregation -v
|
||||
Run ID：`pd-dflash-kvbounds1-20260831-1740`。使用 GSM8K 实际文件前 64 题、5-shot、temperature=0、top_p=1，输入不裁剪，输出上限 512。实际输入 694-794 token，最长输出 225 token。C1 先运行，随后 C8；没有新增 no-spec 或 16K synthetic 测试。
|
||||
|
||||
| 指标 | C1 | C8 |
|
||||
|---|---:|---:|
|
||||
| 请求成功 / 计划 | 64/64 | 64/64 |
|
||||
| 答案检查通过 | 64/64 | 64/64 |
|
||||
| 排除与 5-shot 重叠的前 5 题后 | 59/59 | 59/59 |
|
||||
| 达到输出上限 | 0 | 0 |
|
||||
| 客户端总耗时 | 195.42 s | 55.55 s |
|
||||
| 总输出 token | 6523 | 6530 |
|
||||
| 整体 Output TPS | 33.38 | 117.56 |
|
||||
| 接口接受长度，每题等权平均 | 6.9433 | 7.0070 |
|
||||
| 直方图接受长度，按 verify 步数加权 | **6.4211** | **6.5158** |
|
||||
| Draft 接受率，不含 bonus | 36.14% | 36.77% |
|
||||
| Verify 步数 | 1026 | 1012 |
|
||||
| 接受 / 提议 draft token | 5562 / 15390 | 5582 / 15180 |
|
||||
| 请求回撤 | 0 | 0 |
|
||||
|
||||
**这套 PP8 + PD + DFlash 已能完成本轮部署回归，接受长度没有停留在接近 1 的状态。** C8 的 117.56 token/s 是整体输出吞吐，包含 Prefill、传输、调度及 Decode，不是单请求 Decode 速度。本轮未重跑匹配的 no-spec 对照，不据此宣称投机加速倍数。
|
||||
|
||||
历史客户端的 5-shot 取自同一测试文件前 5 题，与这 64 题重叠，保留此口径用于历史部署回归。剩余 59 题也全部通过；本报告不将此结果作为独立无泄漏的模型能力评测。所有输出均包含 `####` 最终答案，已独立按带符号数值重新解析并与标签比对。
|
||||
|
||||
### 接受长度为什么有两种数值
|
||||
|
||||
接口 `spec_accept_length` 在 `tokenizer_manager.py:2814` 定义为 `completion_tokens / spec_verify_ct`，受 P 侧首 token 和最后一步停止截断影响。例如一题输出 49 token、验证 3 次，接口值是 16.33；实际直方图为 3 次各接受 15 个 draft，含 bonus 的每步长度为 16，没有超出 block 能力。
|
||||
|
||||
```text
|
||||
加权接受长度 = 1 + 总接受 draft token / 总 verify 次数
|
||||
C1 = 1 + 5562 / 1026 = 6.4211
|
||||
C8 = 1 + 5582 / 1012 = 6.5158
|
||||
```
|
||||
|
||||
| 回归 | 数量 | 主要覆盖 |
|
||||
|---|---:|---|
|
||||
| 传输 | 7 | 144 组 KV-head/TP 组合、实际 P-TP4→D-TP32 全 32 rank、逐字节复制、越界/重复写拒绝、注册往返 |
|
||||
| PP 与 PD 输入 | 13 | PP1/2/4/8/16 capture 归属、边界 residual、空 capture stage、BF16/FP32 分段投影、单次 norm、末 stage 写 KV、输入生命周期、P 普通预热不创建 verify metadata、混合池/普通池传输层范围 |
|
||||
| Kimi MoE 合并 | 4 | 两类 gate/up 布局、SiTU 参数、activation 白名单、非连续输入、保留 SwigluStep、API 能力检查 |
|
||||
128 题逐一验证了 `sum(histogram)=verify_ct`、直方图加权计数等于 `accepted_drafts`、`proposed_drafts=15×verify_ct`。接口原值完整保留。客户端是非流式，不由总耗时推造 TTFT/ITL。
|
||||
|
||||
24 项在本地与 601 `kvbounds1` 镜像中通过。此前 `pwarm1` 镜像的 23 项也在 602 导入容器中通过，并完成真实 DFlashWorkerV2、MooncakeKVManager 导入。8 条启动命令的 CLI 解析及两类服务的参数后处理已检查。CPU 测试对实际方法作 AST 提取，使用 CPU tensor 或记录型 engine，核验数学与调用契约，不替代服务级验证。
|
||||
## 4. 验证与运行状态
|
||||
|
||||
601 GPU6 的 `test_kimi_k3_sm120_situ_layout_and_noncontiguous_input` 已通过：检查 Kimi gate/up 及 scale 布局、SiTU(4,25)、非连续输入，以及 SGLang adapter 与直接 FlashInfer 调用的输出一致性。这是小 shape 的集成回归；端到端正确性由后续 PD/GSM8K 检验。
|
||||
- 24 项 CPU 回归通过：13 项 PP/PD、7 项传输、4 项 Kimi MoE，覆盖 PP1/2/4/8/16、BF16/FP32 分段投影、TP4→TP32 全部 GQA head 映射、越界拒绝及输入生命周期。
|
||||
- 601 GPU6 的 Kimi SM120 SiTU 布局与非连续输入集成回归通过；固化镜像独立 GPU 复测 `1 passed`。CPU 回归使用实际方法的 AST 提取与 CPU tensor，不是完整模型 logits 等价测试。
|
||||
- Router smoke 和正式 C1/C8 全部完成。D 日志有 RDMA ready ACK 与实际 Decode CUDA Graph 记录。
|
||||
- 本 Run 服务日志未检出 Traceback、CUDA OOM、KVTransferError 或非法显存访问。存在 NUMA 查询权限警告和 K3 fused KDA decode 布局回退提示，未阻断本轮；后续性能调优可单独审查。
|
||||
- 运行中显存快照：P 每卡 53361-76579 MiB，D 每卡 79849-80019 MiB。这是离散快照，不是全程峰值。
|
||||
- 续跑和清理均退出 0。601-608 本实验容器已删除，GPU compute process 为空，显存回落到 0-3 MiB。
|
||||
|
||||
修复前镜像 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460` 的独立 GPU 复测通过，Mooncake CUDA engine 导入成功，GPU 测试 1 passed、25.28 秒（复用 JIT 缓存）。
|
||||
## 5. 如何运行
|
||||
|
||||
当前镜像为 `local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1`，在 `pwarm1` 上只替换 Python 层范围处理及回归测试，不改变 FlashInfer 层或 GPU kernel。Linux/amd64 manifest 为 `sha256:e74c9b1e7fb6a9509db32576d2c9196b42ef329118909eeebd620cb9db2f91e9`，image config 为 `sha256:26584138eb1e097183ce167cbffe4198be0611e0cfde7422cbe692100d865982`；OCI index 为 `sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f`。区分这三个摘要，不将 Docker 不同模式显示的 ID 当成代码不一致。
|
||||
|
||||
601 证据均位于实验目录 `results/`：
|
||||
|
||||
- `pp_and_transport_cpu_601_20260831.log`：18 项 CPU 回归。
|
||||
- `patched_worker_import_20260831.log`、`patched_launch_help_20260831.log`：真实模块导入与 CLI。
|
||||
- `image_build_situ4460_cpucheck_20260831.log`：集成镜像的完整 API/回归检查。
|
||||
- `command_parse_20260831.log`：P/D 全部 8 条命令通过 `ServerArgs` argparse 检查；该检查不执行 ServerArgs 后处理或服务初始化。
|
||||
- `args_resolve_20260831.log`、`resolved_p_20260831.json`、`resolved_d_20260831.json`：P/D 真实模型配置通过 `resolve_once()` 后处理；保持 BF16 KV、8K chunk 和 FlashInfer。Kimi 投机验证自动选择 `nv_cutedsl`，P 侧 PP 自动关闭 overlap scheduler；该检查不加载模型权重。
|
||||
- `sm120_cutlass_compile_20260831.log`：SM120 预编译进度。首次构建被外部 Docker SIGKILL 终止，退出 137；Docker 事件为显式 kill，无 OOM 事件。已编译的对象文件保留在独立缓存。
|
||||
- `sm120_cutlass_compile_resume_20260831.log`：恢复后编译成功，`SM120_CUTLASS_BUILD_OK`，退出 0。
|
||||
- `gpu_situ_smoke_detached_20260831.log`：首次完整单测发现 SiTU activation 白名单遗漏；已修复并增加 CPU 回归。
|
||||
- `gpu_situ_smoke_fix_20260831.log`：修复后 GPU 回归通过，1 passed；首次 JIT 在内的总用时 771.10 秒。
|
||||
- `image_build_activation_fix_20260831.log`：固化修复后的镜像构建及 22 项 CPU 回归。
|
||||
- `final_image_gpu_verify_20260831.log`、`final_image_gpu_container_20260831.json`：最终镜像独立 GPU 复测、Mooncake 导入、镜像 ID 和挂载证据。
|
||||
- `pd-dflash-20260831-1617/logs/p_0.log` 至 `p_3.log`：首轮 P 预热报错原始证据；该 Run 未进入请求测试，退出码 1。
|
||||
- `image_build_pwarm1_20260831.log`：预热修复镜像构建及 23 项 CPU 回归。
|
||||
- `pwarm1_602_cpu.log`、`pwarm1_602_inspect.json`：约 49 MiB 增量包导入后的容器回归与平台摘要。
|
||||
- `pd-dflash-pwarm1-20260831-1652/logs/p_0.log`：P 预热完成后，在初始化传输管理器时发现 `HybridLinearKVPool.end_layer` 缺失；未进入 D 启动和请求测试。
|
||||
- `image_build_kvbounds1_20260831.log`、`kvbounds_image_context/`：复用原 FlashInfer 层、只修改 Python 的构建配方及 24 项回归。
|
||||
|
||||
已通过 compileall、Black 和 Ruff 的未定义变量/语法检查。
|
||||
|
||||
## 实验入口
|
||||
|
||||
代码位于 601 的 `/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation`。只运行 `deploy_pd_dflash.sh`，它负责 P/D 启动、健康检查、Router、smoke 和评测。`bench_gsm8k_acceptance.py` 沿用原客户端的 prompt 与请求设置，将结果标签改为 DFlash，并增加原始 `meta_info`、verify 次数和输出结束原因归档。
|
||||
唯一入口在 **601**；602-608 不需要手工再启动入口。配置见 [config.env](config.env)，请求由 [bench_gsm8k_acceptance.py](bench_gsm8k_acceptance.py) 生成。维护既有 GitLab 分支 `hzy-kimi3-pd-pp8-standard`。
|
||||
|
||||
```bash
|
||||
# 601：命令检查，不需要 GPU 或 sudo
|
||||
cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation
|
||||
DRY_RUN=1 RUN_ID=command-audit-20260831 bash deploy_pd_dflash.sh all
|
||||
# 展开命令，不使用 GPU
|
||||
DRY_RUN=1 RUN_ID=pd-dflash-command-audit bash deploy_pd_dflash.sh all
|
||||
# 新实验：八节点空闲后，入口负责启动、smoke、C1/C8、归档及清理
|
||||
RUN_ID=pd-dflash-$(date +%Y%m%d-%H%M%S) bash deploy_pd_dflash.sh all
|
||||
```
|
||||
|
||||
此前两轮均在 P 侧启动阶段退出，日志已保存、失败容器已清理。修复镜像同步完成后使用新 Run ID 重试。正式顺序为 `start` → `smoke` → `bench` → `logs` → `stop`，也可用 `all` 串行执行。所有操作使用同一个 `RUN_ID`。失败时保留本任务容器和日志，入口不会自动杀其他实验。
|
||||
本次实际分两段执行，使用同一个 Run ID：
|
||||
|
||||
配置选择：P 为 TP4/PP8/EP4，D 为 TP32/PP1/EP4，两侧 FlashInfer；KV 为 BF16，chunk=8192，page=64。为 C1/C8 评测将活跃请求与 Decode Graph 上限设为 8；未额外改变模型 context 上限。
|
||||
```bash
|
||||
RUN_ID=pd-dflash-kvbounds1-20260831-1740 bash deploy_pd_dflash.sh all
|
||||
# 服务和 smoke 成功后，客户端数据集路径报错。修正挂载，不重启服务：
|
||||
RUN_ID=pd-dflash-kvbounds1-20260831-1740 bash deploy_pd_dflash.sh bench
|
||||
RUN_ID=pd-dflash-kvbounds1-20260831-1740 bash deploy_pd_dflash.sh logs
|
||||
RUN_ID=pd-dflash-kvbounds1-20260831-1740 bash deploy_pd_dflash.sh stop
|
||||
```
|
||||
|
||||
GSM8K 为原数据集前 64 题，沿用历史客户端的 5-shot、temperature=0、输出上限 512，分别运行 C1/C8。输入不裁剪。历史 5-shot 也取自测试集前 5 题，和本次 64 题有重叠：结果适用于与历史流程的部署回归，不作为独立无泄漏的模型准确率评估。逐题输出和截断情况保留。
|
||||
`bench` 只能复用已运行且配置匹配的服务；当前服务已清理，复测应使用新 Run ID 的 `all`。正式执行用 tmux 托管同一入口，没有新增 launch 脚本。
|
||||
|
||||
## 后续
|
||||
## 6. 原始结果与交付
|
||||
|
||||
确认 8 节点模型、镜像与网络一致，然后运行 C1 smoke 和固定 GSM8K 64 题 C1/C8。保留逐题答案、截断、接受直方图、总耗时、吞吐、原始 `meta_info` 与服务日志；不重跑 EAGLE3/no-spec 基线。当前客户端为非流式请求，不把客户端总耗时当作 TTFT 或 ITL。
|
||||
601 原始目录：
|
||||
|
||||
运行时设置 `SGLANG_CACHE_DIR=/cache`，FlashInfer、Triton、PyTorch 扩展和 CUDA 编译缓存也指向该挂载目录，`TMPDIR=/cache/tmp`，宿主路径见 `config.env` 的 `JIT_CACHE`，避免容器可写层占满根盘；预检会创建所需临时目录。
|
||||
```text
|
||||
/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/
|
||||
results/pd-dflash-kvbounds1-20260831-1740/
|
||||
bench/gsm8k_c1.jsonl # 逐题输出及原始 meta_info
|
||||
bench/gsm8k_c8.jsonl
|
||||
bench/gsm8k_c1_summary.json
|
||||
bench/gsm8k_c8_summary.json
|
||||
commands/ # 8 条 docker run、Router、2 条 bench
|
||||
logs/p_0.log ... p_3.log # P 四节点
|
||||
logs/d_0.log ... d_3.log # D 四节点
|
||||
logs/router.log
|
||||
metadata/post_run_resources.txt # 八节点清理核验
|
||||
result_validation.json # 独立答案/统计复核及证据 SHA256
|
||||
```
|
||||
|
||||
## 首轮启动修复
|
||||
本地副本：`/Users/hzy/Desktop/infra/docs/evidence/kimi_k3_pd_dflash/results/pd-dflash-kvbounds1-20260831-1740/`。轻量结果、完整服务日志与补丁随 GitLab 提交；模型与约 14GB OCI 镜像归档保留服务器，不纳入普通 Git。
|
||||
|
||||
`base_runner.py::_dummy_run` 已将 PD Prefill target 设为普通 Decode 预热，但随后仍创建 `DFlashVerifyInput`。普通 Triton Attention 读取 `kv_indptr` 时因此报 `AttributeError`。修复让该分支的 `spec_info=None`，D 侧真正的 TARGET_VERIFY 路径不变。没有强行添加字段，也没有修改 DFlash 接受算法。
|
||||
|
||||
第二轮通过预热后，`PrefillBootstrapQueue._init_kv_manager` 直接访问混合池不存在的 `end_layer`。修复以现有的完整注意力层偏移加 `full_layer_nums` 得到传输范围，KDA 状态和 draft KV 仍分别使用明确的全局层 ID。新增回归覆盖 MLA/MHA 混合池、空完整注意力 stage、普通池和 layer-shard 分支，不将 K/V 两组条目误计为两倍层数。
|
||||
|
||||
镜像同步须检查 Docker 所在根盘，而不只检查模型盘 `/data`。本镜像层展开约 34.4GB，压缩内容约 15GB；共享层会减少增量占用。607 已按用户授权删除两个无容器引用的 vLLM 镜像，根盘恢复到约 107GB，原镜像元数据保存在 `results/607_vllm_images_before_delete_20260831.json`。用户清理 606 后其根盘恢复到约 91GB。601 使用 `ctr images export` 将既有 OCI 压缩层直接导出到 `/data`，其旧 vLLM 镜像尚未删除。
|
||||
首段 `exit_code=1` 如实保留，对应容器内数据集 `../` 路径错误；修复后 `bench_resume_exit_code=0`、`cleanup_exit_code=0`。失败客户端命令和日志另存为 `gsm8k_c1.dataset_path_failure.*`。此前 P 预热和混合池层范围两次失败保留独立 Run，未混入本轮评测结果。
|
||||
|
||||
@ -210,9 +210,9 @@ bench() {
|
||||
local concurrency
|
||||
for concurrency in 1 8; do
|
||||
emit "gsm8k_c${concurrency}" "${P_NODES[0]}" docker run --rm --network host \
|
||||
-v "${GSM8K}:${GSM8K}:ro" -v "${BENCH_CLIENT}:/bench.py:ro" \
|
||||
-v "${GSM8K}:/dataset/gsm8k.jsonl:ro" -v "${BENCH_CLIENT}:/bench.py:ro" \
|
||||
-v "${RESULT_ROOT}:/results" --entrypoint python3 "$PD_IMAGE" /bench.py \
|
||||
--data-path "$GSM8K" --num-questions 64 --num-shots 5 \
|
||||
--data-path /dataset/gsm8k.jsonl --num-questions 64 --num-shots 5 \
|
||||
--max-new-tokens 512 --temperature 0 --top-p 1 --parallel "$concurrency" \
|
||||
--speculative-algorithm DFLASH \
|
||||
--host "${P_NODES[0]}" --port "$ROUTER_PORT" --backend srt \
|
||||
|
||||
@ -0,0 +1,31 @@
|
||||
/tmp/uv-kimi-pd-cache/archive-v0/JtVqGBlr4_hQgwWtLUFzX/lib/python3.12/site-packages/torch/_subclasses/functional_tensor.py:368: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)
|
||||
cpu = _conversion_method_template(device=torch.device("cpu"))
|
||||
test_capture_ownership_including_boundaries (test_dflash_pp_context.TestPPContext.test_capture_ownership_including_boundaries) ... ok
|
||||
test_invalid_capture_configuration (test_dflash_pp_context.TestPPContext.test_invalid_capture_configuration) ... ok
|
||||
test_kimi_boundary_uses_next_stage_weights (test_dflash_pp_context.TestPPContext.test_kimi_boundary_uses_next_stage_weights) ... ok
|
||||
test_last_stage_without_capture_uses_incoming_context (test_dflash_pp_context.TestPPContext.test_last_stage_without_capture_uses_incoming_context) ... ok
|
||||
test_missing_incoming_context_fails_before_kv_write (test_dflash_pp_context.TestPPContext.test_missing_incoming_context_fails_before_kv_write) ... ok
|
||||
test_nonfinal_pool_is_minimal_without_mutating_target_config (test_dflash_pp_context.TestPPContext.test_nonfinal_pool_is_minimal_without_mutating_target_config) ... ok
|
||||
test_only_prefill_skips_draft_graph_initialization (test_dflash_pp_context.TestPPContext.test_only_prefill_skips_draft_graph_initialization) ... ok
|
||||
test_pd_input_builder_and_spec_dispatch (test_dflash_pp_context.TestPPContext.test_pd_input_builder_and_spec_dispatch) ... ok
|
||||
test_pp8_boundary_capture (test_dflash_pp_context.TestPPContext.test_pp8_boundary_capture) ... ok
|
||||
test_pp8_projection_empty_stages_float32_and_bf16 (test_dflash_pp_context.TestPPContext.test_pp8_projection_empty_stages_float32_and_bf16) ... ok
|
||||
test_pp_proxy_is_consumed_once_even_on_forward_failure (test_dflash_pp_context.TestPPContext.test_pp_proxy_is_consumed_once_even_on_forward_failure) ... ok
|
||||
test_prefill_dummy_forward_has_no_verify_metadata (test_dflash_pp_context.TestPPContext.test_prefill_dummy_forward_has_no_verify_metadata) ... ok
|
||||
test_prefill_transfer_bounds_for_hybrid_and_ordinary_pools (test_dflash_pp_context.TestPPContext.test_prefill_transfer_bounds_for_hybrid_and_ordinary_pools) ... ok
|
||||
test_activation_selection_accepts_situ_and_legacy_enum (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_activation_selection_accepts_situ_and_legacy_enum) ... ok
|
||||
test_capability_requires_public_parameters (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_capability_requires_public_parameters) ... ok
|
||||
test_runner_preserves_swiglu_step_and_old_api (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_runner_preserves_swiglu_step_and_old_api) ... ok
|
||||
test_weight_layout_and_situ_parameters (test_flashinfer_kimi_merge.TestFlashInferKimiMerge.test_weight_layout_and_situ_parameters) ... ok
|
||||
test_all_gqa_shards_and_replicas_byte_exact (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_all_gqa_shards_and_replicas_byte_exact) ... ok
|
||||
test_flat_stride_mismatch_rejected (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_flat_stride_mismatch_rejected) ... ok
|
||||
test_invalid_metadata_rejected_before_copy (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_invalid_metadata_rejected_before_copy) ... ok
|
||||
test_mooncake_mixed_entries_match_layers_before_transfer (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_mooncake_mixed_entries_match_layers_before_transfer) ... ok
|
||||
test_real_kimi_dflash_page64_tp4_to_tp32 (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_real_kimi_dflash_page64_tp4_to_tp32) ... ok
|
||||
test_replicated_mla_is_flat_and_coalesced (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_replicated_mla_is_flat_and_coalesced) ... ok
|
||||
test_wire_roundtrip (test_mixed_kv_entry_layout.TestMixedKVEntryLayout.test_wire_roundtrip) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 24 tests in 0.424s
|
||||
|
||||
OK
|
||||
@ -0,0 +1,427 @@
|
||||
[
|
||||
{
|
||||
"Id": "ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9",
|
||||
"Created": "2026-08-31T06:09:44.850021061Z",
|
||||
"Path": "python3",
|
||||
"Args": [
|
||||
"-c",
|
||||
"import mooncake.engine; print(\"MOONCAKE_ENGINE_IMPORT_OK\", flush=True); import pytest; raise SystemExit(pytest.main([\"-v\", \"-s\", \"/test_kimi.py::test_kimi_k3_sm120_situ_layout_and_noncontiguous_input\"]))"
|
||||
],
|
||||
"State": {
|
||||
"Status": "exited",
|
||||
"Running": false,
|
||||
"Paused": false,
|
||||
"Restarting": false,
|
||||
"OOMKilled": false,
|
||||
"Dead": false,
|
||||
"Pid": 0,
|
||||
"ExitCode": 0,
|
||||
"Error": "",
|
||||
"StartedAt": "2026-08-31T06:09:44.909236209Z",
|
||||
"FinishedAt": "2026-08-31T06:10:13.37845969Z"
|
||||
},
|
||||
"Image": "sha256:6f998f448dd3545423e2bd73000c8bce991b4dbc9be66aff329b978b303c5987",
|
||||
"ResolvConfPath": "/var/lib/docker/containers/ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9/resolv.conf",
|
||||
"HostnamePath": "/var/lib/docker/containers/ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9/hostname",
|
||||
"HostsPath": "/var/lib/docker/containers/ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9/hosts",
|
||||
"LogPath": "/var/lib/docker/containers/ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9/ade3678d0340e6eb81b6149c3dd167f85690c009cb3c24f6b8ed61c99529a5e9-json.log",
|
||||
"Name": "/hzy_kimi_dflash_final_image_verify",
|
||||
"RestartCount": 0,
|
||||
"Driver": "overlayfs",
|
||||
"Platform": "linux",
|
||||
"MountLabel": "",
|
||||
"ProcessLabel": "",
|
||||
"AppArmorProfile": "docker-default",
|
||||
"ExecIDs": null,
|
||||
"HostConfig": {
|
||||
"Binds": [
|
||||
"/data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache",
|
||||
"/data/hzy/src/sglang-kimi-sm120-postsync-wt/test/registered/unit/layers/quantization/test_mxfp4_sm120_cutlass.py:/test_kimi.py:ro"
|
||||
],
|
||||
"ContainerIDFile": "",
|
||||
"LogConfig": {
|
||||
"Type": "json-file",
|
||||
"Config": {}
|
||||
},
|
||||
"NetworkMode": "none",
|
||||
"PortBindings": {},
|
||||
"RestartPolicy": {
|
||||
"Name": "no",
|
||||
"MaximumRetryCount": 0
|
||||
},
|
||||
"AutoRemove": false,
|
||||
"VolumeDriver": "",
|
||||
"VolumesFrom": null,
|
||||
"ConsoleSize": [
|
||||
0,
|
||||
0
|
||||
],
|
||||
"CapAdd": null,
|
||||
"CapDrop": null,
|
||||
"CgroupnsMode": "private",
|
||||
"Dns": null,
|
||||
"DnsOptions": [],
|
||||
"DnsSearch": [],
|
||||
"ExtraHosts": null,
|
||||
"GroupAdd": null,
|
||||
"IpcMode": "host",
|
||||
"Cgroup": "",
|
||||
"Links": null,
|
||||
"OomScoreAdj": 0,
|
||||
"PidMode": "",
|
||||
"Privileged": false,
|
||||
"PublishAllPorts": false,
|
||||
"ReadonlyRootfs": false,
|
||||
"SecurityOpt": [
|
||||
"label=disable"
|
||||
],
|
||||
"UTSMode": "",
|
||||
"UsernsMode": "",
|
||||
"ShmSize": 67108864,
|
||||
"Runtime": "runc",
|
||||
"Isolation": "",
|
||||
"CpuShares": 0,
|
||||
"Memory": 0,
|
||||
"NanoCpus": 0,
|
||||
"CgroupParent": "",
|
||||
"BlkioWeight": 0,
|
||||
"BlkioWeightDevice": [],
|
||||
"BlkioDeviceReadBps": [],
|
||||
"BlkioDeviceWriteBps": [],
|
||||
"BlkioDeviceReadIOps": [],
|
||||
"BlkioDeviceWriteIOps": [],
|
||||
"CpuPeriod": 0,
|
||||
"CpuQuota": 0,
|
||||
"CpuRealtimePeriod": 0,
|
||||
"CpuRealtimeRuntime": 0,
|
||||
"CpusetCpus": "",
|
||||
"CpusetMems": "",
|
||||
"Devices": [],
|
||||
"DeviceCgroupRules": null,
|
||||
"DeviceRequests": [
|
||||
{
|
||||
"Driver": "",
|
||||
"Count": 0,
|
||||
"DeviceIDs": [
|
||||
"6"
|
||||
],
|
||||
"Capabilities": [
|
||||
[
|
||||
"gpu"
|
||||
]
|
||||
],
|
||||
"Options": {}
|
||||
}
|
||||
],
|
||||
"MemoryReservation": 0,
|
||||
"MemorySwap": 0,
|
||||
"MemorySwappiness": null,
|
||||
"OomKillDisable": null,
|
||||
"PidsLimit": null,
|
||||
"Ulimits": [],
|
||||
"CpuCount": 0,
|
||||
"CpuPercent": 0,
|
||||
"IOMaximumIOps": 0,
|
||||
"IOMaximumBandwidth": 0,
|
||||
"MaskedPaths": [
|
||||
"/proc/acpi",
|
||||
"/proc/asound",
|
||||
"/proc/interrupts",
|
||||
"/proc/kcore",
|
||||
"/proc/keys",
|
||||
"/proc/latency_stats",
|
||||
"/proc/sched_debug",
|
||||
"/proc/scsi",
|
||||
"/proc/timer_list",
|
||||
"/proc/timer_stats",
|
||||
"/sys/devices/virtual/powercap",
|
||||
"/sys/firmware",
|
||||
"/sys/devices/system/cpu/cpu0/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu1/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu2/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu3/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu4/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu5/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu6/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu7/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu8/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu9/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu10/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu11/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu12/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu13/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu14/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu15/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu16/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu17/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu18/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu19/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu20/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu21/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu22/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu23/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu24/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu25/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu26/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu27/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu28/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu29/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu30/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu31/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu32/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu33/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu34/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu35/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu36/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu37/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu38/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu39/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu40/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu41/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu42/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu43/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu44/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu45/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu46/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu47/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu48/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu49/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu50/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu51/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu52/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu53/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu54/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu55/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu56/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu57/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu58/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu59/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu60/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu61/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu62/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu63/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu64/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu65/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu66/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu67/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu68/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu69/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu70/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu71/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu72/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu73/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu74/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu75/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu76/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu77/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu78/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu79/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu80/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu81/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu82/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu83/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu84/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu85/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu86/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu87/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu88/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu89/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu90/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu91/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu92/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu93/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu94/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu95/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu96/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu97/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu98/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu99/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu100/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu101/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu102/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu103/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu104/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu105/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu106/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu107/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu108/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu109/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu110/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu111/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu112/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu113/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu114/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu115/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu116/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu117/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu118/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu119/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu120/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu121/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu122/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu123/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu124/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu125/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu126/thermal_throttle",
|
||||
"/sys/devices/system/cpu/cpu127/thermal_throttle"
|
||||
],
|
||||
"ReadonlyPaths": [
|
||||
"/proc/bus",
|
||||
"/proc/fs",
|
||||
"/proc/irq",
|
||||
"/proc/sys",
|
||||
"/proc/sysrq-trigger"
|
||||
]
|
||||
},
|
||||
"Storage": {
|
||||
"RootFS": {
|
||||
"Snapshot": {
|
||||
"Name": "overlayfs"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Mounts": [
|
||||
{
|
||||
"Type": "bind",
|
||||
"Source": "/data/hzy/cache/kimi-dflash-fi0618-situ4460",
|
||||
"Destination": "/cache",
|
||||
"Mode": "",
|
||||
"RW": true,
|
||||
"Propagation": "rprivate"
|
||||
},
|
||||
{
|
||||
"Type": "bind",
|
||||
"Source": "/data/hzy/src/sglang-kimi-sm120-postsync-wt/test/registered/unit/layers/quantization/test_mxfp4_sm120_cutlass.py",
|
||||
"Destination": "/test_kimi.py",
|
||||
"Mode": "ro",
|
||||
"RW": false,
|
||||
"Propagation": "rprivate"
|
||||
}
|
||||
],
|
||||
"Config": {
|
||||
"Hostname": "ade3678d0340",
|
||||
"Domainname": "",
|
||||
"User": "",
|
||||
"AttachStdin": false,
|
||||
"AttachStdout": false,
|
||||
"AttachStderr": false,
|
||||
"Tty": false,
|
||||
"OpenStdin": false,
|
||||
"StdinOnce": false,
|
||||
"Env": [
|
||||
"FLASHINFER_CUDA_ARCH_LIST=12.0f",
|
||||
"MAX_JOBS=8",
|
||||
"PYTHONUNBUFFERED=1",
|
||||
"FLASHINFER_WORKSPACE_BASE=/cache",
|
||||
"PATH=/root/.cargo/bin:/opt/sglang/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/nvidia/bin",
|
||||
"NVARCH=x86_64",
|
||||
"NVIDIA_REQUIRE_CUDA=cuda>=13.0 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 brand=unknown,driver>=565,driver<566 brand=grid,driver>=565,driver<566 brand=tesla,driver>=565,driver<566 brand=nvidia,driver>=565,driver<566 brand=quadro,driver>=565,driver<566 brand=quadrortx,driver>=565,driver<566 brand=nvidiartx,driver>=565,driver<566 brand=vapps,driver>=565,driver<566 brand=vpc,driver>=565,driver<566 brand=vcs,driver>=565,driver<566 brand=vws,driver>=565,driver<566 brand=cloudgaming,driver>=565,driver<566 brand=unknown,driver>=570,driver<571 brand=grid,driver>=570,driver<571 brand=tesla,driver>=570,driver<571 brand=nvidia,driver>=570,driver<571 brand=quadro,driver>=570,driver<571 brand=quadrortx,driver>=570,driver<571 brand=nvidiartx,driver>=570,driver<571 brand=vapps,driver>=570,driver<571 brand=vpc,driver>=570,driver<571 brand=vcs,driver>=570,driver<571 brand=vws,driver>=570,driver<571 brand=cloudgaming,driver>=570,driver<571 brand=unknown,driver>=575,driver<576 brand=grid,driver>=575,driver<576 brand=tesla,driver>=575,driver<576 brand=nvidia,driver>=575,driver<576 brand=quadro,driver>=575,driver<576 brand=quadrortx,driver>=575,driver<576 brand=nvidiartx,driver>=575,driver<576 brand=vapps,driver>=575,driver<576 brand=vpc,driver>=575,driver<576 brand=vcs,driver>=575,driver<576 brand=vws,driver>=575,driver<576 brand=cloudgaming,driver>=575,driver<576",
|
||||
"NV_CUDA_CUDART_VERSION=13.0.96-1",
|
||||
"CUDA_VERSION=13.0.3",
|
||||
"LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64",
|
||||
"NVIDIA_VISIBLE_DEVICES=all",
|
||||
"NVIDIA_DRIVER_CAPABILITIES=compute,utility",
|
||||
"NV_CUDA_LIB_VERSION=13.0.3-1",
|
||||
"NV_NVTX_VERSION=13.0.85-1",
|
||||
"NV_LIBNPP_VERSION=13.0.1.2-1",
|
||||
"NV_LIBNPP_PACKAGE=libnpp-13-0=13.0.1.2-1",
|
||||
"NV_LIBCUSPARSE_VERSION=12.6.3.3-1",
|
||||
"NV_LIBCUBLAS_PACKAGE_NAME=libcublas-13-0",
|
||||
"NV_LIBCUBLAS_VERSION=13.1.1.3-1",
|
||||
"NV_LIBCUBLAS_PACKAGE=libcublas-13-0=13.1.1.3-1",
|
||||
"NV_LIBNCCL_PACKAGE_NAME=libnccl2",
|
||||
"NV_LIBNCCL_PACKAGE_VERSION=2.28.3-1",
|
||||
"NCCL_VERSION=2.28.3-1",
|
||||
"NV_LIBNCCL_PACKAGE=libnccl2=2.28.3-1+cuda13.0",
|
||||
"NVIDIA_PRODUCT_NAME=CUDA",
|
||||
"NV_CUDA_CUDART_DEV_VERSION=13.0.96-1",
|
||||
"NV_NVML_DEV_VERSION=13.0.87-1",
|
||||
"NV_LIBCUSPARSE_DEV_VERSION=12.6.3.3-1",
|
||||
"NV_LIBNPP_DEV_VERSION=13.0.1.2-1",
|
||||
"NV_LIBNPP_DEV_PACKAGE=libnpp-dev-13-0=13.0.1.2-1",
|
||||
"NV_LIBCUBLAS_DEV_VERSION=13.1.1.3-1",
|
||||
"NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-13-0",
|
||||
"NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-13-0=13.1.1.3-1",
|
||||
"NV_CUDA_NSIGHT_COMPUTE_VERSION=13.0.3-1",
|
||||
"NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-13-0=13.0.3-1",
|
||||
"NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev",
|
||||
"NV_LIBNCCL_DEV_PACKAGE_VERSION=2.28.3-1",
|
||||
"NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.28.3-1+cuda13.0",
|
||||
"LIBRARY_PATH=/usr/local/cuda/lib64/stubs",
|
||||
"NV_CUDNN_VERSION=9.14.0.64-1",
|
||||
"NV_CUDNN_PACKAGE_NAME=libcudnn9-cuda-13",
|
||||
"NV_CUDNN_PACKAGE=libcudnn9-cuda-13=9.14.0.64-1",
|
||||
"NV_CUDNN_PACKAGE_DEV=libcudnn9-dev-cuda-13=9.14.0.64-1",
|
||||
"NV_CUDNN_PACKAGE_DEV_HEADERS=libcudnn9-headers-cuda-13=9.14.0.64-1",
|
||||
"DEBIAN_FRONTEND=noninteractive",
|
||||
"CUDA_HOME=/usr/local/cuda",
|
||||
"GDRCOPY_HOME=/usr/src/gdrdrv-2.5.1/",
|
||||
"FLASHINFER_VERSION=0.6.17",
|
||||
"LANG=en_US.UTF-8",
|
||||
"LANGUAGE=en_US:en",
|
||||
"LC_ALL=en_US.UTF-8",
|
||||
"SGLANG_BUILD_COMMIT=daf631719690e18d13f67a20eb513fd48c712327",
|
||||
"SGLANG_BUILD_URL=https://github.com/sgl-project/sglang/actions/runs/33137164725",
|
||||
"SGLANG_IMAGE_TAG=lmsysorg/sglang:nightly-dev-20260828-daf63171",
|
||||
"PYTHONPATH=/opt/kimi-dflash/python",
|
||||
"PYTHONDONTWRITEBYTECODE=1",
|
||||
"FLASHINFER_DISABLE_VERSION_CHECK=",
|
||||
"SGLANG_SOURCE_ROOT=/opt/kimi-dflash"
|
||||
],
|
||||
"Cmd": [
|
||||
"-c",
|
||||
"import mooncake.engine; print(\"MOONCAKE_ENGINE_IMPORT_OK\", flush=True); import pytest; raise SystemExit(pytest.main([\"-v\", \"-s\", \"/test_kimi.py::test_kimi_k3_sm120_situ_layout_and_noncontiguous_input\"]))"
|
||||
],
|
||||
"Image": "local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460",
|
||||
"Volumes": null,
|
||||
"WorkingDir": "/sgl-workspace/sglang",
|
||||
"Entrypoint": [
|
||||
"python3"
|
||||
],
|
||||
"Labels": {
|
||||
"ai.sglang.build.commit": "daf631719690e18d13f67a20eb513fd48c712327",
|
||||
"ai.sglang.build.url": "https://github.com/sgl-project/sglang/actions/runs/33137164725",
|
||||
"ai.sglang.image.tag": "lmsysorg/sglang:nightly-dev-20260828-daf63171",
|
||||
"com.nvidia.cudnn.version": "9.14.0.64-1",
|
||||
"maintainer": "NVIDIA CORPORATION <cudatools@nvidia.com>",
|
||||
"org.opencontainers.image.revision": "daf631719690e18d13f67a20eb513fd48c712327",
|
||||
"org.opencontainers.image.source": "https://github.com/sgl-project/sglang",
|
||||
"org.opencontainers.image.url": "https://github.com/sgl-project/sglang/actions/runs/33137164725",
|
||||
"org.opencontainers.image.version": "lmsysorg/sglang:nightly-dev-20260828-daf63171"
|
||||
}
|
||||
},
|
||||
"NetworkSettings": {
|
||||
"SandboxID": "",
|
||||
"SandboxKey": "",
|
||||
"Ports": {},
|
||||
"Networks": {
|
||||
"none": {
|
||||
"IPAMConfig": null,
|
||||
"Links": null,
|
||||
"Aliases": null,
|
||||
"DriverOpts": null,
|
||||
"GwPriority": 0,
|
||||
"NetworkID": "13a7d53fe7d1eb2261fa2476b4a017a554e9d1341e28d1fa6b6b39c57231ed2d",
|
||||
"EndpointID": "",
|
||||
"Gateway": "",
|
||||
"IPAddress": "",
|
||||
"MacAddress": "",
|
||||
"IPPrefixLen": 0,
|
||||
"IPv6Gateway": "",
|
||||
"GlobalIPv6Address": "",
|
||||
"GlobalIPv6PrefixLen": 0,
|
||||
"DNSNames": null
|
||||
}
|
||||
}
|
||||
},
|
||||
"ImageManifestDescriptor": {
|
||||
"mediaType": "application/vnd.oci.image.manifest.v1+json",
|
||||
"digest": "sha256:14dc45076169180f94715109022fa5ba8116bdc68b5394ebf9c1801640aa04aa",
|
||||
"size": 15020,
|
||||
"platform": {
|
||||
"architecture": "amd64",
|
||||
"os": "linux"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
@ -0,0 +1,18 @@
|
||||
MOONCAKE_ENGINE_IMPORT_OK
|
||||
============================= test session starts ==============================
|
||||
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0 -- /opt/sglang/bin/python3
|
||||
cachedir: .pytest_cache
|
||||
rootdir: /
|
||||
plugins: anyio-4.14.2, typeguard-4.6.0
|
||||
collecting ... collected 1 item
|
||||
|
||||
../../test_kimi.py::test_kimi_k3_sm120_situ_layout_and_noncontiguous_input [TensorRT-LLM][INFO] Set logger level to INFO
|
||||
PASSED
|
||||
|
||||
=============================== warnings summary ===============================
|
||||
test_kimi.py: 14 warnings
|
||||
/opt/sglang/lib/python3.12/site-packages/torch/jit/_script.py:365: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
|
||||
warnings.warn(
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
======================= 1 passed, 14 warnings in 25.28s ========================
|
||||
@ -0,0 +1,16 @@
|
||||
gsm8k_c1 174.1.60.1: docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 1 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c1.jsonl --summary-file /results/bench/gsm8k_c1_summary.json
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/awq/awq.py:53: UserWarning: Only CUDA, HIP and XPU support AWQ currently.
|
||||
warnings.warn(f"Only CUDA, HIP and XPU support AWQ currently.")
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/gguf.py:69: UserWarning: Only CUDA, MUSA and NPU support GGUF quantization currently.
|
||||
warnings.warn(f"Only CUDA, MUSA and NPU support GGUF quantization currently.")
|
||||
Traceback (most recent call last):
|
||||
File "/bench.py", line 204, in <module>
|
||||
main(args)
|
||||
File "/bench.py", line 57, in main
|
||||
lines = list(read_jsonl(args.data_path))
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/opt/kimi-dflash/python/sglang/utils.py", line 137, in read_jsonl
|
||||
with open(filename) as fin:
|
||||
^^^^^^^^^^^^^^
|
||||
FileNotFoundError: [Errno 2] No such file or directory: '/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl'
|
||||
FAILED rc=1; logs: /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740 (containers retained for diagnosis)
|
||||
@ -0,0 +1,64 @@
|
||||
{"prompt_id": 0, "output": " Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.\nShe makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market.\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 14.0, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 60, "spec_proposed_drafts": 60, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "spec_verify_ct": 4, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "37f7d04630254733bbc17eb3d897dbad", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 730, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170792.4341302, "queue_time": 0.0005555152893066406, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.0575672248378396, "spec_accept_rate": 1.0, "spec_accept_length": 14.0, "spec_num_correct_drafts": 60, "spec_num_proposed_drafts": 60, "spec_verify_ct": 4, "spec_accepted_drafts": 60, "spec_proposed_drafts": 60, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "request_received_ts": 1788170790.7995293, "api_server_dispatch_finish_ts": 1788170790.8062823, "request_finished_ts": 1788170792.8570967, "decode_throughput": 9868950.588235294, "response_sent_to_client_ts": 1788170792.857933}}
|
||||
{"prompt_id": 1, "output": " It takes 2/2=<<2/2=1>>1 bolt of white fiber\nSo the total amount of fabric is 2+1=<<2+1=3>>3 bolts of fabric\n#### 3\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 16.333333333333332, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 45, "spec_proposed_drafts": 45, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "spec_verify_ct": 3, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ad3037fbf5a44d47adf9454dec639530", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 694, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170794.2979336, "queue_time": 0.00025272369384765625, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 1.7520954250358045, "spec_accept_rate": 1.0, "spec_accept_length": 16.333333333333332, "spec_num_correct_drafts": 45, "spec_num_proposed_drafts": 45, "spec_verify_ct": 3, "spec_accepted_drafts": 45, "spec_proposed_drafts": 45, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "request_received_ts": 1788170792.8703833, "api_server_dispatch_finish_ts": 1788170792.873931, "request_finished_ts": 1788170794.6224787, "decode_throughput": 12269874.432091417, "response_sent_to_client_ts": 1788170794.6230125}}
|
||||
{"prompt_id": 2, "output": " The cost of the house and repairs came out to 80,000+50,000=$<<80000+50000=130000>>130,000\nHe increased the value of the house by 80,000*1.5=<<80000*1.5=120000>>120,000\nSo the new value of the house is 120,000+80,000=$<<120000+80000=200000>>200,000\nSo he made a profit of 200,000-130,000=$<<200000-130000=70000>>70,000\n#### 70000\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 14.555555555555555, "spec_accept_length_from_histogram": 14.666666666666666, "spec_accept_rate": 0.9111111111111111, "spec_accepted_drafts": 123, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c5133d42fece4d9ba2525ced4167d296", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170796.1166427, "queue_time": 0.0003952980041503906, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.386726868804544, "spec_accept_rate": 0.9111111111111111, "spec_accept_length": 14.555555555555555, "spec_num_correct_drafts": 123, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 123, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "request_received_ts": 1788170794.6340747, "api_server_dispatch_finish_ts": 1788170794.6374896, "request_finished_ts": 1788170797.0208015, "decode_throughput": 31081370.99087063, "response_sent_to_client_ts": 1788170797.0213184}}
|
||||
{"prompt_id": 3, "output": " He sprints 3*3=<<3*3=9>>9 times\nSo he runs 9*60=<<9*60=540>>540 meters\n#### 540\n\n", "correct": true, "error": null, "completion_tokens": 41, "spec_accept_length": 13.666666666666666, "spec_accept_length_from_histogram": 14.0, "spec_accept_rate": 0.8666666666666667, "spec_accepted_drafts": 39, "spec_proposed_drafts": 45, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "spec_verify_ct": 3, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "eab08018cad64015aa70846692653714", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 702, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170798.4501424, "queue_time": 0.0004611015319824219, "reasoning_tokens": 0, "completion_tokens": 41, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 41}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 1.7492218469269574, "spec_accept_rate": 0.8666666666666667, "spec_accept_length": 13.666666666666666, "spec_num_correct_drafts": 39, "spec_num_proposed_drafts": 45, "spec_verify_ct": 3, "spec_accepted_drafts": 39, "spec_proposed_drafts": 45, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "request_received_ts": 1788170797.031961, "api_server_dispatch_finish_ts": 1788170797.0353174, "request_finished_ts": 1788170798.7811828, "decode_throughput": 9854232.63966961, "response_sent_to_client_ts": 1788170798.781778}}
|
||||
{"prompt_id": 4, "output": " If each chicken eats 3 cups of feed per day, then for 20 chickens they would need 3*20=<<3*20=60>>60 cups of feed per day.\nIf she feeds the flock 15 cups of feed in the morning, and 25 cups in the afternoon, then the final meal would require 60-15-25=<<60-15-25=20>>20 cups of chicken feed.\n#### 20\n\n", "correct": true, "error": null, "completion_tokens": 94, "spec_accept_length": 15.666666666666666, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 90, "spec_proposed_drafts": 90, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "spec_verify_ct": 6, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3a2f9c8c906a4f3a82f133d1decf27f8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 775, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170800.3156128, "queue_time": 0.0003247261047363281, "reasoning_tokens": 0, "completion_tokens": 94, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 94}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.1533947782590985, "spec_accept_rate": 1.0, "spec_accept_length": 15.666666666666666, "spec_num_correct_drafts": 90, "spec_num_proposed_drafts": 90, "spec_verify_ct": 6, "spec_accepted_drafts": 90, "spec_proposed_drafts": 90, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "request_received_ts": 1788170798.7940645, "api_server_dispatch_finish_ts": 1788170798.7976167, "request_finished_ts": 1788170800.9474595, "decode_throughput": 22012121.59858922, "response_sent_to_client_ts": 1788170800.9479864}}
|
||||
{"prompt_id": 5, "output": " The discount price of one glass is 60/100 * 5 = $<<60/100*5=3>>3.\nIf every second glass is cheaper, that means Kylar is going to buy 16 / 2 = <<16/2=8>>8 cheaper glasses.\nSo for the cheaper glasses, Kylar is going to pay 8 * 3 = $<<8*3=24>>24.\nAnd for the regular-priced glasses, Kylar will pay 8 * 5 = $<<8*5=40>>40.\nSo in total Kylar needs to pay 24 + 40 = $<<24+40=64>>64 for the glasses he wants to buy.\n#### 64\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 7.142857142857143, "spec_accept_length_from_histogram": 7.190476190476191, "spec_accept_rate": 0.4126984126984127, "spec_accepted_drafts": 130, "spec_proposed_drafts": 315, "spec_accept_histogram": [0, 0, 3, 4, 4, 0, 1, 1, 2, 0, 2, 1, 3], "spec_verify_ct": 21, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "9f60e0e725b34cdbba4207dc3456bd5c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 720, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170802.37141, "queue_time": 0.00030422210693359375, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.544350159354508, "spec_accept_rate": 0.4126984126984127, "spec_accept_length": 7.142857142857143, "spec_num_correct_drafts": 130, "spec_num_proposed_drafts": 315, "spec_verify_ct": 21, "spec_accepted_drafts": 130, "spec_proposed_drafts": 315, "spec_correct_drafts_histogram": [0, 0, 3, 4, 4, 0, 1, 1, 2, 0, 2, 1, 3], "spec_accept_histogram": [0, 0, 3, 4, 4, 0, 1, 1, 2, 0, 2, 1, 3], "request_received_ts": 1788170800.9583046, "api_server_dispatch_finish_ts": 1788170800.961818, "request_finished_ts": 1788170804.5026548, "decode_throughput": 68856265.02087368, "response_sent_to_client_ts": 1788170804.5029614}}
|
||||
{"prompt_id": 6, "output": " If Seattle has 20 sheep, Charleston has 4 * 20 sheep = <<20*4=80>>80 sheep\nToulouse has twice as many sheep as Charleston, which is 2 * 80 sheep = <<2*80=160>>160 sheep\nTogether, the three has 20 sheep + 160 sheep + 80 sheep = <<20+160+80=260>>260 sheep\n#### 260\n\n", "correct": true, "error": null, "completion_tokens": 90, "spec_accept_length": 6.428571428571429, "spec_accept_length_from_histogram": 6.428571428571429, "spec_accept_rate": 0.3619047619047619, "spec_accepted_drafts": 76, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "67473c8a9abe4ca8bec80fcefac1cae2", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170805.9356039, "queue_time": 0.00034356117248535156, "reasoning_tokens": 0, "completion_tokens": 90, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 90}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.8471194826997817, "spec_accept_rate": 0.3619047619047619, "spec_accept_length": 6.428571428571429, "spec_num_correct_drafts": 76, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 76, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788170804.5125027, "api_server_dispatch_finish_ts": 1788170804.514273, "request_finished_ts": 1788170807.359622, "decode_throughput": 44928548.34790785, "response_sent_to_client_ts": 1788170807.359861}}
|
||||
{"prompt_id": 7, "output": " First find how many gigabytes are in 40% of the file: 200 GB * 40% = <<200*40*.01=80>>80 GB\nThen divide that number by the download rate to find the time until Windows restarts: 80 GB / 2 GB/minute = <<80/2=40>>40 minutes\nThen find the time to download the whole file after the restart: 200 GB / 2 GB/minute = <<200/2=100>>100 minutes\nThen add the time to download 40% of the file, to download the whole file, and to wait for Windows to update: 40 minutes + 100 minutes + 20 minutes = <<40+100+20=160>>160 minutes\n#### 160\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.1923076923076925, "spec_accept_length_from_histogram": 6.1923076923076925, "spec_accept_rate": 0.34615384615384615, "spec_accepted_drafts": 135, "spec_proposed_drafts": 390, "spec_accept_histogram": [0, 1, 7, 5, 1, 3, 2, 1, 1, 1, 1, 0, 1, 1, 0, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e94effbfd96940738fbd013388125977", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 733, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170808.787831, "queue_time": 0.00044083595275878906, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.029898695182055, "spec_accept_rate": 0.34615384615384615, "spec_accept_length": 6.1923076923076925, "spec_num_correct_drafts": 135, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 135, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [0, 1, 7, 5, 1, 3, 2, 1, 1, 1, 1, 0, 1, 1, 0, 1], "spec_accept_histogram": [0, 1, 7, 5, 1, 3, 2, 1, 1, 1, 1, 0, 1, 1, 0, 1], "request_received_ts": 1788170807.369362, "api_server_dispatch_finish_ts": 1788170807.3711884, "request_finished_ts": 1788170811.399261, "decode_throughput": 57611902.02548625, "response_sent_to_client_ts": 1788170811.3995519}}
|
||||
{"prompt_id": 8, "output": " When he turned around he was 3*60=<<3*60=180>>180 miles from home\nHe was only able to drive 4-2=<<4-2=2>>2 hours in the first four hours\nIn half an hour he goes 30*.5=<<30*.5=15>>15 miles\nHe then drives another 2-.5=<<2-.5=1.5>>1.5 hours\nIn that time he goes 80*1.5=<<80*1.5=120>>120 miles\nSo he drove 120+15=<<120+15=135>>135 miles\nSo he is 180-135=<<180-135=45>>45 miles away from home\n#### 45\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.708333333333333, "spec_accept_length_from_histogram": 6.75, "spec_accept_rate": 0.38333333333333336, "spec_accepted_drafts": 138, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 1, 3, 3, 5, 3, 0, 2, 1, 3, 0, 1, 1, 0, 0, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a2d7d1d506934d3aafd2dd1d5518ff71", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 769, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170812.9181814, "queue_time": 0.0003256797790527344, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.9094296619296074, "spec_accept_rate": 0.38333333333333336, "spec_accept_length": 6.708333333333333, "spec_num_correct_drafts": 138, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 138, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 1, 3, 3, 5, 3, 0, 2, 1, 3, 0, 1, 1, 0, 0, 1], "spec_accept_histogram": [0, 1, 3, 3, 5, 3, 0, 2, 1, 3, 0, 1, 1, 0, 0, 1], "request_received_ts": 1788170811.4101784, "api_server_dispatch_finish_ts": 1788170811.4123018, "request_finished_ts": 1788170815.319608, "decode_throughput": 43859763.04314526, "response_sent_to_client_ts": 1788170815.3202214}}
|
||||
{"prompt_id": 9, "output": " Eliza is entitled to 45 -40 = <<45-40=5>>5 hours overtime pay.\nHer hourly rate for the overtime pay is $10 x 1.2 = $<<10*1.2=12>>12.\nSo, Eliza will receive $12 x 5 =$<<12*5=60>>60 for overtime pay.\nHer regular weekly earning is $10 x 40 = $<<10*40=400>>400.\nThus, Eliza will receive a total of $400 + $60 = $<<400+60=460>>460 for this week's work.\n#### 460\n\n", "correct": true, "error": null, "completion_tokens": 130, "spec_accept_length": 5.416666666666667, "spec_accept_length_from_histogram": 5.541666666666667, "spec_accept_rate": 0.30277777777777776, "spec_accepted_drafts": 109, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 3, 4, 3, 2, 6, 1, 0, 3, 0, 1, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "693952583c9c4eb394926a7dade4d661", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 725, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170816.8394153, "queue_time": 0.00033354759216308594, "reasoning_tokens": 0, "completion_tokens": 130, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 130}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.938992687035352, "spec_accept_rate": 0.30277777777777776, "spec_accept_length": 5.416666666666667, "spec_num_correct_drafts": 109, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 109, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 3, 4, 3, 2, 6, 1, 0, 3, 0, 1, 1], "spec_accept_histogram": [0, 3, 4, 3, 2, 6, 1, 0, 3, 0, 1, 1], "request_received_ts": 1788170815.331121, "api_server_dispatch_finish_ts": 1788170815.3345551, "request_finished_ts": 1788170819.2701137, "decode_throughput": 30876659.67365136, "response_sent_to_client_ts": 1788170819.2707355}}
|
||||
{"prompt_id": 10, "output": " The number of downloads of the program in the second month increased to 3*60 = <<3*60=180>>180\nIn the first two months, the total number of downloads of the program was 180+60 = <<180+60=240>>240\nIn the third month, the number of downloads of the program reduced by 30/100*180 = <<30/100*180=54>>54\nThere were 180-54 = <<180-54=126>>126 downloads in the third month.\nIn the three months, the total number of downloads of the program was 126+240 = <<126+240=366>>366\n#### 366\n\n", "correct": true, "error": null, "completion_tokens": 142, "spec_accept_length": 6.454545454545454, "spec_accept_length_from_histogram": 6.5, "spec_accept_rate": 0.36666666666666664, "spec_accepted_drafts": 121, "spec_proposed_drafts": 330, "spec_accept_histogram": [0, 0, 1, 6, 1, 2, 6, 1, 2, 2, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "def23d431f4c454282ca128ab87ed164", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 726, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170820.7009501, "queue_time": 0.0004684925079345703, "reasoning_tokens": 0, "completion_tokens": 142, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 142}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.6686324710026383, "spec_accept_rate": 0.36666666666666664, "spec_accept_length": 6.454545454545454, "spec_num_correct_drafts": 121, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 121, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [0, 0, 1, 6, 1, 2, 6, 1, 2, 2, 1], "spec_accept_histogram": [0, 0, 1, 6, 1, 2, 6, 1, 2, 2, 1], "request_received_ts": 1788170819.281607, "api_server_dispatch_finish_ts": 1788170819.2849867, "request_finished_ts": 1788170822.9502394, "decode_throughput": 34392911.67287596, "response_sent_to_client_ts": 1788170822.9507558}}
|
||||
{"prompt_id": 11, "output": " The total charge for the doughnuts was 3 x $68 = $<<3*68=204>>204.\nThe total charge for the mini cupcakes was 2 x $80 = $<<2*80=160>>160.\nThe total charge for the mini cheesecakes was 6 x $55 = $<<6*55=330>>330.\nTherefore the total amount Toula paid for the pastries was $204 + $160 + $330 = $<<204+160+330=694>>694.\n#### 694\n\n", "correct": true, "error": null, "completion_tokens": 111, "spec_accept_length": 7.928571428571429, "spec_accept_length_from_histogram": 8.0, "spec_accept_rate": 0.4666666666666667, "spec_accepted_drafts": 98, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 3, 1, 1, 1, 0, 0, 2, 0, 1, 1, 1, 1, 0, 0, 2], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "9f3a4eddf37b4df486b1c590a1b9c98c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 727, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170824.4234366, "queue_time": 0.000354766845703125, "reasoning_tokens": 0, "completion_tokens": 111, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 111}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.866735917981714, "spec_accept_rate": 0.4666666666666667, "spec_accept_length": 7.928571428571429, "spec_num_correct_drafts": 98, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 98, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 3, 1, 1, 1, 0, 0, 2, 0, 1, 1, 1, 1, 0, 0, 2], "spec_accept_histogram": [0, 3, 1, 1, 1, 0, 0, 2, 0, 1, 1, 1, 1, 0, 0, 2], "request_received_ts": 1788170822.9617724, "api_server_dispatch_finish_ts": 1788170822.9652433, "request_finished_ts": 1788170825.8285084, "decode_throughput": 1146.1920226028992, "response_sent_to_client_ts": 1788170825.8302448}}
|
||||
{"prompt_id": 12, "output": " He makes $10.5 selling lemons each year because 7 x 1.5 = <<7*1.5=10.5>>10.5\nHe earns $7.5 each year from the lemon tree because 10.5 - 3 = <<10.5-3=7.5>>7.5\nIt will take 12 years to earn enough to pay off the tree because 90 / 7.5 = <<90/7.5=12>>12\nHe will make money in year 13 because 12 + 1 = <<12+1=13>>13\n#### 13\n\n", "correct": true, "error": null, "completion_tokens": 133, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 6.947368421052632, "spec_accept_rate": 0.39649122807017545, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 1, 4, 3, 2, 1, 1, 0, 0, 2, 2, 1, 1, 0, 0, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "74583eeeeb86491ba2a992b05e9ecdf0", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 733, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170827.2494457, "queue_time": 0.00029158592224121094, "reasoning_tokens": 0, "completion_tokens": 133, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 133}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.313430523034185, "spec_accept_rate": 0.39649122807017545, "spec_accept_length": 7.0, "spec_num_correct_drafts": 113, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 1, 4, 3, 2, 1, 1, 0, 0, 2, 2, 1, 1, 0, 0, 1], "spec_accept_histogram": [0, 1, 4, 3, 2, 1, 1, 0, 0, 2, 2, 1, 1, 0, 0, 1], "request_received_ts": 1788170825.841031, "api_server_dispatch_finish_ts": 1788170825.8444462, "request_finished_ts": 1788170829.1544616, "decode_throughput": 35809479.72915614, "response_sent_to_client_ts": 1788170829.1549692}}
|
||||
{"prompt_id": 13, "output": " First multiply the five remaining vacuum cleaners by two to find out how many Melanie had before she visited the orange house: 5 * 2 = <<5*2=10>>10\nThen add two to figure out how many vacuum cleaners she had before visiting the red house: 10 + 2 = <<10+2=12>>12\nNow we know that 2/3 * x = 12, where x is the number of vacuum cleaners Melanie started with. We can find x by dividing each side of the equation by 2/3, which produces x = 18\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 126, "spec_accept_length": 5.478260869565218, "spec_accept_length_from_histogram": 5.521739130434782, "spec_accept_rate": 0.30144927536231886, "spec_accepted_drafts": 104, "spec_proposed_drafts": 345, "spec_accept_histogram": [0, 2, 5, 3, 3, 2, 1, 3, 3, 0, 1], "spec_verify_ct": 23, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "39d8067b5c374c0c8e57222af97c20c1", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 725, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170830.595103, "queue_time": 0.00033545494079589844, "reasoning_tokens": 0, "completion_tokens": 126, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 126}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.7262040907517076, "spec_accept_rate": 0.30144927536231886, "spec_accept_length": 5.478260869565218, "spec_num_correct_drafts": 104, "spec_num_proposed_drafts": 345, "spec_verify_ct": 23, "spec_accepted_drafts": 104, "spec_proposed_drafts": 345, "spec_correct_drafts_histogram": [0, 2, 5, 3, 3, 2, 1, 3, 3, 0, 1], "spec_accept_histogram": [0, 2, 5, 3, 3, 2, 1, 3, 3, 0, 1], "request_received_ts": 1788170829.1658404, "api_server_dispatch_finish_ts": 1788170829.1692853, "request_finished_ts": 1788170832.8920445, "decode_throughput": 27877812.44158272, "response_sent_to_client_ts": 1788170832.8926399}}
|
||||
{"prompt_id": 14, "output": " There are 20 x 20/100 = <<20*20/100=4>>4 students who enrolled in contemporary dance.\nSo, 20 - 4 = <<20-4=16>>16 students are enrolled in either jazz or hip-hop dance.\nThere are 16 x 25/100 = <<16*25/100=4>>4 students who enrolled in jazz dance.\nHence, 16 - 4 = <<16-4=12>>12 students enrolled in hip-hop dance.\nThis is 12/20 x 100% = 60% of the entire students.\n#### 60\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 6.7368421052631575, "spec_accept_length_from_histogram": 6.7894736842105265, "spec_accept_rate": 0.38596491228070173, "spec_accepted_drafts": 110, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 1, 4, 1, 1, 5, 0, 1, 0, 3, 0, 1, 2], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a7730e48a01f470fa67425b4ca0859ab", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170834.349421, "queue_time": 0.0004432201385498047, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.3648715745657682, "spec_accept_rate": 0.38596491228070173, "spec_accept_length": 6.7368421052631575, "spec_num_correct_drafts": 110, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 110, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 1, 4, 1, 1, 5, 0, 1, 0, 3, 0, 1, 2], "spec_accept_histogram": [0, 1, 4, 1, 1, 5, 0, 1, 0, 3, 0, 1, 2], "request_received_ts": 1788170832.9033282, "api_server_dispatch_finish_ts": 1788170832.9066398, "request_finished_ts": 1788170836.2682, "decode_throughput": 33976631.779743366, "response_sent_to_client_ts": 1788170836.2732253}}
|
||||
{"prompt_id": 15, "output": " If he purchases jewelry, he will make a profit of 2.5% which is $5000*(2.5/100) = $<<5000*(2.5/100)=125>>125\nIf he purchases electronic gadgets, he will make a profit of 1.2% which is $8000*(1.2/100) = $<<8000*(1.2/100)=96>>96\nIf he wants to maximize profit, since $125 > $96, he will choose to purchase jewelry, thereby making a profit of $<<125=125>>125\n#### 125\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 7.529411764705882, "spec_accept_length_from_histogram": 7.588235294117647, "spec_accept_rate": 0.4392156862745098, "spec_accepted_drafts": 112, "spec_proposed_drafts": 255, "spec_accept_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "spec_verify_ct": 17, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "262436b409644a148400619cc4982392", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 758, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170837.7120037, "queue_time": 0.00043201446533203125, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.1382477888837457, "spec_accept_rate": 0.4392156862745098, "spec_accept_length": 7.529411764705882, "spec_num_correct_drafts": 112, "spec_num_proposed_drafts": 255, "spec_verify_ct": 17, "spec_accepted_drafts": 112, "spec_proposed_drafts": 255, "spec_correct_drafts_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "spec_accept_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "request_received_ts": 1788170836.2847843, "api_server_dispatch_finish_ts": 1788170836.288778, "request_finished_ts": 1788170839.4230323, "decode_throughput": 16817563.254362706, "response_sent_to_client_ts": 1788170839.4235678}}
|
||||
{"prompt_id": 16, "output": " On the first day, the trains covered 2 trains * 80 miles/train = <<2*80=160>>160 miles together.\nThey also covered 150 miles/train * 2 trains = <<150*2=300>>300 miles together on the second day.\nThe combined distance the two trains covered in the two days is 300 miles + 160 miles = <<300+160=460>>460 miles\nThe average distance for the two days is 460 miles / 2 trains = <<460/2=230>>230 miles/train\n#### 230\n\n", "correct": true, "error": null, "completion_tokens": 117, "spec_accept_length": 5.85, "spec_accept_length_from_histogram": 5.9, "spec_accept_rate": 0.32666666666666666, "spec_accepted_drafts": 98, "spec_proposed_drafts": 300, "spec_accept_histogram": [0, 1, 3, 5, 3, 0, 0, 5, 0, 1, 2], "spec_verify_ct": 20, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "99c77fce13bc47df977defafe52793f4", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170840.8697152, "queue_time": 0.0003342628479003906, "reasoning_tokens": 0, "completion_tokens": 117, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 117}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.421430238056928, "spec_accept_rate": 0.32666666666666666, "spec_accept_length": 5.85, "spec_num_correct_drafts": 98, "spec_num_proposed_drafts": 300, "spec_verify_ct": 20, "spec_accepted_drafts": 98, "spec_proposed_drafts": 300, "spec_correct_drafts_histogram": [0, 1, 3, 5, 3, 0, 0, 5, 0, 1, 2], "spec_accept_histogram": [0, 1, 3, 5, 3, 0, 0, 5, 0, 1, 2], "request_received_ts": 1788170839.4343305, "api_server_dispatch_finish_ts": 1788170839.4378765, "request_finished_ts": 1788170842.8557606, "decode_throughput": 29546685.22927292, "response_sent_to_client_ts": 1788170842.8563435}}
|
||||
{"prompt_id": 17, "output": " First find the total amount Jill makes per week teaching: $20/hour * 35 hours/week = $<<20*35=700>>700/week\nThen find the total amount Jill makes per week coaching: $30/hour * 15 hours/week = $<<30*15=450>>450/week\nThen add those two amounts to find the total amount Jill makes per week: $700/week + $450/week = $<<700+450=1150>>1150/week\nThen multiply that number by the number of weeks Jill works in a year to find her annual salary: $1150/week * 50 weeks/year = $<<1150*50=57500>>57,500\n#### 57500\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 6.0, "spec_accept_length_from_histogram": 6.08, "spec_accept_rate": 0.33866666666666667, "spec_accepted_drafts": 127, "spec_proposed_drafts": 375, "spec_accept_histogram": [0, 1, 5, 4, 2, 3, 5, 1, 1, 0, 1, 0, 1, 0, 1], "spec_verify_ct": 25, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "47468684c3f040aca3fe737647943ddb", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 719, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170844.3002982, "queue_time": 0.0002906322479248047, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.931107017211616, "spec_accept_rate": 0.33866666666666667, "spec_accept_length": 6.0, "spec_num_correct_drafts": 127, "spec_num_proposed_drafts": 375, "spec_verify_ct": 25, "spec_accepted_drafts": 127, "spec_proposed_drafts": 375, "spec_correct_drafts_histogram": [0, 1, 5, 4, 2, 3, 5, 1, 1, 0, 1, 0, 1, 0, 1], "spec_accept_histogram": [0, 1, 5, 4, 2, 3, 5, 1, 1, 0, 1, 0, 1, 0, 1], "request_received_ts": 1788170842.867236, "api_server_dispatch_finish_ts": 1788170842.8706605, "request_finished_ts": 1788170846.798343, "decode_throughput": 33678040.58014946, "response_sent_to_client_ts": 1788170846.7989378}}
|
||||
{"prompt_id": 18, "output": " She eats 3 eggs every day and there are 7 days in a week so she eats 3*7 = <<3*7=21>>21 eggs a week\nAfter 4 weeks she will have eaten 4*21 = <<4*21=84>>84 eggs\nThere are 12 eggs in 1 dozen and she'll eat 84 eggs so that's 84/12 = <<84/12=7>>7 dozen eggs\n#### 7\n\n", "correct": true, "error": null, "completion_tokens": 98, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.1875, "spec_accept_rate": 0.3458333333333333, "spec_accepted_drafts": 83, "spec_proposed_drafts": 240, "spec_accept_histogram": [0, 1, 2, 3, 1, 5, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "spec_verify_ct": 16, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "67a3528939554615a843434a4d47c7f5", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 695, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170848.2723129, "queue_time": 0.0005748271942138672, "reasoning_tokens": 0, "completion_tokens": 98, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 98}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.0536309089511633, "spec_accept_rate": 0.3458333333333333, "spec_accept_length": 6.125, "spec_num_correct_drafts": 83, "spec_num_proposed_drafts": 240, "spec_verify_ct": 16, "spec_accepted_drafts": 83, "spec_proposed_drafts": 240, "spec_correct_drafts_histogram": [0, 1, 2, 3, 1, 5, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 1, 5, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "request_received_ts": 1788170846.8097692, "api_server_dispatch_finish_ts": 1788170846.8130748, "request_finished_ts": 1788170849.8634002, "decode_throughput": 22454016.800258704, "response_sent_to_client_ts": 1788170849.8639147}}
|
||||
{"prompt_id": 19, "output": " First figure out how many hours it takes to hike a 12-mile trail at 4 mph by dividing the distance by the speed: 12 miles / 4 mph = <<12/4=3>>3 hours\nNext subtract the time Marissa already spent walking to find out how much time she has left: 3 hours - 1 hour- 1 hour = <<3-1-1=1>>1 hour\nNow figure out how much distance she has left by subtracting the distance she already traveled from the total distance: 12 miles - 4 miles - 2 miles = <<12-4-2=6>>6 miles\nNow divide the remaining distance by the remaining time to find out how fast in miles per hour Marissa has to travel: 6 miles / 1 hour = <<6/1=6>>6 mph\n#### 6\n\n", "correct": true, "error": null, "completion_tokens": 181, "spec_accept_length": 5.65625, "spec_accept_length_from_histogram": 5.6875, "spec_accept_rate": 0.3125, "spec_accepted_drafts": 150, "spec_proposed_drafts": 480, "spec_accept_histogram": [1, 1, 2, 10, 4, 4, 3, 0, 3, 3, 1], "spec_verify_ct": 32, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5de38f47c87c48d5941172703a124b19", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 732, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170851.2814784, "queue_time": 0.00037932395935058594, "reasoning_tokens": 0, "completion_tokens": 181, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 181}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.63415600406006, "spec_accept_rate": 0.3125, "spec_accept_length": 5.65625, "spec_num_correct_drafts": 150, "spec_num_proposed_drafts": 480, "spec_verify_ct": 32, "spec_accepted_drafts": 150, "spec_proposed_drafts": 480, "spec_correct_drafts_histogram": [1, 1, 2, 10, 4, 4, 3, 0, 3, 3, 1], "spec_accept_histogram": [1, 1, 2, 10, 4, 4, 3, 0, 3, 3, 1], "request_received_ts": 1788170849.8748236, "api_server_dispatch_finish_ts": 1788170849.8781939, "request_finished_ts": 1788170854.5089796, "decode_throughput": 44822246.827458255, "response_sent_to_client_ts": 1788170854.5095549}}
|
||||
{"prompt_id": 20, "output": " There are 15 x 3/5 = <<15*3/5=9>>9 liters of water from the 15 liters pineapple drink.\nAfter 1 liter of orange drink was spilled, there were 10 - 1 = <<10-1=9>>9 liters of orange drink left.\nOut of the 9 liters, 9 x 2/3 = <<9*2/3=6>>6 liters are water.\nThus, there are a total of 9 + 6 = <<9+6=15>>15 liters of water out of the 24 liters.\n#### 15\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 5.818181818181818, "spec_accept_length_from_histogram": 5.863636363636363, "spec_accept_rate": 0.3242424242424242, "spec_accepted_drafts": 107, "spec_proposed_drafts": 330, "spec_accept_histogram": [1, 1, 4, 3, 2, 6, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3d3b58f2ce5247739068430353295dc8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170855.9673853, "queue_time": 0.0003261566162109375, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.6571107748895884, "spec_accept_rate": 0.3242424242424242, "spec_accept_length": 5.818181818181818, "spec_num_correct_drafts": 107, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 107, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [1, 1, 4, 3, 2, 6, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [1, 1, 4, 3, 2, 6, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788170854.5221355, "api_server_dispatch_finish_ts": 1788170854.5255566, "request_finished_ts": 1788170858.1792462, "decode_throughput": 105.74452865638081, "response_sent_to_client_ts": 1788170858.1798604}}
|
||||
{"prompt_id": 21, "output": " When Raymond's son was born Samantha was 23 - 6 = <<23-6=17>>17 years old.\nThus it has been 31 - 17 = <<31-17=14>>14 years since Raymond's son was born.\n#### 14\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.125, "spec_accept_rate": 0.4083333333333333, "spec_accepted_drafts": 49, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "995465f2450c4564b3b909a13642a52f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 711, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170859.6130674, "queue_time": 0.00034308433532714844, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.2496120859868824, "spec_accept_rate": 0.4083333333333333, "spec_accept_length": 7.0, "spec_num_correct_drafts": 49, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 49, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "request_received_ts": 1788170858.1908987, "api_server_dispatch_finish_ts": 1788170858.19429, "request_finished_ts": 1788170860.4405107, "decode_throughput": 14554725.8952557, "response_sent_to_client_ts": 1788170860.4410005}}
|
||||
{"prompt_id": 22, "output": " His first 3 customers buy 3 * 1 = <<3*1=3>>3 DVDs.\nHis next 2 buy 2 * 2 = <<2*2=4>>4 DVDs.\nHe sells a total of 3 + 4 + 0 = <<3+4+0=7>>7 DVDs.\n#### 7\n\n", "correct": true, "error": null, "completion_tokens": 74, "spec_accept_length": 8.222222222222221, "spec_accept_length_from_histogram": 8.333333333333332, "spec_accept_rate": 0.4888888888888889, "spec_accepted_drafts": 66, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 1, 1, 1, 0, 1, 0, 0, 3, 1, 0, 0, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3df38f96cfea4831b51639ef9f0d6dec", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 722, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170861.8394158, "queue_time": 0.0003330707550048828, "reasoning_tokens": 0, "completion_tokens": 74, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 74}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.297401685733348, "spec_accept_rate": 0.4888888888888889, "spec_accept_length": 8.222222222222221, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 66, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 1, 1, 1, 0, 1, 0, 0, 3, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 1, 0, 1, 0, 0, 3, 1, 0, 0, 0, 1], "request_received_ts": 1788170860.451573, "api_server_dispatch_finish_ts": 1788170860.4548833, "request_finished_ts": 1788170862.7489746, "decode_throughput": 21773098.097777776, "response_sent_to_client_ts": 1788170862.7495334}}
|
||||
{"prompt_id": 23, "output": " The candle burns for 5 - 1 = <<5-1=4>>4 hours.\nThus, the candle will be 2 * 4 = <<2*4=8>>8 centimeters shorter.\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.25, "spec_accept_rate": 0.35, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 1, 1, 2, 1, 1, 0, 0, 2], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b069d0d44b624c119ebd3cbbf125058f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 706, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170864.1499302, "queue_time": 0.00030159950256347656, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.209783522877842, "spec_accept_rate": 0.35, "spec_accept_length": 6.125, "spec_num_correct_drafts": 42, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 1, 1, 2, 1, 1, 0, 0, 2], "spec_accept_histogram": [0, 0, 1, 1, 2, 1, 1, 0, 0, 2], "request_received_ts": 1788170862.7600243, "api_server_dispatch_finish_ts": 1788170862.7633939, "request_finished_ts": 1788170864.9698079, "decode_throughput": 13116072.668787377, "response_sent_to_client_ts": 1788170864.9703045}}
|
||||
{"prompt_id": 24, "output": " Let X be the original price of the book. The discounted price is X - X*25% = $19.50.\nCombining like terms, we get .75X = $19.50\nDividing both sides by .75, we get X = $26.\n#### 26\n\n", "correct": true, "error": null, "completion_tokens": 63, "spec_accept_length": 5.25, "spec_accept_length_from_histogram": 5.166666666666667, "spec_accept_rate": 0.2777777777777778, "spec_accepted_drafts": 50, "spec_proposed_drafts": 180, "spec_accept_histogram": [1, 0, 2, 4, 1, 1, 1, 0, 1, 0, 0, 1], "spec_verify_ct": 12, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "1736ed4e30af4f4dad4bf92e5aaf7dcc", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 703, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170866.342023, "queue_time": 0.0008862018585205078, "reasoning_tokens": 0, "completion_tokens": 63, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 63}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.56409065797925, "spec_accept_rate": 0.2777777777777778, "spec_accept_length": 5.25, "spec_num_correct_drafts": 50, "spec_num_proposed_drafts": 180, "spec_verify_ct": 12, "spec_accepted_drafts": 50, "spec_proposed_drafts": 180, "spec_correct_drafts_histogram": [1, 0, 2, 4, 1, 1, 1, 0, 1, 0, 0, 1], "spec_accept_histogram": [1, 0, 2, 4, 1, 1, 1, 0, 1, 0, 0, 1], "request_received_ts": 1788170864.982442, "api_server_dispatch_finish_ts": 1788170864.9856894, "request_finished_ts": 1788170867.5465326, "decode_throughput": 17878875.54397744, "response_sent_to_client_ts": 1788170867.5470998}}
|
||||
{"prompt_id": 25, "output": " Five packs of milk cost $3 x 5 = $<<3*5=15>>15.\nFour apples cost $1.50 x 4 = $<<1.5*4=6>>6.\nThe total cost of the chicken meal, milk, and apples is $12 + $15 + $6 = $<<12+15+6=33>>33.\nThus, the boxes of pizza cost $50 - $33 = $<<50-33=17>>17.\nTherefore, Marie ordered $17/$8.50 = <<17/8.5=2>>2 boxes of pizza.\n#### 2\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 6.894736842105263, "spec_accept_length_from_histogram": 6.947368421052632, "spec_accept_rate": 0.39649122807017545, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 1, 1, 2, 4, 2, 1, 3, 1, 1, 2, 0, 0, 0, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5f79cc5908a944638013287169873acf", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170868.959407, "queue_time": 0.0003733634948730469, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.3248235899955034, "spec_accept_rate": 0.39649122807017545, "spec_accept_length": 6.894736842105263, "spec_num_correct_drafts": 113, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 1, 1, 2, 4, 2, 1, 3, 1, 1, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 1, 2, 4, 2, 1, 3, 1, 1, 2, 0, 0, 0, 1], "request_received_ts": 1788170867.5586872, "api_server_dispatch_finish_ts": 1788170867.5619972, "request_finished_ts": 1788170870.8835108, "decode_throughput": 32499752.530849826, "response_sent_to_client_ts": 1788170870.884021}}
|
||||
{"prompt_id": 26, "output": " 3 * (16.50 + 22.50 + 42) = $<<3*(16.50+22.50+42)=243>>243\nMishka spent $243 on clothing.\n#### 243\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.142857142857143, "spec_accept_rate": 0.4095238095238095, "spec_accepted_drafts": 43, "spec_proposed_drafts": 105, "spec_accept_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "76e516be97654927966ce64f0496cd47", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170872.3097615, "queue_time": 0.00034809112548828125, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.130047915969044, "spec_accept_rate": 0.4095238095238095, "spec_accept_length": 7.0, "spec_num_correct_drafts": 43, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 43, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "spec_accept_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "request_received_ts": 1788170870.8989816, "api_server_dispatch_finish_ts": 1788170870.9024105, "request_finished_ts": 1788170873.0290294, "decode_throughput": 12646204.772911299, "response_sent_to_client_ts": 1788170873.0296059}}
|
||||
{"prompt_id": 27, "output": " Each container of ice cream has 15 servings and she eats 1 serving a night so after 60 days she will need 60/15 = 4 containers of ice cream\nIf each carton costs $4.00 and she needs 4 containers then it will cost her 4*4 = $<<4*4=16.00>>16.00\n#### 16\n\n", "correct": true, "error": null, "completion_tokens": 80, "spec_accept_length": 5.333333333333333, "spec_accept_length_from_histogram": 5.4, "spec_accept_rate": 0.29333333333333333, "spec_accepted_drafts": 66, "spec_proposed_drafts": 225, "spec_accept_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "spec_verify_ct": 15, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "6f1e8ef80f3c406cb2aee1be04249460", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 721, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170874.4416146, "queue_time": 0.0002911090850830078, "reasoning_tokens": 0, "completion_tokens": 80, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 80}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.9219643361866474, "spec_accept_rate": 0.29333333333333333, "spec_accept_length": 5.333333333333333, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 225, "spec_verify_ct": 15, "spec_accepted_drafts": 66, "spec_proposed_drafts": 225, "spec_correct_drafts_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "request_received_ts": 1788170873.0405114, "api_server_dispatch_finish_ts": 1788170873.0438094, "request_finished_ts": 1788170875.9624758, "decode_throughput": 19749849.614901047, "response_sent_to_client_ts": 1788170875.9629645}}
|
||||
{"prompt_id": 28, "output": " He traveled 20 miles + 15 miles = <<20+15=35>>35 miles not counting the distance between stops.\nHenry traveled 60 miles - 35 miles = <<60-35=25>>25 miles between his first and second stop.\n#### 25\n\n", "correct": true, "error": null, "completion_tokens": 57, "spec_accept_length": 5.7, "spec_accept_length_from_histogram": 5.8, "spec_accept_rate": 0.32, "spec_accepted_drafts": 48, "spec_proposed_drafts": 150, "spec_accept_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 10, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "1eebb668dd9843c29ac8824870f64a3a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 715, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170877.3851192, "queue_time": 0.0002970695495605469, "reasoning_tokens": 0, "completion_tokens": 57, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 57}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.4182498469017446, "spec_accept_rate": 0.32, "spec_accept_length": 5.7, "spec_num_correct_drafts": 48, "spec_num_proposed_drafts": 150, "spec_verify_ct": 10, "spec_accepted_drafts": 48, "spec_proposed_drafts": 150, "spec_correct_drafts_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788170875.9738064, "api_server_dispatch_finish_ts": 1788170875.9770806, "request_finished_ts": 1788170878.3920562, "decode_throughput": 14503025.119150989, "response_sent_to_client_ts": 1788170878.3926554}}
|
||||
{"prompt_id": 29, "output": " The second pair of heels costs 33 * 2 = $<<33*2=66>>66.\nThe heels together cost 66 + 33 = $<<66+33=99>>99.\nThe boots cost $5 more than both pairs of heels together, so the boots cost 99 + 5 = $104.\n#### 104\n\n", "correct": true, "error": null, "completion_tokens": 73, "spec_accept_length": 6.636363636363637, "spec_accept_length_from_histogram": 6.7272727272727275, "spec_accept_rate": 0.38181818181818183, "spec_accepted_drafts": 63, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e911caa130794ed0a7b436b4ce248e75", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 734, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170879.8008873, "queue_time": 0.0003352165222167969, "reasoning_tokens": 0, "completion_tokens": 73, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 73}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.5309368656016886, "spec_accept_rate": 0.38181818181818183, "spec_accept_length": 6.636363636363637, "spec_num_correct_drafts": 63, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 63, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "spec_accept_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "request_received_ts": 1788170878.4037273, "api_server_dispatch_finish_ts": 1788170878.4071407, "request_finished_ts": 1788170880.9346642, "decode_throughput": 180.1111053272638, "response_sent_to_client_ts": 1788170880.9351585}}
|
||||
{"prompt_id": 30, "output": " The total ratio representing their ages is 7+11= <<7+11=18>>18\nSince the fraction of the ratio that represents Allen's age is 11/18, Allen's current age is 11/18*162 = <<11/18*162=99>>99\nIf Allen is currently 99 years old, in 10 years he will be 99+10 = <<99+10=109>>109 years old\n#### 109\n\n", "correct": true, "error": null, "completion_tokens": 98, "spec_accept_length": 5.444444444444445, "spec_accept_length_from_histogram": 5.5, "spec_accept_rate": 0.3, "spec_accepted_drafts": 81, "spec_proposed_drafts": 270, "spec_accept_histogram": [0, 1, 4, 4, 1, 3, 1, 0, 3, 0, 0, 1], "spec_verify_ct": 18, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "837cec6d9f1542b9832531174d35dde4", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 703, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170882.3101165, "queue_time": 0.0002810955047607422, "reasoning_tokens": 0, "completion_tokens": 98, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 98}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.1735073472373188, "spec_accept_rate": 0.3, "spec_accept_length": 5.444444444444445, "spec_num_correct_drafts": 81, "spec_num_proposed_drafts": 270, "spec_verify_ct": 18, "spec_accepted_drafts": 81, "spec_proposed_drafts": 270, "spec_correct_drafts_histogram": [0, 1, 4, 4, 1, 3, 1, 0, 3, 0, 0, 1], "spec_accept_histogram": [0, 1, 4, 4, 1, 3, 1, 0, 3, 0, 0, 1], "request_received_ts": 1788170880.9458122, "api_server_dispatch_finish_ts": 1788170880.9491415, "request_finished_ts": 1788170884.1193197, "decode_throughput": 126.50049173681164, "response_sent_to_client_ts": 1788170884.1198432}}
|
||||
{"prompt_id": 31, "output": " The second person guessed 60 because 20 + (80/2)= <<20+(80/2)=60>>60\nThe second guessed 100 because 80 x 1.25 = <<80*1.25=100>>100\nThe total amount guessed is 240 because 80 + 60 + 100= <<80+60+100=240>>240\nThe average amount guessed is 80 because 240 / 3 = <<240/3=80>>80\n#### 80\n\n", "correct": true, "error": null, "completion_tokens": 107, "spec_accept_length": 6.294117647058823, "spec_accept_length_from_histogram": 6.235294117647059, "spec_accept_rate": 0.34901960784313724, "spec_accepted_drafts": 89, "spec_proposed_drafts": 255, "spec_accept_histogram": [1, 2, 3, 3, 1, 0, 1, 2, 0, 1, 0, 1, 0, 1, 0, 1], "spec_verify_ct": 17, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "4cd11dbacf534e95b4048bc227108c12", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170885.5601656, "queue_time": 0.0004112720489501953, "reasoning_tokens": 0, "completion_tokens": 107, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 107}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.1245789229869843, "spec_accept_rate": 0.34901960784313724, "spec_accept_length": 6.294117647058823, "spec_num_correct_drafts": 89, "spec_num_proposed_drafts": 255, "spec_verify_ct": 17, "spec_accepted_drafts": 89, "spec_proposed_drafts": 255, "spec_correct_drafts_histogram": [1, 2, 3, 3, 1, 0, 1, 2, 0, 1, 0, 1, 0, 1, 0, 1], "spec_accept_histogram": [1, 2, 3, 3, 1, 0, 1, 2, 0, 1, 0, 1, 0, 1, 0, 1], "request_received_ts": 1788170884.1305885, "api_server_dispatch_finish_ts": 1788170884.1337116, "request_finished_ts": 1788170887.2551675, "decode_throughput": 30832082.71542733, "response_sent_to_client_ts": 1788170887.255636}}
|
||||
{"prompt_id": 32, "output": " He spends 10*.5=<<10*.5=5>>5 hours per day\nThat means he spends 5*7=<<5*7=35>>35 hours per week\n#### 35\n\n", "correct": true, "error": null, "completion_tokens": 45, "spec_accept_length": 6.428571428571429, "spec_accept_length_from_histogram": 6.571428571428571, "spec_accept_rate": 0.37142857142857144, "spec_accepted_drafts": 39, "spec_proposed_drafts": 105, "spec_accept_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "836fd5daef5144cdb439b4f103eda647", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 708, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170888.6946337, "queue_time": 0.000484466552734375, "reasoning_tokens": 0, "completion_tokens": 45, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 45}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.1361825759522617, "spec_accept_rate": 0.37142857142857144, "spec_accept_length": 6.428571428571429, "spec_num_correct_drafts": 39, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 39, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_accept_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "request_received_ts": 1788170887.2667506, "api_server_dispatch_finish_ts": 1788170887.269842, "request_finished_ts": 1788170889.4029331, "decode_throughput": 13513913.116704805, "response_sent_to_client_ts": 1788170889.4034712}}
|
||||
{"prompt_id": 33, "output": " Let x be the number of silver coins Gretchen has\nGretchen has x+30 gold coins.\nx+x+30=110\n2*x=80\nx=<<40=40>>40\nGretchen has 40+30=<<40+30=70>>70 gold coins\n#### 70\n\n", "correct": true, "error": null, "completion_tokens": 68, "spec_accept_length": 4.857142857142857, "spec_accept_length_from_histogram": 4.928571428571429, "spec_accept_rate": 0.2619047619047619, "spec_accepted_drafts": 55, "spec_proposed_drafts": 210, "spec_accept_histogram": [1, 4, 1, 3, 1, 0, 1, 1, 0, 0, 0, 1, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "f2631788ee9347938c35faa4222343a1", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 695, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170890.770247, "queue_time": 0.00026726722717285156, "reasoning_tokens": 0, "completion_tokens": 68, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 68}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.734878122806549, "spec_accept_rate": 0.2619047619047619, "spec_accept_length": 4.857142857142857, "spec_num_correct_drafts": 55, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 55, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [1, 4, 1, 3, 1, 0, 1, 1, 0, 0, 0, 1, 1], "spec_accept_histogram": [1, 4, 1, 3, 1, 0, 1, 1, 0, 0, 0, 1, 1], "request_received_ts": 1788170889.4146614, "api_server_dispatch_finish_ts": 1788170889.417757, "request_finished_ts": 1788170892.1495395, "decode_throughput": 20858423.371412005, "response_sent_to_client_ts": 1788170892.150239}}
|
||||
{"prompt_id": 34, "output": " Half of Raymond's jewels is 40/2 = <<40/2=20>>20\nSince Aaron has 5 more jewels than half of Raymond's jewels, he has 20+5 = <<5+20=25>>25 jewels.\nIf Siobhan has 2 fewer jewels than Aaron, he has 25-2 = 23 jewels.\n#### 23\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 6.583333333333333, "spec_accept_length_from_histogram": 6.666666666666667, "spec_accept_rate": 0.37777777777777777, "spec_accepted_drafts": 68, "spec_proposed_drafts": 180, "spec_accept_histogram": [0, 2, 2, 0, 2, 2, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 12, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "4ee7e8e51159464db9141628a93e943b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 708, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170893.5660138, "queue_time": 0.00028061866760253906, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.622302720323205, "spec_accept_rate": 0.37777777777777777, "spec_accept_length": 6.583333333333333, "spec_num_correct_drafts": 68, "spec_num_proposed_drafts": 180, "spec_verify_ct": 12, "spec_accepted_drafts": 68, "spec_proposed_drafts": 180, "spec_correct_drafts_histogram": [0, 2, 2, 0, 2, 2, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 2, 2, 0, 2, 2, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788170892.1608543, "api_server_dispatch_finish_ts": 1788170892.163948, "request_finished_ts": 1788170894.783157, "decode_throughput": 18867281.430952918, "response_sent_to_client_ts": 1788170894.783741}}
|
||||
{"prompt_id": 35, "output": " He scored 4*.25=<<4*.25=1>>1 point more in the second 20 minutes\nSo he scored 4+1=<<4+1=5>>5 points in the second 20 minutes\nSo in total, he scored 4+5=<<4+5=9>>9 points in the 40 minutes\n#### 9\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 8.777777777777779, "spec_accept_length_from_histogram": 8.88888888888889, "spec_accept_rate": 0.5259259259259259, "spec_accepted_drafts": 71, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c0f2a0cb4c4241c7986ff13200f1b879", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 716, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170896.2150328, "queue_time": 0.00034117698669433594, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.348722190130502, "spec_accept_rate": 0.5259259259259259, "spec_accept_length": 8.777777777777779, "spec_num_correct_drafts": 71, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 71, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "request_received_ts": 1788170894.7946024, "api_server_dispatch_finish_ts": 1788170894.7978692, "request_finished_ts": 1788170897.1433246, "decode_throughput": 250.96850569911032, "response_sent_to_client_ts": 1788170897.1438336}}
|
||||
{"prompt_id": 36, "output": " He eats 2 yogurts a day so over 30 days he will need 2*30 = <<2*30=60>>60 yogurts\nThe yogurt is sold as 4 units for $5.00 and he needs 60 yogurts so that’s 60/4 = <<60/4=15>>15 4-packs\nEach 4-pack is $5.00 and he needs 15 4-packs so he will spend $5*15 = $<<5*15=75.00>>75.00 on yogurt\n#### 75\n\n", "correct": true, "error": null, "completion_tokens": 124, "spec_accept_length": 5.636363636363637, "spec_accept_length_from_histogram": 5.681818181818182, "spec_accept_rate": 0.31212121212121213, "spec_accepted_drafts": 103, "spec_proposed_drafts": 330, "spec_accept_histogram": [0, 2, 3, 3, 5, 2, 1, 1, 4, 0, 0, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "60d64fa240ab46f295f1e3eb7d0f95d5", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170898.5405302, "queue_time": 0.0003113746643066406, "reasoning_tokens": 0, "completion_tokens": 124, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 124}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.6006797631271183, "spec_accept_rate": 0.31212121212121213, "spec_accept_length": 5.636363636363637, "spec_num_correct_drafts": 103, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 103, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [0, 2, 3, 3, 5, 2, 1, 1, 4, 0, 0, 1], "spec_accept_histogram": [0, 2, 3, 3, 5, 2, 1, 1, 4, 0, 0, 1], "request_received_ts": 1788170897.1546216, "api_server_dispatch_finish_ts": 1788170897.15787, "request_finished_ts": 1788170900.7553015, "decode_throughput": 31130287.41355333, "response_sent_to_client_ts": 1788170900.75588}}
|
||||
{"prompt_id": 37, "output": " He spent $160 on video games because 8 x 20 = <<8*20=160>>160\nHe earned $165 from his sale because 160 + 5 = <<160+5=165>>165\nHe sold 11 lego sets because 11 x 15 = <<11*15=165>>165\nHe has 2 sets left because 13 - 11 = <<13-11=2>>2\n#### 2\n\n", "correct": true, "error": null, "completion_tokens": 96, "spec_accept_length": 6.857142857142857, "spec_accept_length_from_histogram": 6.928571428571429, "spec_accept_rate": 0.3952380952380952, "spec_accepted_drafts": 83, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 2, 1, 3, 2, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "23312e6485564498a0fe6e6bcc8a27a6", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170902.2157185, "queue_time": 0.0003323554992675781, "reasoning_tokens": 0, "completion_tokens": 96, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 96}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.8655119501054287, "spec_accept_rate": 0.3952380952380952, "spec_accept_length": 6.857142857142857, "spec_num_correct_drafts": 83, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 83, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 2, 1, 3, 2, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1], "spec_accept_histogram": [0, 2, 1, 3, 2, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1], "request_received_ts": 1788170900.7675862, "api_server_dispatch_finish_ts": 1788170900.7709525, "request_finished_ts": 1788170903.6330981, "decode_throughput": 27431887.395455156, "response_sent_to_client_ts": 1788170903.6335719}}
|
||||
{"prompt_id": 38, "output": " He has 7-1=<<7-1=6>>6 possible running days.\nHe runs half of those so he runs 6/2=<<6/2=3>>3 days.\nSo for two days, he runs 3/2=<<3/2=1.5>>1.5 hours.\nSo in total, he runs 3+1.5+1.5=<<3+1.5+1.5=6>>6 hours.\nSo he runs 60/6=<<60/6=10>>10 mph.\n#### 10\n\n", "correct": true, "error": null, "completion_tokens": 124, "spec_accept_length": 6.526315789473684, "spec_accept_length_from_histogram": 6.578947368421052, "spec_accept_rate": 0.3719298245614035, "spec_accepted_drafts": 106, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 3, 1, 2, 2, 0, 4, 3, 1, 1, 1, 0, 0, 0, 0, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "d134ebde1f024e59b90d8dce1a393197", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 711, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170905.058391, "queue_time": 0.00033736228942871094, "reasoning_tokens": 0, "completion_tokens": 124, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 124}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.3058398980647326, "spec_accept_rate": 0.3719298245614035, "spec_accept_length": 6.526315789473684, "spec_num_correct_drafts": 106, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 106, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 3, 1, 2, 2, 0, 4, 3, 1, 1, 1, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 3, 1, 2, 2, 0, 4, 3, 1, 1, 1, 0, 0, 0, 0, 1], "request_received_ts": 1788170903.6487288, "api_server_dispatch_finish_ts": 1788170903.6522913, "request_finished_ts": 1788170906.9545686, "decode_throughput": 27127502.177672796, "response_sent_to_client_ts": 1788170906.955192}}
|
||||
{"prompt_id": 39, "output": " If Dana can skip at half the speed she can run, then she can run at 3*2=<<3*2=6>>6 miles per hour.\nAnd since she can run at a speed that is 4 times faster than she can walk, this means she can walk at 6/4=1.5 miles per hour.\nIf two-thirds of the time is spent walking, then she walks for 6*(2/3)=<<6*(2/3)=4>>4 hours.\nIf one-third of the time is spent running, then she runs for 6-4=<<6-4=2>>2 hours.\nThus, she runs for 2 hours at 6 miles per hour, or 2*6=<<2*6=12>>12 miles.\nShe walks for 4 hours at 1.5 miles per hour, or 4*1.5=<<4*1.5=6>>6 miles.\nThus, altogether, she travels 12+6=<<12+6=18>>18 miles.\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 225, "spec_accept_length": 7.03125, "spec_accept_length_from_histogram": 7.0625, "spec_accept_rate": 0.4041666666666667, "spec_accepted_drafts": 194, "spec_proposed_drafts": 480, "spec_accept_histogram": [0, 3, 2, 4, 6, 3, 2, 2, 2, 2, 2, 0, 1, 0, 1, 2], "spec_verify_ct": 32, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "f1860404c1d845f99c25cc042f49482d", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 740, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170908.3806145, "queue_time": 0.0004875659942626953, "reasoning_tokens": 0, "completion_tokens": 225, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 225}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.6847480637952685, "spec_accept_rate": 0.4041666666666667, "spec_accept_length": 7.03125, "spec_num_correct_drafts": 194, "spec_num_proposed_drafts": 480, "spec_verify_ct": 32, "spec_accepted_drafts": 194, "spec_proposed_drafts": 480, "spec_correct_drafts_histogram": [0, 3, 2, 4, 6, 3, 2, 2, 2, 2, 2, 0, 1, 0, 1, 2], "spec_accept_histogram": [0, 3, 2, 4, 6, 3, 2, 2, 2, 2, 2, 0, 1, 0, 1, 2], "request_received_ts": 1788170906.9674482, "api_server_dispatch_finish_ts": 1788170906.9708724, "request_finished_ts": 1788170911.6521964, "decode_throughput": 734.5027528778522, "response_sent_to_client_ts": 1788170911.6527364}}
|
||||
{"prompt_id": 40, "output": " Ben’s iPhone is 1*2 = <<1*2=2>>2 years old.\nBrandon’s iPhone is 4*2 = <<4*2=8>>8 years old.\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 48, "spec_accept_length": 6.857142857142857, "spec_accept_length_from_histogram": 6.714285714285714, "spec_accept_rate": 0.38095238095238093, "spec_accepted_drafts": 40, "spec_proposed_drafts": 105, "spec_accept_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3e4111ce49e44adabcc227370f1f9645", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 718, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170913.0856385, "queue_time": 0.0003712177276611328, "reasoning_tokens": 0, "completion_tokens": 48, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 48}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.1347756078466773, "spec_accept_rate": 0.38095238095238093, "spec_accept_length": 6.857142857142857, "spec_num_correct_drafts": 40, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 40, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "request_received_ts": 1788170911.6645515, "api_server_dispatch_finish_ts": 1788170911.6679451, "request_finished_ts": 1788170913.799327, "decode_throughput": 13632054.491626147, "response_sent_to_client_ts": 1788170913.7998452}}
|
||||
{"prompt_id": 41, "output": " With the gemstone, Polly could throw the javelin 3 times farther than 400 feet, for a distance of 3*400=<<3*400=1200>>1200 feet.\n1200 feet is beyond the 1000-foot reach of the dragon's flames by a distance of 1200-1000=<<1200-1000=200>>200 feet.\n#### 200\n\n", "correct": true, "error": null, "completion_tokens": 86, "spec_accept_length": 6.615384615384615, "spec_accept_length_from_histogram": 6.6923076923076925, "spec_accept_rate": 0.37948717948717947, "spec_accepted_drafts": 74, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "6ff0240e27714c2887242f8eecd0c728", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 794, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170915.2833483, "queue_time": 0.00030422210693359375, "reasoning_tokens": 0, "completion_tokens": 86, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 86}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.812144835945219, "spec_accept_rate": 0.37948717948717947, "spec_accept_length": 6.615384615384615, "spec_num_correct_drafts": 74, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 74, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "spec_accept_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "request_received_ts": 1788170913.811771, "api_server_dispatch_finish_ts": 1788170913.81557, "request_finished_ts": 1788170916.623916, "decode_throughput": 51259789.40746981, "response_sent_to_client_ts": 1788170916.624199}}
|
||||
{"prompt_id": 42, "output": " To start the evening, there were 5 pies, each with 8 pieces, which is 5*8=<<5*8=40>>40 pieces of pie.\nIf only 14 remained, then 40-14=<<40-14=26>>26 pieces of pie had been taken by guests.\n#### 26\n\n", "correct": true, "error": null, "completion_tokens": 70, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.1, "spec_accept_rate": 0.4066666666666667, "spec_accepted_drafts": 61, "spec_proposed_drafts": 150, "spec_accept_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "spec_verify_ct": 10, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "aa0c8ad83c654ce49999421aafd5be58", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 746, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170918.042095, "queue_time": 0.0004487037658691406, "reasoning_tokens": 0, "completion_tokens": 70, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 70}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.4132698122411966, "spec_accept_rate": 0.4066666666666667, "spec_accept_length": 7.0, "spec_num_correct_drafts": 61, "spec_num_proposed_drafts": 150, "spec_verify_ct": 10, "spec_accepted_drafts": 61, "spec_proposed_drafts": 150, "spec_correct_drafts_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "spec_accept_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "request_received_ts": 1788170916.634394, "api_server_dispatch_finish_ts": 1788170916.6367202, "request_finished_ts": 1788170919.0476637, "decode_throughput": 16578247.002908928, "response_sent_to_client_ts": 1788170919.048161}}
|
||||
{"prompt_id": 43, "output": " If the total calorie target is 2000 and I have consumed 1800 calories then I have 2000-1800 = <<2000-1800=200>>200 calories left to eat\nIf each serving of chips has 250 calories and I only have 200 calories left to eat, then I can only eat 200/250 of a serving = 4/5 of a serving\nWe also know that a 300g bag of chips has 5 servings, hence each serving has 300g/5 = <<300/5=60>>60 grams\nIf I can only eat 4/5 of a serving, then I can eat only 60g * 4/5 = 240g/5 = 48 grams\n#### 48\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.1923076923076925, "spec_accept_length_from_histogram": 6.230769230769231, "spec_accept_rate": 0.3487179487179487, "spec_accepted_drafts": 136, "spec_proposed_drafts": 390, "spec_accept_histogram": [0, 1, 5, 3, 3, 1, 5, 3, 2, 1, 0, 1, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "fbd524aa363b49a289cb675e65ac87fc", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 721, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170920.4506898, "queue_time": 0.0002930164337158203, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.021148658823222, "spec_accept_rate": 0.3487179487179487, "spec_accept_length": 6.1923076923076925, "spec_num_correct_drafts": 136, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 136, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [0, 1, 5, 3, 3, 1, 5, 3, 2, 1, 0, 1, 1], "spec_accept_histogram": [0, 1, 5, 3, 3, 1, 5, 3, 2, 1, 0, 1, 1], "request_received_ts": 1788170919.0589588, "api_server_dispatch_finish_ts": 1788170919.062263, "request_finished_ts": 1788170923.0801075, "decode_throughput": 131.50150367354524, "response_sent_to_client_ts": 1788170923.0807073}}
|
||||
{"prompt_id": 44, "output": " 1 pound of beeswax and the wick make 10 candles and he wants to make 20 candles so he needs 20/10 = <<20/10=2>>2 pounds of beeswax\n1 pound of beeswax costs $10.00 and he needs 2 pounds so his supplies will cost 10*2 = $<<10*2=20.00>>20.00\nHe wants to sell each candle for $2.00 each and he's selling 20 candles so that's 2*20 = $<<2*20=40.00>>40.00\nIf he makes $40.00 selling candles and his supplies cost $20.00 then his net profit is 40-20 = $<<40-20=20.00>>20.00\n#### 20\n\n", "correct": true, "error": null, "completion_tokens": 168, "spec_accept_length": 6.222222222222222, "spec_accept_length_from_histogram": 6.333333333333333, "spec_accept_rate": 0.35555555555555557, "spec_accepted_drafts": 144, "spec_proposed_drafts": 405, "spec_accept_histogram": [1, 0, 6, 4, 1, 0, 6, 3, 1, 2, 1, 1, 1], "spec_verify_ct": 27, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "4dd241c6b9714cbc893f134ecd562359", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 737, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170924.4980018, "queue_time": 0.0002124309539794922, "reasoning_tokens": 0, "completion_tokens": 168, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 168}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.110803227871656, "spec_accept_rate": 0.35555555555555557, "spec_accept_length": 6.222222222222222, "spec_num_correct_drafts": 144, "spec_num_proposed_drafts": 405, "spec_verify_ct": 27, "spec_accepted_drafts": 144, "spec_proposed_drafts": 405, "spec_correct_drafts_histogram": [1, 0, 6, 4, 1, 0, 6, 3, 1, 2, 1, 1, 1], "spec_accept_histogram": [1, 0, 6, 4, 1, 0, 6, 3, 1, 2, 1, 1, 1], "request_received_ts": 1788170923.0916774, "api_server_dispatch_finish_ts": 1788170923.0951164, "request_finished_ts": 1788170927.2024808, "decode_throughput": 37647466.85030443, "response_sent_to_client_ts": 1788170927.203062}}
|
||||
{"prompt_id": 45, "output": " If she wrote five articles on Monday, then on Tuesday she wrote 2/5* 5 = 2 more articles.\nThe total number of articles she wrote on Tuesday is 5+2 = <<5+2=7>>7\nOn Wednesday, the number of articles was twice the number she wrote on Tuesday, so it was 2*7 = <<2*7=14>>14 articles.\nOver the three days, she wrote 5+7 +14 = <<5+7+14=26>>26 articles.\nIf each article takes her 4 hours to research and write about, she spent 4*26 = <<4*26=104>>104 hours on all the articles.\n#### 104\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 6.25, "spec_accept_length_from_histogram": 6.291666666666667, "spec_accept_rate": 0.3527777777777778, "spec_accepted_drafts": 127, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 1, 2, 4, 5, 3, 0, 3, 3, 1, 1, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c430218e6d0b43acaa3bc6df0939f90f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 760, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170928.6509275, "queue_time": 0.00034546852111816406, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.8713552551344037, "spec_accept_rate": 0.3527777777777778, "spec_accept_length": 6.25, "spec_num_correct_drafts": 127, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 127, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 1, 2, 4, 5, 3, 0, 3, 3, 1, 1, 1], "spec_accept_histogram": [0, 1, 2, 4, 5, 3, 0, 3, 3, 1, 1, 1], "request_received_ts": 1788170927.2145765, "api_server_dispatch_finish_ts": 1788170927.2181273, "request_finished_ts": 1788170931.0859318, "decode_throughput": 98.2484840542105, "response_sent_to_client_ts": 1788170931.0865207}}
|
||||
{"prompt_id": 46, "output": " Let's let x be the number of post-it notes in the package that Candice purchased.\nThen, the total number of post-it notes she had when she left the store was 80+x.\nAnd if she used 220 Post-it notes at work and had 23 left, then the total number she had when she left the store was 220+23=243.\nSubtracting 80 from 243 gives 163 post-it notes.\nThe number of post-it notes in the package was 163.\n#### 163\n\n", "correct": true, "error": null, "completion_tokens": 107, "spec_accept_length": 4.458333333333333, "spec_accept_length_from_histogram": 4.5, "spec_accept_rate": 0.23333333333333334, "spec_accepted_drafts": 84, "spec_proposed_drafts": 360, "spec_accept_histogram": [1, 7, 1, 5, 1, 4, 1, 2, 2], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "73459aa2681b428fb6c2d0773635d798", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 757, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170932.5309951, "queue_time": 0.0003325939178466797, "reasoning_tokens": 0, "completion_tokens": 107, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 107}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.8755744202062488, "spec_accept_rate": 0.23333333333333334, "spec_accept_length": 4.458333333333333, "spec_num_correct_drafts": 84, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 84, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [1, 7, 1, 5, 1, 4, 1, 2, 2], "spec_accept_histogram": [1, 7, 1, 5, 1, 4, 1, 2, 2], "request_received_ts": 1788170931.0979564, "api_server_dispatch_finish_ts": 1788170931.101428, "request_finished_ts": 1788170934.9735308, "decode_throughput": 69.89575710612316, "response_sent_to_client_ts": 1788170934.9739356}}
|
||||
{"prompt_id": 47, "output": " He bought 200 / 40 = <<200/40=5>>5 blue ties\nSo he bought 5 * 2 = <<5*2=10>>10 red ties\nEach red tie cost $40 * .5 = $<<40*.5=20>>20 more than blue ties\nSo they each cost $40 + $20 = $<<40+20=60>>60\nThus he spent $60 * 10 = $<<60*10=600>>600 on red ties\nSo he spent $200 + $600 = $<<200+600=800>>800 on ties\n#### 800\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 5.954545454545454, "spec_accept_length_from_histogram": 6.0, "spec_accept_rate": 0.3333333333333333, "spec_accepted_drafts": 110, "spec_proposed_drafts": 330, "spec_accept_histogram": [0, 0, 4, 5, 2, 2, 2, 3, 2, 1, 0, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ae4ca2248a9246bd8c19996f1bd17e9a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 714, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170936.397959, "queue_time": 0.00032448768615722656, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.644868925213814, "spec_accept_rate": 0.3333333333333333, "spec_accept_length": 5.954545454545454, "spec_num_correct_drafts": 110, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 110, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [0, 0, 4, 5, 2, 2, 2, 3, 2, 1, 0, 1], "spec_accept_histogram": [0, 0, 4, 5, 2, 2, 2, 3, 2, 1, 0, 1], "request_received_ts": 1788170934.9867418, "api_server_dispatch_finish_ts": 1788170934.989878, "request_finished_ts": 1788170938.6316106, "decode_throughput": 34757578.964143425, "response_sent_to_client_ts": 1788170938.632176}}
|
||||
{"prompt_id": 48, "output": " If Tracy used a piece of wire 4 feet long, since 1 foot is equal to 12 inches, the total length of the wire in inches was 12*4=<<12*4=48>>48 inches.\nSince the wire was cut into pieces 6 inches long, Tracy obtained 48/6=<<48/6=8>>8 pieces\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 81, "spec_accept_length": 6.230769230769231, "spec_accept_length_from_histogram": 6.3076923076923075, "spec_accept_rate": 0.35384615384615387, "spec_accepted_drafts": 69, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "602cef132db241ce80687ec47e0ce055", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 704, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170940.0062568, "queue_time": 0.0003838539123535156, "reasoning_tokens": 0, "completion_tokens": 81, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 81}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.686933293938637, "spec_accept_rate": 0.35384615384615387, "spec_accept_length": 6.230769230769231, "spec_num_correct_drafts": 69, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 69, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "request_received_ts": 1788170938.6451051, "api_server_dispatch_finish_ts": 1788170938.6483345, "request_finished_ts": 1788170941.3320384, "decode_throughput": 19797037.54782208, "response_sent_to_client_ts": 1788170941.332557}}
|
||||
{"prompt_id": 49, "output": " The total number of units in the building will be 8 units/floor * 15 floors = <<8*15=120>>120 units.\nIf 3/4 of the building is occupied, then the total number of occupied units is 3/4 * 120 units = <<3/4*120=90>>90 units.\nThe total number of unoccupied units is 120 units - 90 units = <<120-90=30>>30 units.\n#### 30\n\n", "correct": true, "error": null, "completion_tokens": 101, "spec_accept_length": 7.214285714285714, "spec_accept_length_from_histogram": 7.285714285714286, "spec_accept_rate": 0.41904761904761906, "spec_accepted_drafts": 88, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 2, 1, 2, 1, 0, 1, 3, 0, 1, 0, 1, 0, 1, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "fc378a26e81346f682f5454ecab64b2c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170942.768243, "queue_time": 0.00035500526428222656, "reasoning_tokens": 0, "completion_tokens": 101, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 101}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.820067374035716, "spec_accept_rate": 0.41904761904761906, "spec_accept_length": 7.214285714285714, "spec_num_correct_drafts": 88, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 88, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 2, 1, 2, 1, 0, 1, 3, 0, 1, 0, 1, 0, 1, 1], "spec_accept_histogram": [0, 2, 1, 2, 1, 0, 1, 3, 0, 1, 0, 1, 0, 1, 1], "request_received_ts": 1788170941.34332, "api_server_dispatch_finish_ts": 1788170941.3466604, "request_finished_ts": 1788170944.1633873, "decode_throughput": 27419351.991828397, "response_sent_to_client_ts": 1788170944.1639774}}
|
||||
{"prompt_id": 50, "output": " In a week, Lloyd's egg farm produces 252 x 7 = <<252*7=1764>>1764 eggs.\nThere are 1764/12 = <<1764/12=147>>147 dozens of eggs produced in a week.\nSo, Lloyd makes 147 x $2 = $<<147*2=294>>294 on eggs.\n#### 294\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 5.642857142857143, "spec_accept_length_from_histogram": 5.714285714285714, "spec_accept_rate": 0.3142857142857143, "spec_accepted_drafts": 66, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "d9b5adec79cd4136b8cf7cfc45f3b6b5", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 702, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170945.5427258, "queue_time": 0.00029969215393066406, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.7703654430806637, "spec_accept_rate": 0.3142857142857143, "spec_accept_length": 5.642857142857143, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 66, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "spec_accept_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "request_received_ts": 1788170944.1768603, "api_server_dispatch_finish_ts": 1788170944.1800888, "request_finished_ts": 1788170946.9472258, "decode_throughput": 19815890.754051816, "response_sent_to_client_ts": 1788170946.9478188}}
|
||||
{"prompt_id": 51, "output": " He was travelling at full speed for 4-1=<<4-1=3>>3 hours\nSo he went 3*10=<<3*10=30>>30 miles\nThat means it takes 30/6=<<30/6=5>>5 hours to get back\n#### 5\n\n", "correct": true, "error": null, "completion_tokens": 67, "spec_accept_length": 6.090909090909091, "spec_accept_length_from_histogram": 6.181818181818182, "spec_accept_rate": 0.34545454545454546, "spec_accepted_drafts": 57, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "cb56dc2d5917400cb9bfca40bf752c63", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 716, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170948.3656008, "queue_time": 0.0003681182861328125, "reasoning_tokens": 0, "completion_tokens": 67, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 67}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.5290144341997802, "spec_accept_rate": 0.34545454545454546, "spec_accept_length": 6.090909090909091, "spec_num_correct_drafts": 57, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 57, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "spec_accept_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "request_received_ts": 1788170946.9586985, "api_server_dispatch_finish_ts": 1788170946.9620543, "request_finished_ts": 1788170949.487713, "decode_throughput": 18206027.073603082, "response_sent_to_client_ts": 1788170949.4882185}}
|
||||
{"prompt_id": 52, "output": " 30 comic books weigh 7.5 pounds because 30 x .25 = <<30*.25=7.5>>7.5\nHe needs to remove 7.5 more pounds because 15 - 7.5 = <<15-7.5=7.5>>7.5\nHe needs to remove 15 toys because 7.5 / .5 = <<15=15>>15\n#### 15\n\n", "correct": true, "error": null, "completion_tokens": 92, "spec_accept_length": 8.363636363636363, "spec_accept_length_from_histogram": 8.363636363636363, "spec_accept_rate": 0.4909090909090909, "spec_accepted_drafts": 81, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 0, 1, 0, 3, 1, 1, 2, 1, 1, 0, 0, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5dd6f40e1ce845c581cfd5b8b6bf1ea2", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170950.8897812, "queue_time": 0.00029587745666503906, "reasoning_tokens": 0, "completion_tokens": 92, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 92}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.497947982046753, "spec_accept_rate": 0.4909090909090909, "spec_accept_length": 8.363636363636363, "spec_num_correct_drafts": 81, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 81, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 0, 1, 0, 3, 1, 1, 2, 1, 1, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 0, 1, 0, 3, 1, 1, 2, 1, 1, 0, 0, 0, 0, 1], "request_received_ts": 1788170949.4994202, "api_server_dispatch_finish_ts": 1788170949.5027676, "request_finished_ts": 1788170951.9973683, "decode_throughput": 21503192.33802817, "response_sent_to_client_ts": 1788170951.9979472}}
|
||||
{"prompt_id": 53, "output": " On Thursday the mechanic charged a total of 6*60 + 4*40 = $<<6*60+4*40=520>>520.\nOn Friday the mechanic charged a total of 12*40 = $<<12*40=480>>480.\nThus, on Thursday, the day with higher revenue, the mechanic had 520-480 = $<<520-480=40>>40 more revenue than on Friday.\n#### 40\n\n", "correct": true, "error": null, "completion_tokens": 94, "spec_accept_length": 6.714285714285714, "spec_accept_length_from_histogram": 6.785714285714286, "spec_accept_rate": 0.38571428571428573, "spec_accepted_drafts": 81, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 0, 1, 4, 0, 2, 2, 1, 2, 1, 0, 0, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b29843a0cbe647879f8eb5b7e439cdee", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 764, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170953.4081776, "queue_time": 0.0003342628479003906, "reasoning_tokens": 0, "completion_tokens": 94, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 94}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.8082566410303116, "spec_accept_rate": 0.38571428571428573, "spec_accept_length": 6.714285714285714, "spec_num_correct_drafts": 81, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 81, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 0, 1, 4, 0, 2, 2, 1, 2, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 4, 0, 2, 2, 1, 2, 1, 0, 0, 0, 1], "request_received_ts": 1788170952.0086718, "api_server_dispatch_finish_ts": 1788170952.0121293, "request_finished_ts": 1788170954.8169284, "decode_throughput": 24775583.583178267, "response_sent_to_client_ts": 1788170954.8174517}}
|
||||
{"prompt_id": 54, "output": " Patchy has just had 3 * 7 = <<3*7=21>>21 kittens\nSince Trixie's kittens are 12, both cats have 21 +12 =<<21+12=33>>33 kittens.\nCombining the adopted kittens with those from the two cats, the Doubtfire family now has 7 + 33 = <<7+33=40>>40 kittens.\n#### 40\n\n", "correct": true, "error": null, "completion_tokens": 88, "spec_accept_length": 5.176470588235294, "spec_accept_length_from_histogram": 5.235294117647059, "spec_accept_rate": 0.2823529411764706, "spec_accepted_drafts": 72, "spec_proposed_drafts": 255, "spec_accept_histogram": [0, 3, 3, 2, 3, 0, 2, 2, 0, 1, 1], "spec_verify_ct": 17, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "4ac1598caf344caf8ad8402e5268b49f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 749, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170956.2303514, "queue_time": 0.00029850006103515625, "reasoning_tokens": 0, "completion_tokens": 88, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 88}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.0979787912219763, "spec_accept_rate": 0.2823529411764706, "spec_accept_length": 5.176470588235294, "spec_num_correct_drafts": 72, "spec_num_proposed_drafts": 255, "spec_verify_ct": 17, "spec_accepted_drafts": 72, "spec_proposed_drafts": 255, "spec_correct_drafts_histogram": [0, 3, 3, 2, 3, 0, 2, 2, 0, 1, 1], "spec_accept_histogram": [0, 3, 3, 2, 3, 0, 2, 2, 0, 1, 1], "request_received_ts": 1788170954.8284464, "api_server_dispatch_finish_ts": 1788170954.8319623, "request_finished_ts": 1788170957.9264252, "decode_throughput": 20752091.233588804, "response_sent_to_client_ts": 1788170957.9269996}}
|
||||
{"prompt_id": 55, "output": " Jean has 30 - 2 = <<30-2=28>>28 lollipops\nJean can fill up 28 / 2 = <<28/2=14>>14 bags with lollipops\n#### 14\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.25, "spec_accept_rate": 0.35, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 3, 2, 0, 0, 0, 0, 0, 1, 1, 1], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e65e52bbc78a4a169fbb49666c856d8b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 713, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170959.3429775, "queue_time": 0.00034999847412109375, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.1901370440609753, "spec_accept_rate": 0.35, "spec_accept_length": 6.125, "spec_num_correct_drafts": 42, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 3, 2, 0, 0, 0, 0, 0, 1, 1, 1], "spec_accept_histogram": [0, 0, 3, 2, 0, 0, 0, 0, 0, 1, 1, 1], "request_received_ts": 1788170957.9378848, "api_server_dispatch_finish_ts": 1788170957.9412148, "request_finished_ts": 1788170960.128022, "decode_throughput": 12630708.871951967, "response_sent_to_client_ts": 1788170960.1285446}}
|
||||
{"prompt_id": 56, "output": " It costs him $14 to go to the movies because 7 + 7 = <<7+7=14>>14\nHe can go to the movies 3 times because 42 / 14 = <<42/14=3>>3\n#### 3\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 6.222222222222222, "spec_accept_length_from_histogram": 6.333333333333333, "spec_accept_rate": 0.35555555555555557, "spec_accepted_drafts": 48, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "8c7b7a0f725d43a3a4cf1d0eb9135bda", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 712, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170961.5381656, "queue_time": 0.0002911090850830078, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.324792387895286, "spec_accept_rate": 0.35555555555555557, "spec_accept_length": 6.222222222222222, "spec_num_correct_drafts": 48, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 48, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "request_received_ts": 1788170960.13944, "api_server_dispatch_finish_ts": 1788170960.1427076, "request_finished_ts": 1788170962.4642324, "decode_throughput": 14508242.309298612, "response_sent_to_client_ts": 1788170962.4648404}}
|
||||
{"prompt_id": 57, "output": " The boxes can weigh up to 5000 pounds - 3755 pounds = <<5000-3755=1245>>1245 pounds in total.\nThere can be 1245 / 15 = <<1245/15=83>>83 boxes loaded onto the truck without exceeding the bridge's weight limit.\n#### 83\n\n", "correct": true, "error": null, "completion_tokens": 69, "spec_accept_length": 6.2727272727272725, "spec_accept_length_from_histogram": 6.363636363636363, "spec_accept_rate": 0.3575757575757576, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "dbdb51eb8318431eb15249464822bc30", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 740, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170963.8661215, "queue_time": 0.0003185272216796875, "reasoning_tokens": 0, "completion_tokens": 69, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 69}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.500556231942028, "spec_accept_rate": 0.3575757575757576, "spec_accept_length": 6.2727272727272725, "spec_num_correct_drafts": 59, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "request_received_ts": 1788170962.4756625, "api_server_dispatch_finish_ts": 1788170962.479111, "request_finished_ts": 1788170964.9762187, "decode_throughput": 17236648.732766762, "response_sent_to_client_ts": 1788170964.9767613}}
|
||||
{"prompt_id": 58, "output": " He spent $40.00 on groceries but they charged him a 25% fee so that's 40*.25 = $10.00\nThere is also a $3.00 delivery fee and a $4.00 tip for an extra $3 + $4 = $<<3+4=7.00>>7.00 in fees\nHis groceries were $40.00, there's a $10.00 fee and another $7.00 in delivery/tip fees for a final total of $40 + $10 + $7 = $<<40+10+7=57.00>>57.00\n#### 57\n\n", "correct": true, "error": null, "completion_tokens": 132, "spec_accept_length": 5.076923076923077, "spec_accept_length_from_histogram": 5.115384615384615, "spec_accept_rate": 0.2743589743589744, "spec_accepted_drafts": 107, "spec_proposed_drafts": 390, "spec_accept_histogram": [0, 5, 5, 4, 1, 4, 1, 4, 0, 0, 1, 0, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b632f959d3c44d589277669423e2cc07", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 747, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170966.426919, "queue_time": 0.0002925395965576172, "reasoning_tokens": 0, "completion_tokens": 132, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 132}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.059209485072643, "spec_accept_rate": 0.2743589743589744, "spec_accept_length": 5.076923076923077, "spec_num_correct_drafts": 107, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 107, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [0, 5, 5, 4, 1, 4, 1, 4, 0, 0, 1, 0, 1], "spec_accept_histogram": [0, 5, 5, 4, 1, 4, 1, 4, 0, 0, 1, 0, 1], "request_received_ts": 1788170964.9874096, "api_server_dispatch_finish_ts": 1788170964.9908442, "request_finished_ts": 1788170969.0466192, "decode_throughput": 94.08200305710004, "response_sent_to_client_ts": 1788170969.046887}}
|
||||
{"prompt_id": 59, "output": " First find the number of fruit in clusters: 20 fruit/cluster * 6 clusters = <<20*6=120>>120 fruit\nThen add the number of individual fruit to find the total number of fruit: 120 fruit + 67 fruit = <<120+67=187>>187 fruit\n#### 187\n\n", "correct": true, "error": null, "completion_tokens": 66, "spec_accept_length": 5.076923076923077, "spec_accept_length_from_histogram": 5.153846153846154, "spec_accept_rate": 0.27692307692307694, "spec_accepted_drafts": 54, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e47775fe69004910ae7b10397def434b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 697, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170970.4016008, "queue_time": 0.0003666877746582031, "reasoning_tokens": 0, "completion_tokens": 66, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 66}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.659180947113782, "spec_accept_rate": 0.27692307692307694, "spec_accept_length": 5.076923076923077, "spec_num_correct_drafts": 54, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 54, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "request_received_ts": 1788170969.0565703, "api_server_dispatch_finish_ts": 1788170969.0579855, "request_finished_ts": 1788170971.7157512, "decode_throughput": 331.9374122752028, "response_sent_to_client_ts": 1788170971.716194}}
|
||||
{"prompt_id": 60, "output": " 20% of 25 oranges is (20/100)*25 = <<(20/100)*25=5>>5 oranges\nThe total number of bad, unripe and sour oranges is therefore 1+5+2 = <<1+5+2=8>>8 oranges\nThe rest which is 25-8 = <<25-8=17>>17 oranges are good\n#### 17\n\n", "correct": true, "error": null, "completion_tokens": 86, "spec_accept_length": 5.375, "spec_accept_length_from_histogram": 5.5, "spec_accept_rate": 0.3, "spec_accepted_drafts": 72, "spec_proposed_drafts": 240, "spec_accept_histogram": [0, 3, 2, 3, 2, 2, 0, 0, 1, 2, 0, 0, 1], "spec_verify_ct": 16, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "bbbae42ec31544da9d4ae91d37d2a5ef", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 704, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170973.1517472, "queue_time": 0.0003097057342529297, "reasoning_tokens": 0, "completion_tokens": 86, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 86}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.0415330259129405, "spec_accept_rate": 0.3, "spec_accept_length": 5.375, "spec_num_correct_drafts": 72, "spec_num_proposed_drafts": 240, "spec_verify_ct": 16, "spec_accepted_drafts": 72, "spec_proposed_drafts": 240, "spec_correct_drafts_histogram": [0, 3, 2, 3, 2, 2, 0, 0, 1, 2, 0, 0, 1], "spec_accept_histogram": [0, 3, 2, 3, 2, 2, 0, 0, 1, 2, 0, 0, 1], "request_received_ts": 1788170971.7265978, "api_server_dispatch_finish_ts": 1788170971.7308564, "request_finished_ts": 1788170974.7681308, "decode_throughput": 21622377.40819711, "response_sent_to_client_ts": 1788170974.7687805}}
|
||||
{"prompt_id": 61, "output": " The total cost of the broch was 500+800=$<<500+800=1300>>1300\nThe insurance was 1300*.1=$<<1300*.1=130>>130\nSo she pays 1300+130=$<<1300+130=1430>>1430\n#### 1430\n\n", "correct": true, "error": null, "completion_tokens": 69, "spec_accept_length": 6.2727272727272725, "spec_accept_length_from_histogram": 6.363636363636363, "spec_accept_rate": 0.3575757575757576, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3ed92c224f8d4df99b1781424cb302cd", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 724, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170976.1704917, "queue_time": 0.00030684471130371094, "reasoning_tokens": 0, "completion_tokens": 69, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 69}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 2.513619842939079, "spec_accept_rate": 0.3575757575757576, "spec_accept_length": 6.2727272727272725, "spec_num_correct_drafts": 59, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "request_received_ts": 1788170974.7800016, "api_server_dispatch_finish_ts": 1788170974.7833724, "request_finished_ts": 1788170977.2936213, "decode_throughput": 14406954.228887135, "response_sent_to_client_ts": 1788170977.294123}}
|
||||
{"prompt_id": 62, "output": " First find how many years Marcy works after 20 years: 30 years - 20 years = <<30-20=10>>10 years\nThen multiply that number by the amount of her pension she gets per year: 10 years * 5% = 50%\nThen multiply that percentage by the total value of the pension to find how much she gets: $50,000 * 50% = $<<50000*50*.01=25000>>25,000\n#### 25000\n\n", "correct": true, "error": null, "completion_tokens": 106, "spec_accept_length": 4.076923076923077, "spec_accept_length_from_histogram": 4.115384615384615, "spec_accept_rate": 0.2076923076923077, "spec_accepted_drafts": 81, "spec_proposed_drafts": 390, "spec_accept_histogram": [1, 4, 5, 5, 5, 5, 0, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "785f7ea373de47eaadf4a082f85d3f82", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170978.7152352, "queue_time": 0.0004622936248779297, "reasoning_tokens": 0, "completion_tokens": 106, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 106}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.0503142033703625, "spec_accept_rate": 0.2076923076923077, "spec_accept_length": 4.076923076923077, "spec_num_correct_drafts": 81, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 81, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [1, 4, 5, 5, 5, 5, 0, 1], "spec_accept_histogram": [1, 4, 5, 5, 5, 5, 0, 1], "request_received_ts": 1788170977.304984, "api_server_dispatch_finish_ts": 1788170977.3082948, "request_finished_ts": 1788170981.3552983, "decode_throughput": 25227767.178339675, "response_sent_to_client_ts": 1788170981.3558595}}
|
||||
{"prompt_id": 63, "output": " A year has twelve months, so a half of the year is 12/2 = <<12/2=6>>6 months,\nFor the first six months, the streaming service charged her a total of 6*140 = $<<6*140=840>>840\nFor the next half of the year, the streaming service company charged her 10/100*140 = $<<10/100*140=14>>14 less\nShe paid $140-$14 = $<<140-14=126>>126 each month for the remaining half of the year,\nIn total, she paid 126*6 = $<<126*6=756>>756 for the other half of the year for the streaming service.\nThe total amount of money she paid at the end of the year is $840+$756 = $<<840+756=1596>>1596\n#### 1596\n\n", "correct": true, "error": null, "completion_tokens": 185, "spec_accept_length": 5.606060606060606, "spec_accept_length_from_histogram": 5.636363636363637, "spec_accept_rate": 0.3090909090909091, "spec_accepted_drafts": 153, "spec_proposed_drafts": 495, "spec_accept_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "spec_verify_ct": 33, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "37ec794090594520912cfa6a92abf5d2", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 734, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788170982.8566256, "queue_time": 0.00032067298889160156, "reasoning_tokens": 0, "completion_tokens": 185, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 185}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.821898098103702, "spec_accept_rate": 0.3090909090909091, "spec_accept_length": 5.606060606060606, "spec_num_correct_drafts": 153, "spec_num_proposed_drafts": 495, "spec_verify_ct": 33, "spec_accepted_drafts": 153, "spec_proposed_drafts": 495, "spec_correct_drafts_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "spec_accept_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "request_received_ts": 1788170981.3679082, "api_server_dispatch_finish_ts": 1788170981.3712585, "request_finished_ts": 1788170986.1898065, "decode_throughput": 140.27188270750304, "response_sent_to_client_ts": 1788170986.1903255}}
|
||||
@ -0,0 +1,28 @@
|
||||
gsm8k_c1 174.1.60.1: docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/dataset/gsm8k.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /dataset/gsm8k.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 1 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c1.jsonl --summary-file /results/bench/gsm8k_c1_summary.json
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/awq/awq.py:53: UserWarning: Only CUDA, HIP and XPU support AWQ currently.
|
||||
warnings.warn(f"Only CUDA, HIP and XPU support AWQ currently.")
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/gguf.py:69: UserWarning: Only CUDA, MUSA and NPU support GGUF quantization currently.
|
||||
warnings.warn(f"Only CUDA, MUSA and NPU support GGUF quantization currently.")
|
||||
0%| | 0/64 [00:00<?, ?it/s]
2%|▏ | 1/64 [00:02<02:10, 2.07s/it]
3%|▎ | 2/64 [00:03<01:57, 1.89s/it]
5%|▍ | 3/64 [00:06<02:09, 2.12s/it]
6%|▋ | 4/64 [00:07<01:58, 1.98s/it]
8%|▊ | 5/64 [00:10<02:00, 2.05s/it]
9%|▉ | 6/64 [00:13<02:28, 2.56s/it]
11%|█ | 7/64 [00:16<02:31, 2.66s/it]
12%|█▎ | 8/64 [00:20<02:53, 3.10s/it]
14%|█▍ | 9/64 [00:24<03:04, 3.35s/it]
16%|█▌ | 10/64 [00:28<03:11, 3.54s/it]
17%|█▋ | 11/64 [00:32<03:09, 3.58s/it]
19%|█▉ | 12/64 [00:35<02:55, 3.37s/it]
20%|██ | 13/64 [00:38<02:51, 3.36s/it]
22%|██▏ | 14/64 [00:42<02:53, 3.47s/it]
23%|██▎ | 15/64 [00:45<02:48, 3.44s/it]
25%|██▌ | 16/64 [00:48<02:41, 3.36s/it]
27%|██▋ | 17/64 [00:52<02:38, 3.38s/it]
28%|██▊ | 18/64 [00:56<02:43, 3.55s/it]
30%|██▉ | 19/64 [00:59<02:33, 3.40s/it]
31%|███▏ | 20/64 [01:03<02:46, 3.78s/it]
33%|███▎ | 21/64 [01:07<02:41, 3.74s/it]
34%|███▍ | 22/64 [01:09<02:18, 3.30s/it]
36%|███▌ | 23/64 [01:11<02:03, 3.00s/it]
38%|███▊ | 24/64 [01:14<01:50, 2.77s/it]
39%|███▉ | 25/64 [01:16<01:45, 2.71s/it]
41%|████ | 26/64 [01:20<01:50, 2.90s/it]
42%|████▏ | 27/64 [01:22<01:38, 2.67s/it]
44%|████▍ | 28/64 [01:25<01:39, 2.75s/it]
45%|████▌ | 29/64 [01:27<01:32, 2.65s/it]
47%|████▋ | 30/64 [01:30<01:29, 2.62s/it]
48%|████▊ | 31/64 [01:33<01:32, 2.79s/it]
50%|█████ | 32/64 [01:36<01:32, 2.89s/it]
52%|█████▏ | 33/64 [01:38<01:22, 2.67s/it]
53%|█████▎ | 34/64 [01:41<01:20, 2.69s/it]
55%|█████▍ | 35/64 [01:43<01:17, 2.68s/it]
56%|█████▋ | 36/64 [01:46<01:12, 2.58s/it]
58%|█████▊ | 37/64 [01:49<01:18, 2.89s/it]
59%|█████▉ | 38/64 [01:52<01:15, 2.89s/it]
61%|██████ | 39/64 [01:56<01:15, 3.02s/it]
62%|██████▎ | 40/64 [02:00<01:24, 3.52s/it]
64%|██████▍ | 41/64 [02:03<01:11, 3.11s/it]
66%|██████▌ | 42/64 [02:05<01:06, 3.02s/it]
67%|██████▋ | 43/64 [02:08<00:59, 2.84s/it]
69%|██████▉ | 44/64 [02:12<01:04, 3.20s/it]
70%|███████ | 45/64 [02:16<01:06, 3.48s/it]
72%|███████▏ | 46/64 [02:20<01:04, 3.60s/it]
73%|███████▎ | 47/64 [02:24<01:02, 3.69s/it]
75%|███████▌ | 48/64 [02:27<00:58, 3.68s/it]
77%|███████▋ | 49/64 [02:30<00:50, 3.38s/it]
78%|███████▊ | 50/64 [02:33<00:45, 3.22s/it]
80%|███████▉ | 51/64 [02:36<00:40, 3.09s/it]
81%|████████▏ | 52/64 [02:38<00:35, 2.92s/it]
83%|████████▎ | 53/64 [02:41<00:30, 2.80s/it]
84%|████████▍ | 54/64 [02:44<00:28, 2.81s/it]
86%|████████▌ | 55/64 [02:47<00:26, 2.90s/it]
88%|████████▊ | 56/64 [02:49<00:21, 2.69s/it]
89%|████████▉ | 57/64 [02:51<00:18, 2.58s/it]
91%|█████████ | 58/64 [02:54<00:15, 2.56s/it]
92%|█████████▏| 59/64 [02:58<00:15, 3.01s/it]
94%|█████████▍| 60/64 [03:00<00:11, 2.91s/it]
95%|█████████▌| 61/64 [03:03<00:08, 2.95s/it]
97%|█████████▋| 62/64 [03:06<00:05, 2.82s/it]
98%|█████████▊| 63/64 [03:10<00:03, 3.20s/it]
100%|██████████| 64/64 [03:15<00:00, 3.69s/it]
100%|██████████| 64/64 [03:15<00:00, 3.05s/it]
|
||||
{
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 1,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 6.943340536821033,
|
||||
"median_accept_length_per_question": 6.2727272727272725,
|
||||
"min_accept_length_per_question": 4.076923076923077,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 195.41776519827545,
|
||||
"output_throughput": 33.379769712245
|
||||
}
|
||||
@ -0,0 +1,22 @@
|
||||
{
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 1,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 6.943340536821033,
|
||||
"median_accept_length_per_question": 6.2727272727272725,
|
||||
"min_accept_length_per_question": 4.076923076923077,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 195.41776519827545,
|
||||
"output_throughput": 33.379769712245
|
||||
}
|
||||
@ -0,0 +1,64 @@
|
||||
{"prompt_id": 0, "output": " Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.\nShe makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market.\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 14.0, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 60, "spec_proposed_drafts": 60, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "spec_verify_ct": 4, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "848073136d5b44ae9c495cd42713e46a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 730, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171023.1028814, "queue_time": 0.0005366802215576172, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.1312459097243845, "spec_accept_rate": 1.0, "spec_accept_length": 14.0, "spec_num_correct_drafts": 60, "spec_num_proposed_drafts": 60, "spec_verify_ct": 4, "spec_accepted_drafts": 60, "spec_proposed_drafts": 60, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4], "request_received_ts": 1788171020.7597132, "api_server_dispatch_finish_ts": 1788171020.76377, "request_finished_ts": 1788171024.8909593, "decode_throughput": 11111157.162746944, "response_sent_to_client_ts": 1788171024.8915987}}
|
||||
{"prompt_id": 1, "output": " It takes 2/2=<<2/2=1>>1 bolt of white fiber\nSo the total amount of fabric is 2+1=<<2+1=3>>3 bolts of fabric\n#### 3\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 16.333333333333332, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 45, "spec_proposed_drafts": 45, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "spec_verify_ct": 3, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "0acc787b45ac41ecaddf8af7ce2b017a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 694, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171023.33857, "queue_time": 0.00016498565673828125, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.07897964399308, "spec_accept_rate": 1.0, "spec_accept_length": 16.333333333333332, "spec_num_correct_drafts": 45, "spec_num_proposed_drafts": 45, "spec_verify_ct": 3, "spec_accepted_drafts": 45, "spec_proposed_drafts": 45, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "request_received_ts": 1788171020.766183, "api_server_dispatch_finish_ts": 1788171020.7694356, "request_finished_ts": 1788171025.8451626, "decode_throughput": 9327591.630078726, "response_sent_to_client_ts": 1788171025.8457088}}
|
||||
{"prompt_id": 2, "output": " The cost of the house and repairs came out to 80,000+50,000=$<<80000+50000=130000>>130,000\nHe increased the value of the house by 80,000*1.5=<<80000*1.5=120000>>120,000\nSo the new value of the house is 120,000+80,000=$<<120000+80000=200000>>200,000\nSo he made a profit of 200,000-130,000=$<<200000-130000=70000>>70,000\n#### 70000\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 14.555555555555555, "spec_accept_length_from_histogram": 14.666666666666666, "spec_accept_rate": 0.9111111111111111, "spec_accepted_drafts": 123, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e985a8f76559406e80304386ce2a7a0f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587296, "queue_time": 0.0016186237335205078, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.843204802367836, "spec_accept_rate": 0.9111111111111111, "spec_accept_length": 14.555555555555555, "spec_num_correct_drafts": 123, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 123, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 7], "request_received_ts": 1788171020.7704744, "api_server_dispatch_finish_ts": 1788171020.7735264, "request_finished_ts": 1788171027.6136792, "decode_throughput": 34461532.43303296, "response_sent_to_client_ts": 1788171027.6141963}}
|
||||
{"prompt_id": 3, "output": " He sprints 3*3=<<3*3=9>>9 times\nSo he runs 9*60=<<9*60=540>>540 meters\n#### 540\n\n", "correct": true, "error": null, "completion_tokens": 41, "spec_accept_length": 13.666666666666666, "spec_accept_length_from_histogram": 14.0, "spec_accept_rate": 0.8666666666666667, "spec_accepted_drafts": 39, "spec_proposed_drafts": 45, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "spec_verify_ct": 3, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "f926e87789ac4c85ae7262439c13f475", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 702, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587296, "queue_time": 0.0013353824615478516, "reasoning_tokens": 0, "completion_tokens": 41, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 41}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.413379465695471, "spec_accept_rate": 0.8666666666666667, "spec_accept_length": 13.666666666666666, "spec_num_correct_drafts": 39, "spec_num_proposed_drafts": 45, "spec_verify_ct": 3, "spec_accepted_drafts": 39, "spec_proposed_drafts": 45, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2], "request_received_ts": 1788171020.7783697, "api_server_dispatch_finish_ts": 1788171020.7815278, "request_finished_ts": 1788171026.191749, "decode_throughput": 12354285.332949806, "response_sent_to_client_ts": 1788171026.1922545}}
|
||||
{"prompt_id": 4, "output": " If each chicken eats 3 cups of feed per day, then for 20 chickens they would need 3*20=<<3*20=60>>60 cups of feed per day.\nIf she feeds the flock 15 cups of feed in the morning, and 25 cups in the afternoon, then the final meal would require 60-15-25=<<60-15-25=20>>20 cups of chicken feed.\n#### 20\n\n", "correct": true, "error": null, "completion_tokens": 94, "spec_accept_length": 15.666666666666666, "spec_accept_length_from_histogram": 16.0, "spec_accept_rate": 1.0, "spec_accepted_drafts": 90, "spec_proposed_drafts": 90, "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "spec_verify_ct": 6, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "f07da8d9d24a4aabb25d0f8b0fdf4990", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 775, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587294, "queue_time": 0.0011067390441894531, "reasoning_tokens": 0, "completion_tokens": 94, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 94}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.137197333853692, "spec_accept_rate": 1.0, "spec_accept_length": 15.666666666666666, "spec_num_correct_drafts": 90, "spec_num_proposed_drafts": 90, "spec_verify_ct": 6, "spec_accepted_drafts": 90, "spec_proposed_drafts": 90, "spec_correct_drafts_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "spec_accept_histogram": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6], "request_received_ts": 1788171020.783438, "api_server_dispatch_finish_ts": 1788171020.7867067, "request_finished_ts": 1788171026.9206352, "decode_throughput": 23366792.94068094, "response_sent_to_client_ts": 1788171026.9211564}}
|
||||
{"prompt_id": 5, "output": " The discount price of one glass is 60/100 * 5 = $<<60/100*5=3>>3.\nIf every second glass is cheaper, that means Kylar is going to buy 16 / 2 = <<16/2=8>>8 cheaper glasses.\nSo for the cheaper glasses, Kylar is going to pay 8 * 3 = $<<8*3=24>>24.\nAnd for the regular-priced glasses, Kylar will pay 8 * 5 = $<<8*5=40>>40.\nSo in total Kylar needs to pay 24 + 40 = $<<24+40=64>>64 for the glasses he wants to buy.\n#### 64\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 7.142857142857143, "spec_accept_length_from_histogram": 7.190476190476191, "spec_accept_rate": 0.4126984126984127, "spec_accepted_drafts": 130, "spec_proposed_drafts": 315, "spec_accept_histogram": [0, 0, 0, 6, 4, 1, 2, 2, 0, 2, 1, 0, 2, 1], "spec_verify_ct": 21, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "09598fb30f674038b87fe902d127895a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 720, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587298, "queue_time": 0.0008935928344726562, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.352511872071773, "spec_accept_rate": 0.4126984126984127, "spec_accept_length": 7.142857142857143, "spec_num_correct_drafts": 130, "spec_num_proposed_drafts": 315, "spec_verify_ct": 21, "spec_accepted_drafts": 130, "spec_proposed_drafts": 315, "spec_correct_drafts_histogram": [0, 0, 0, 6, 4, 1, 2, 2, 0, 2, 1, 0, 2, 1], "spec_accept_histogram": [0, 0, 0, 6, 4, 1, 2, 2, 0, 2, 1, 0, 2, 1], "request_received_ts": 1788171020.7881374, "api_server_dispatch_finish_ts": 1788171020.7912104, "request_finished_ts": 1788171030.1406493, "decode_throughput": 45986643.22391492, "response_sent_to_client_ts": 1788171030.1411421}}
|
||||
{"prompt_id": 6, "output": " If Seattle has 20 sheep, Charleston has 4 * 20 sheep = <<20*4=80>>80 sheep\nToulouse has twice as many sheep as Charleston, which is 2 * 80 sheep = <<2*80=160>>160 sheep\nTogether, the three has 20 sheep + 160 sheep + 80 sheep = <<20+160+80=260>>260 sheep\n#### 260\n\n", "correct": true, "error": null, "completion_tokens": 90, "spec_accept_length": 6.428571428571429, "spec_accept_length_from_histogram": 6.428571428571429, "spec_accept_rate": 0.3619047619047619, "spec_accepted_drafts": 76, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "0d1651250fe3410ba33665048ed44256", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587296, "queue_time": 0.0006866455078125, "reasoning_tokens": 0, "completion_tokens": 90, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 90}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.772955406922847, "spec_accept_rate": 0.3619047619047619, "spec_accept_length": 6.428571428571429, "spec_num_correct_drafts": 76, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 76, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 2, 3, 2, 0, 0, 2, 2, 0, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788171020.7930553, "api_server_dispatch_finish_ts": 1788171020.79599, "request_finished_ts": 1788171028.5660107, "decode_throughput": 27233691.17583357, "response_sent_to_client_ts": 1788171028.5665023}}
|
||||
{"prompt_id": 7, "output": " First find how many gigabytes are in 40% of the file: 200 GB * 40% = <<200*40*.01=80>>80 GB\nThen divide that number by the download rate to find the time until Windows restarts: 80 GB / 2 GB/minute = <<80/2=40>>40 minutes\nThen find the time to download the whole file after the restart: 200 GB / 2 GB/minute = <<200/2=100>>100 minutes\nThen add the time to download 40% of the file, to download the whole file, and to wait for Windows to update: 40 minutes + 100 minutes + 20 minutes = <<40+100+20=160>>160 minutes\n#### 160\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.44, "spec_accept_length_from_histogram": 6.44, "spec_accept_rate": 0.3626666666666667, "spec_accepted_drafts": 136, "spec_proposed_drafts": 375, "spec_accept_histogram": [0, 1, 6, 6, 0, 3, 1, 1, 2, 0, 1, 1, 1, 1, 0, 1], "spec_verify_ct": 25, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ff09165a10204fd3aa5ec5f4ab8eaa54", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 733, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171024.3587298, "queue_time": 0.0004792213439941406, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 10.349999848753214, "spec_accept_rate": 0.3626666666666667, "spec_accept_length": 6.44, "spec_num_correct_drafts": 136, "spec_num_proposed_drafts": 375, "spec_verify_ct": 25, "spec_accepted_drafts": 136, "spec_proposed_drafts": 375, "spec_correct_drafts_histogram": [0, 1, 6, 6, 0, 3, 1, 1, 2, 0, 1, 1, 1, 1, 0, 1], "spec_accept_histogram": [0, 1, 6, 6, 0, 3, 1, 1, 2, 0, 1, 1, 1, 1, 0, 1], "request_received_ts": 1788171020.7974195, "api_server_dispatch_finish_ts": 1788171020.8003094, "request_finished_ts": 1788171031.1474192, "decode_throughput": 41886795.52358893, "response_sent_to_client_ts": 1788171031.1479979}}
|
||||
{"prompt_id": 8, "output": " When he turned around he was 3*60=<<3*60=180>>180 miles from home\nHe was only able to drive 4-2=<<4-2=2>>2 hours in the first four hours\nIn half an hour he goes 30*.5=<<30*.5=15>>15 miles\nHe then drives another 2-.5=<<2-.5=1.5>>1.5 hours\nIn that time he goes 80*1.5=<<80*1.5=120>>120 miles\nSo he drove 120+15=<<120+15=135>>135 miles\nSo he is 180-135=<<180-135=45>>45 miles away from home\n#### 45\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.708333333333333, "spec_accept_length_from_histogram": 6.75, "spec_accept_rate": 0.38333333333333336, "spec_accepted_drafts": 138, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 1, 3, 3, 4, 3, 0, 4, 1, 2, 1, 0, 1, 0, 0, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "18a9b297329c48818d49be7b1f96c1a8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 769, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171027.1447597, "queue_time": 0.0002117156982421875, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.962136375717819, "spec_accept_rate": 0.38333333333333336, "spec_accept_length": 6.708333333333333, "spec_num_correct_drafts": 138, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 138, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 1, 3, 3, 4, 3, 0, 4, 1, 2, 1, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 1, 3, 3, 4, 3, 0, 4, 1, 2, 1, 0, 1, 0, 0, 1], "request_received_ts": 1788171024.9036212, "api_server_dispatch_finish_ts": 1788171024.907199, "request_finished_ts": 1788171032.8657577, "decode_throughput": 43241553.44575887, "response_sent_to_client_ts": 1788171032.8662865}}
|
||||
{"prompt_id": 9, "output": " Eliza is entitled to 45 -40 = <<45-40=5>>5 hours overtime pay.\nHer hourly rate for the overtime pay is $10 x 1.2 = $<<10*1.2=12>>12.\nSo, Eliza will receive $12 x 5 =$<<12*5=60>>60 for overtime pay.\nHer regular weekly earning is $10 x 40 = $<<10*40=400>>400.\nThus, Eliza will receive a total of $400 + $60 = $<<400+60=460>>460 for this week's work.\n#### 460\n\n", "correct": true, "error": null, "completion_tokens": 130, "spec_accept_length": 5.416666666666667, "spec_accept_length_from_histogram": 5.458333333333333, "spec_accept_rate": 0.2972222222222222, "spec_accepted_drafts": 107, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 2, 4, 5, 2, 5, 1, 0, 3, 1, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c6e8f479e25a40bf82a7559d898a4e36", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 725, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171028.3827302, "queue_time": 0.00017261505126953125, "reasoning_tokens": 0, "completion_tokens": 130, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 130}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.063318707048893, "spec_accept_rate": 0.2972222222222222, "spec_accept_length": 5.416666666666667, "spec_num_correct_drafts": 107, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 107, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 2, 4, 5, 2, 5, 1, 0, 3, 1, 1], "spec_accept_histogram": [0, 2, 4, 5, 2, 5, 1, 0, 3, 1, 1], "request_received_ts": 1788171025.8568895, "api_server_dispatch_finish_ts": 1788171025.860344, "request_finished_ts": 1788171034.9202082, "decode_throughput": 35953976.715379626, "response_sent_to_client_ts": 1788171034.920715}}
|
||||
{"prompt_id": 10, "output": " The number of downloads of the program in the second month increased to 3*60 = <<3*60=180>>180\nIn the first two months, the total number of downloads of the program was 180+60 = <<180+60=240>>240\nIn the third month, the number of downloads of the program reduced by 30/100*180 = <<30/100*180=54>>54\nThere were 180-54 = <<180-54=126>>126 downloads in the third month.\nIn the three months, the total number of downloads of the program was 126+240 = <<126+240=366>>366\n#### 366\n\n", "correct": true, "error": null, "completion_tokens": 142, "spec_accept_length": 6.761904761904762, "spec_accept_length_from_histogram": 6.714285714285714, "spec_accept_rate": 0.38095238095238093, "spec_accepted_drafts": 120, "spec_proposed_drafts": 315, "spec_accept_histogram": [0, 0, 0, 6, 1, 2, 6, 1, 2, 1, 2], "spec_verify_ct": 21, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "29ab70fb26a148be92b65b0093e4e998", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 726, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171028.793264, "queue_time": 0.00014519691467285156, "reasoning_tokens": 0, "completion_tokens": 142, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 142}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 8.149700137786567, "spec_accept_rate": 0.38095238095238093, "spec_accept_length": 6.761904761904762, "spec_num_correct_drafts": 120, "spec_num_proposed_drafts": 315, "spec_verify_ct": 21, "spec_accepted_drafts": 120, "spec_proposed_drafts": 315, "spec_correct_drafts_histogram": [0, 0, 0, 6, 1, 2, 6, 1, 2, 1, 2], "spec_accept_histogram": [0, 0, 0, 6, 1, 2, 6, 1, 2, 1, 2], "request_received_ts": 1788171026.2023664, "api_server_dispatch_finish_ts": 1788171026.2057364, "request_finished_ts": 1788171034.3520665, "decode_throughput": 45836390.30699364, "response_sent_to_client_ts": 1788171034.3525615}}
|
||||
{"prompt_id": 11, "output": " The total charge for the doughnuts was 3 x $68 = $<<3*68=204>>204.\nThe total charge for the mini cupcakes was 2 x $80 = $<<2*80=160>>160.\nThe total charge for the mini cheesecakes was 6 x $55 = $<<6*55=330>>330.\nTherefore the total amount Toula paid for the pastries was $204 + $160 + $330 = $<<204+160+330=694>>694.\n#### 694\n\n", "correct": true, "error": null, "completion_tokens": 111, "spec_accept_length": 7.928571428571429, "spec_accept_length_from_histogram": 8.0, "spec_accept_rate": 0.4666666666666667, "spec_accepted_drafts": 98, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 3, 1, 1, 1, 0, 0, 1, 1, 2, 0, 1, 1, 0, 0, 2], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3f9fef92411447ca968d88d0fb4b7582", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 727, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171029.9221838, "queue_time": 0.00012731552124023438, "reasoning_tokens": 0, "completion_tokens": 111, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 111}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.115966156125069, "spec_accept_rate": 0.4666666666666667, "spec_accept_length": 7.928571428571429, "spec_num_correct_drafts": 98, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 98, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 3, 1, 1, 1, 0, 0, 1, 1, 2, 0, 1, 1, 0, 0, 2], "spec_accept_histogram": [0, 3, 1, 1, 1, 0, 0, 1, 1, 2, 0, 1, 1, 0, 0, 2], "request_received_ts": 1788171026.9315464, "api_server_dispatch_finish_ts": 1788171026.9348552, "request_finished_ts": 1788171034.0475128, "decode_throughput": 408.72742744392724, "response_sent_to_client_ts": 1788171034.0480049}}
|
||||
{"prompt_id": 12, "output": " He makes $10.5 selling lemons each year because 7 x 1.5 = <<7*1.5=10.5>>10.5\nHe earns $7.5 each year from the lemon tree because 10.5 - 3 = <<10.5-3=7.5>>7.5\nIt will take 12 years to earn enough to pay off the tree because 90 / 7.5 = <<90/7.5=12>>12\nHe will make money in year 13 because 12 + 1 = <<12+1=13>>13\n#### 13\n\n", "correct": true, "error": null, "completion_tokens": 133, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 6.947368421052632, "spec_accept_rate": 0.39649122807017545, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 0, 4, 2, 3, 1, 3, 0, 0, 3, 1, 0, 1, 0, 0, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e4e838a9eacc4db1b9c8641db898300a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 733, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171030.3793256, "queue_time": 0.00014209747314453125, "reasoning_tokens": 0, "completion_tokens": 133, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 133}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.992074851877987, "spec_accept_rate": 0.39649122807017545, "spec_accept_length": 7.0, "spec_num_correct_drafts": 113, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 0, 4, 2, 3, 1, 3, 0, 0, 3, 1, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 4, 2, 3, 1, 3, 0, 0, 3, 1, 0, 1, 0, 0, 1], "request_received_ts": 1788171027.6220148, "api_server_dispatch_finish_ts": 1788171027.6253722, "request_finished_ts": 1788171035.6140895, "decode_throughput": 43941689.89862037, "response_sent_to_client_ts": 1788171035.6145766}}
|
||||
{"prompt_id": 13, "output": " First multiply the five remaining vacuum cleaners by two to find out how many Melanie had before she visited the orange house: 5 * 2 = <<5*2=10>>10\nThen add two to figure out how many vacuum cleaners she had before visiting the red house: 10 + 2 = <<10+2=12>>12\nNow we know that 2/3 * x = 12, where x is the number of vacuum cleaners Melanie started with. We can find x by dividing each side of the equation by 2/3, which produces x = 18\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 126, "spec_accept_length": 5.478260869565218, "spec_accept_length_from_histogram": 5.521739130434782, "spec_accept_rate": 0.30144927536231886, "spec_accepted_drafts": 104, "spec_proposed_drafts": 345, "spec_accept_histogram": [0, 2, 5, 4, 1, 3, 1, 3, 3, 0, 1], "spec_verify_ct": 23, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b73f38c3bc0f4b8db6f3a49334b359e8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 725, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171030.8956745, "queue_time": 0.000148773193359375, "reasoning_tokens": 0, "completion_tokens": 126, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 126}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.123382485937327, "spec_accept_rate": 0.30144927536231886, "spec_accept_length": 5.478260869565218, "spec_num_correct_drafts": 104, "spec_num_proposed_drafts": 345, "spec_verify_ct": 23, "spec_accepted_drafts": 104, "spec_proposed_drafts": 345, "spec_correct_drafts_histogram": [0, 2, 5, 4, 1, 3, 1, 3, 3, 0, 1], "spec_accept_histogram": [0, 2, 5, 4, 1, 3, 1, 3, 3, 0, 1], "request_received_ts": 1788171028.57671, "api_server_dispatch_finish_ts": 1788171028.5799716, "request_finished_ts": 1788171037.7000926, "decode_throughput": 27186090.338262103, "response_sent_to_client_ts": 1788171037.7010555}}
|
||||
{"prompt_id": 14, "output": " There are 20 x 20/100 = <<20*20/100=4>>4 students who enrolled in contemporary dance.\nSo, 20 - 4 = <<20-4=16>>16 students are enrolled in either jazz or hip-hop dance.\nThere are 16 x 25/100 = <<16*25/100=4>>4 students who enrolled in jazz dance.\nHence, 16 - 4 = <<16-4=12>>12 students enrolled in hip-hop dance.\nThis is 12/20 x 100% = 60% of the entire students.\n#### 60\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 6.7368421052631575, "spec_accept_length_from_histogram": 6.7894736842105265, "spec_accept_rate": 0.38596491228070173, "spec_accepted_drafts": 110, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 1, 4, 1, 1, 4, 1, 1, 1, 2, 0, 1, 2], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5bc5c6ed56d542deb92f92da62b5d36f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171031.9601445, "queue_time": 0.00015735626220703125, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.549075374845415, "spec_accept_rate": 0.38596491228070173, "spec_accept_length": 6.7368421052631575, "spec_num_correct_drafts": 110, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 110, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 1, 4, 1, 1, 4, 1, 1, 1, 2, 0, 1, 2], "spec_accept_histogram": [0, 1, 4, 1, 1, 4, 1, 1, 1, 2, 0, 1, 2], "request_received_ts": 1788171030.1515276, "api_server_dispatch_finish_ts": 1788171030.1548295, "request_finished_ts": 1788171037.700603, "decode_throughput": 61661863.73411711, "response_sent_to_client_ts": 1788171037.7021062}}
|
||||
{"prompt_id": 15, "output": " If he purchases jewelry, he will make a profit of 2.5% which is $5000*(2.5/100) = $<<5000*(2.5/100)=125>>125\nIf he purchases electronic gadgets, he will make a profit of 1.2% which is $8000*(1.2/100) = $<<8000*(1.2/100)=96>>96\nIf he wants to maximize profit, since $125 > $96, he will choose to purchase jewelry, thereby making a profit of $<<125=125>>125\n#### 125\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 7.529411764705882, "spec_accept_length_from_histogram": 7.588235294117647, "spec_accept_rate": 0.4392156862745098, "spec_accepted_drafts": 112, "spec_proposed_drafts": 255, "spec_accept_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "spec_verify_ct": 17, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a857109e76af45a484d7f3f1ca46860c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 758, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171033.1573238, "queue_time": 0.00012373924255371094, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.794642649590969, "spec_accept_rate": 0.4392156862745098, "spec_accept_length": 7.529411764705882, "spec_num_correct_drafts": 112, "spec_num_proposed_drafts": 255, "spec_verify_ct": 17, "spec_accepted_drafts": 112, "spec_proposed_drafts": 255, "spec_correct_drafts_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "spec_accept_histogram": [0, 1, 0, 4, 3, 1, 1, 2, 1, 0, 1, 0, 0, 0, 1, 2], "request_received_ts": 1788171031.1587017, "api_server_dispatch_finish_ts": 1788171031.162011, "request_finished_ts": 1788171037.9533443, "decode_throughput": 27300342.672272272, "response_sent_to_client_ts": 1788171037.9538622}}
|
||||
{"prompt_id": 16, "output": " On the first day, the trains covered 2 trains * 80 miles/train = <<2*80=160>>160 miles together.\nThey also covered 150 miles/train * 2 trains = <<150*2=300>>300 miles together on the second day.\nThe combined distance the two trains covered in the two days is 300 miles + 160 miles = <<300+160=460>>460 miles\nThe average distance for the two days is 460 miles / 2 trains = <<460/2=230>>230 miles/train\n#### 230\n\n", "correct": true, "error": null, "completion_tokens": 117, "spec_accept_length": 6.157894736842105, "spec_accept_length_from_histogram": 6.2105263157894735, "spec_accept_rate": 0.3473684210526316, "spec_accepted_drafts": 99, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 0, 2, 7, 2, 1, 0, 3, 0, 1, 2, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "415d10be71f548c8b034d08878a4aca3", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 717, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171034.622519, "queue_time": 0.00016617774963378906, "reasoning_tokens": 0, "completion_tokens": 117, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 117}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.528884414117783, "spec_accept_rate": 0.3473684210526316, "spec_accept_length": 6.157894736842105, "spec_num_correct_drafts": 99, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 99, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 0, 2, 7, 2, 1, 0, 3, 0, 1, 2, 1], "spec_accept_histogram": [0, 0, 2, 7, 2, 1, 0, 3, 0, 1, 2, 1], "request_received_ts": 1788171032.8771527, "api_server_dispatch_finish_ts": 1788171032.8804798, "request_finished_ts": 1788171039.406037, "decode_throughput": 24764698.595088974, "response_sent_to_client_ts": 1788171039.40656}}
|
||||
{"prompt_id": 17, "output": " First find the total amount Jill makes per week teaching: $20/hour * 35 hours/week = $<<20*35=700>>700/week\nThen find the total amount Jill makes per week coaching: $30/hour * 15 hours/week = $<<30*15=450>>450/week\nThen add those two amounts to find the total amount Jill makes per week: $700/week + $450/week = $<<700+450=1150>>1150/week\nThen multiply that number by the number of weeks Jill works in a year to find her annual salary: $1150/week * 50 weeks/year = $<<1150*50=57500>>57,500\n#### 57500\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 6.521739130434782, "spec_accept_length_from_histogram": 6.608695652173913, "spec_accept_rate": 0.3739130434782609, "spec_accepted_drafts": 129, "spec_proposed_drafts": 345, "spec_accept_histogram": [0, 2, 5, 3, 0, 4, 2, 2, 1, 0, 0, 0, 1, 1, 1, 1], "spec_verify_ct": 23, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e81f0f7b73aa4b7a82980ac581bcfd4b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 719, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171036.3461308, "queue_time": 0.0004336833953857422, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.630079943686724, "spec_accept_rate": 0.3739130434782609, "spec_accept_length": 6.521739130434782, "spec_num_correct_drafts": 129, "spec_num_proposed_drafts": 345, "spec_verify_ct": 23, "spec_accepted_drafts": 129, "spec_proposed_drafts": 345, "spec_correct_drafts_histogram": [0, 2, 5, 3, 0, 4, 2, 2, 1, 0, 0, 0, 1, 1, 1, 1], "spec_accept_histogram": [0, 2, 5, 3, 0, 4, 2, 2, 1, 0, 0, 0, 1, 1, 1, 1], "request_received_ts": 1788171034.0583067, "api_server_dispatch_finish_ts": 1788171034.0615325, "request_finished_ts": 1788171041.6883867, "decode_throughput": 52472132.42899311, "response_sent_to_client_ts": 1788171041.6888897}}
|
||||
{"prompt_id": 18, "output": " She eats 3 eggs every day and there are 7 days in a week so she eats 3*7 = <<3*7=21>>21 eggs a week\nAfter 4 weeks she will have eaten 4*21 = <<4*21=84>>84 eggs\nThere are 12 eggs in 1 dozen and she'll eat 84 eggs so that's 84/12 = <<84/12=7>>7 dozen eggs\n#### 7\n\n", "correct": true, "error": null, "completion_tokens": 98, "spec_accept_length": 5.764705882352941, "spec_accept_length_from_histogram": 5.823529411764706, "spec_accept_rate": 0.3215686274509804, "spec_accepted_drafts": 82, "spec_proposed_drafts": 255, "spec_accept_histogram": [0, 1, 4, 3, 1, 4, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "spec_verify_ct": 17, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "1f8113c421dc4ba0bffcd2ce18988f86", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 695, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171037.4735415, "queue_time": 0.00016260147094726562, "reasoning_tokens": 0, "completion_tokens": 98, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 98}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.947669697925448, "spec_accept_rate": 0.3215686274509804, "spec_accept_length": 5.764705882352941, "spec_num_correct_drafts": 82, "spec_num_proposed_drafts": 255, "spec_verify_ct": 17, "spec_accepted_drafts": 82, "spec_proposed_drafts": 255, "spec_correct_drafts_histogram": [0, 1, 4, 3, 1, 4, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 4, 3, 1, 4, 0, 0, 2, 1, 0, 0, 0, 0, 0, 1], "request_received_ts": 1788171034.3628626, "api_server_dispatch_finish_ts": 1788171034.366082, "request_finished_ts": 1788171041.3105323, "decode_throughput": 27247339.941922825, "response_sent_to_client_ts": 1788171041.311102}}
|
||||
{"prompt_id": 19, "output": " First figure out how many hours it takes to hike a 12-mile trail at 4 mph by dividing the distance by the speed: 12 miles / 4 mph = <<12/4=3>>3 hours\nNext subtract the time Marissa already spent walking to find out how much time she has left: 3 hours - 1 hour - 1 hour = <<3-1-1=1>>1 hour\nNow figure out how much distance she has left by subtracting the distance she already traveled from the total distance: 12 miles - 4 miles - 2 miles = <<12-4-2=6>>6 miles\nNow divide the remaining distance by the remaining time to find out how fast in miles per hour Marissa has to travel: 6 miles / 1 hour = <<6/1=6>>6 mph\n#### 6\n\n", "correct": true, "error": null, "completion_tokens": 181, "spec_accept_length": 6.241379310344827, "spec_accept_length_from_histogram": 6.275862068965517, "spec_accept_rate": 0.35172413793103446, "spec_accepted_drafts": 153, "spec_proposed_drafts": 435, "spec_accept_histogram": [1, 1, 1, 8, 4, 2, 3, 1, 3, 2, 1, 1, 1], "spec_verify_ct": 29, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "9b9cd642d1ea4a24987b9eef643f92c6", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 732, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171038.1344798, "queue_time": 0.00012922286987304688, "reasoning_tokens": 0, "completion_tokens": 181, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 181}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.535634547006339, "spec_accept_rate": 0.35172413793103446, "spec_accept_length": 6.241379310344827, "spec_num_correct_drafts": 153, "spec_num_proposed_drafts": 435, "spec_verify_ct": 29, "spec_accepted_drafts": 153, "spec_proposed_drafts": 435, "spec_correct_drafts_histogram": [1, 1, 1, 8, 4, 2, 3, 1, 3, 2, 1, 1, 1], "spec_accept_histogram": [1, 1, 1, 8, 4, 2, 3, 1, 3, 2, 1, 1, 1], "request_received_ts": 1788171034.9309604, "api_server_dispatch_finish_ts": 1788171034.9341989, "request_finished_ts": 1788171044.4665952, "decode_throughput": 56298726.5715118, "response_sent_to_client_ts": 1788171044.4673266}}
|
||||
{"prompt_id": 20, "output": " There are 15 x 3/5 = <<15*3/5=9>>9 liters of water from the 15 liters pineapple drink.\nAfter 1 liter of orange drink was spilled, there were 10 - 1 = <<10-1=9>>9 liters of orange drink left.\nOut of the 9 liters, 9 x 2/3 = <<9*2/3=6>>6 liters are water.\nThus, there are a total of 9 + 6 = <<9+6=15>>15 liters of water out of the 24 liters.\n#### 15\n\n", "correct": true, "error": null, "completion_tokens": 128, "spec_accept_length": 5.818181818181818, "spec_accept_length_from_histogram": 5.863636363636363, "spec_accept_rate": 0.3242424242424242, "spec_accepted_drafts": 107, "spec_proposed_drafts": 330, "spec_accept_histogram": [1, 1, 3, 5, 0, 7, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "14c4803b2d744c588ea0d9c6b6a28f40", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171038.3365996, "queue_time": 0.0001316070556640625, "reasoning_tokens": 0, "completion_tokens": 128, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 128}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 8.020555406808853, "spec_accept_rate": 0.3242424242424242, "spec_accept_length": 5.818181818181818, "spec_num_correct_drafts": 107, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 107, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [1, 1, 3, 5, 0, 7, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [1, 1, 3, 5, 0, 7, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788171035.6247268, "api_server_dispatch_finish_ts": 1788171035.6279175, "request_finished_ts": 1788171043.645282, "decode_throughput": 40.777434737710614, "response_sent_to_client_ts": 1788171043.6458201}}
|
||||
{"prompt_id": 21, "output": " When Raymond's son was born Samantha was 23 - 6 = <<23-6=17>>17 years old.\nThus it has been 31 - 17 = <<31-17=14>>14 years since Raymond's son was born.\n#### 14\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.125, "spec_accept_rate": 0.4083333333333333, "spec_accepted_drafts": 49, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ea994b86f889459aba915a3ba2c4d988", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 711, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171040.3478446, "queue_time": 0.00013136863708496094, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.044767498970032, "spec_accept_rate": 0.4083333333333333, "spec_accept_length": 7.0, "spec_num_correct_drafts": 49, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 49, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 0, 2, 0, 3, 0, 0, 2, 0, 0, 0, 1], "request_received_ts": 1788171037.7175963, "api_server_dispatch_finish_ts": 1788171037.7211146, "request_finished_ts": 1788171042.7623637, "decode_throughput": 14612346.980081653, "response_sent_to_client_ts": 1788171042.7628555}}
|
||||
{"prompt_id": 22, "output": " His first 3 customers buy 3 * 1 = <<3*1=3>>3 DVDs.\nHis next 2 buy 2 * 2 = <<2*2=4>>4 DVDs.\nHe sells a total of 3 + 4 + 0 = <<3+4+0=7>>7 DVDs.\n#### 7\n\n", "correct": true, "error": null, "completion_tokens": 74, "spec_accept_length": 8.222222222222221, "spec_accept_length_from_histogram": 8.333333333333332, "spec_accept_rate": 0.4888888888888889, "spec_accepted_drafts": 66, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 1, 1, 1, 0, 0, 0, 1, 3, 1, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "34c9312202904c7bad1e8a638b797b9e", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 722, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171040.5366993, "queue_time": 0.00018906593322753906, "reasoning_tokens": 0, "completion_tokens": 74, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 74}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.506346830166876, "spec_accept_rate": 0.4888888888888889, "spec_accept_length": 8.222222222222221, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 66, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 1, 1, 1, 0, 0, 0, 1, 3, 1, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 1, 0, 0, 0, 1, 3, 1, 0, 1], "request_received_ts": 1788171037.7225342, "api_server_dispatch_finish_ts": 1788171037.7254233, "request_finished_ts": 1788171043.2288811, "decode_throughput": 21634875.28346674, "response_sent_to_client_ts": 1788171043.2296798}}
|
||||
{"prompt_id": 23, "output": " The candle burns for 5 - 1 = <<5-1=4>>4 hours.\nThus, the candle will be 2 * 4 = <<2*4=8>>8 centimeters shorter.\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.25, "spec_accept_rate": 0.35, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 1, 1, 2, 1, 0, 1, 1, 1], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "29a98b669d774f0c85e8d62849cddf5a", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 706, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171040.7640514, "queue_time": 0.00013399124145507812, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.265191392973065, "spec_accept_rate": 0.35, "spec_accept_length": 6.125, "spec_num_correct_drafts": 42, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 1, 1, 2, 1, 0, 1, 1, 1], "spec_accept_histogram": [0, 0, 1, 1, 2, 1, 0, 1, 1, 1], "request_received_ts": 1788171037.964066, "api_server_dispatch_finish_ts": 1788171037.967336, "request_finished_ts": 1788171043.2292573, "decode_throughput": 24910395.143547606, "response_sent_to_client_ts": 1788171043.2308676}}
|
||||
{"prompt_id": 24, "output": " Let X be the original price of the book. The discounted price is X - X*25% = $19.50.\nCombining like terms, we get .75X = $19.50\nDividing both sides by .75, we get X = $26.\n#### 26\n\n", "correct": true, "error": null, "completion_tokens": 63, "spec_accept_length": 4.846153846153846, "spec_accept_length_from_histogram": 4.923076923076923, "spec_accept_rate": 0.26153846153846155, "spec_accepted_drafts": 51, "spec_proposed_drafts": 195, "spec_accept_histogram": [1, 0, 3, 4, 1, 1, 1, 0, 1, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a26b8f007ffd4104be14fea17857003c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 703, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171041.6730297, "queue_time": 0.0002231597900390625, "reasoning_tokens": 0, "completion_tokens": 63, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 63}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.050237540155649, "spec_accept_rate": 0.26153846153846155, "spec_accept_length": 4.846153846153846, "spec_num_correct_drafts": 51, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 51, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [1, 0, 3, 4, 1, 1, 1, 0, 1, 0, 1], "spec_accept_histogram": [1, 0, 3, 4, 1, 1, 1, 0, 1, 0, 1], "request_received_ts": 1788171039.4167023, "api_server_dispatch_finish_ts": 1788171039.4199271, "request_finished_ts": 1788171044.4669397, "decode_throughput": 30963717.715348836, "response_sent_to_client_ts": 1788171044.4682224}}
|
||||
{"prompt_id": 25, "output": " Five packs of milk cost $3 x 5 = $<<3*5=15>>15.\nFour apples cost $1.50 x 4 = $<<1.5*4=6>>6.\nThe total cost of the chicken meal, milk, and apples is $12 + $15 + $6 = $<<12+15+6=33>>33.\nThus, the boxes of pizza cost $50 - $33 = $<<50-33=17>>17.\nTherefore, Marie ordered $17/$8.50 = <<17/8.5=2>>2 boxes of pizza.\n#### 2\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 6.894736842105263, "spec_accept_length_from_histogram": 6.947368421052632, "spec_accept_rate": 0.39649122807017545, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_accept_histogram": [0, 1, 2, 3, 2, 2, 1, 3, 0, 1, 3, 0, 0, 0, 0, 1], "spec_verify_ct": 19, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5b4bff694e1d4f628743b9930f7bb727", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171043.4438434, "queue_time": 0.00015735626220703125, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.790783246979117, "spec_accept_rate": 0.39649122807017545, "spec_accept_length": 6.894736842105263, "spec_num_correct_drafts": 113, "spec_num_proposed_drafts": 285, "spec_verify_ct": 19, "spec_accepted_drafts": 113, "spec_proposed_drafts": 285, "spec_correct_drafts_histogram": [0, 1, 2, 3, 2, 2, 1, 3, 0, 1, 3, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 2, 2, 1, 3, 0, 1, 3, 0, 0, 0, 0, 1], "request_received_ts": 1788171041.3213995, "api_server_dispatch_finish_ts": 1788171041.3247201, "request_finished_ts": 1788171046.1121826, "decode_throughput": 37099385.281063125, "response_sent_to_client_ts": 1788171046.1126842}}
|
||||
{"prompt_id": 26, "output": " 3 * (16.50 + 22.50 + 42) = $<<3*(16.50+22.50+42)=243>>243\nMishka spent $243 on clothing.\n#### 243\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.142857142857143, "spec_accept_rate": 0.4095238095238095, "spec_accepted_drafts": 43, "spec_proposed_drafts": 105, "spec_accept_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "db02870e04c24dc1989d6dfc3fa91b80", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171044.3085938, "queue_time": 0.0001575946807861328, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.7027414380572736, "spec_accept_rate": 0.4095238095238095, "spec_accept_length": 7.0, "spec_num_correct_drafts": 43, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 43, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "spec_accept_histogram": [0, 1, 0, 0, 1, 0, 1, 2, 0, 2], "request_received_ts": 1788171041.698959, "api_server_dispatch_finish_ts": 1788171041.7022228, "request_finished_ts": 1788171045.4017005, "decode_throughput": 15941728.2870399, "response_sent_to_client_ts": 1788171045.4021764}}
|
||||
{"prompt_id": 27, "output": " Each container of ice cream has 15 servings and she eats 1 serving a night so after 60 days she will need 60/15 = 4 containers of ice cream\nIf each carton costs $4.00 and she needs 4 containers then it will cost her 4*4 = $<<4*4=16.00>>16.00\n#### 16\n\n", "correct": true, "error": null, "completion_tokens": 80, "spec_accept_length": 5.333333333333333, "spec_accept_length_from_histogram": 5.4, "spec_accept_rate": 0.29333333333333333, "spec_accepted_drafts": 66, "spec_proposed_drafts": 225, "spec_accept_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "spec_verify_ct": 15, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b40f89caf08944f0b320386fc37b702d", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 721, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171045.6396155, "queue_time": 0.00026488304138183594, "reasoning_tokens": 0, "completion_tokens": 80, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 80}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.235636665020138, "spec_accept_rate": 0.29333333333333333, "spec_accept_length": 5.333333333333333, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 225, "spec_verify_ct": 15, "spec_accepted_drafts": 66, "spec_proposed_drafts": 225, "spec_correct_drafts_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 4, 2, 0, 2, 0, 0, 0, 0, 1], "request_received_ts": 1788171042.7703478, "api_server_dispatch_finish_ts": 1788171042.7749627, "request_finished_ts": 1788171048.0059845, "decode_throughput": 26852043.08198797, "response_sent_to_client_ts": 1788171048.0064573}}
|
||||
{"prompt_id": 28, "output": " He traveled 20 miles + 15 miles = <<20+15=35>>35 miles not counting the distance between stops.\nHenry traveled 60 miles - 35 miles = <<60-35=25>>25 miles between his first and second stop.\n#### 25\n\n", "correct": true, "error": null, "completion_tokens": 57, "spec_accept_length": 5.7, "spec_accept_length_from_histogram": 5.8, "spec_accept_rate": 0.32, "spec_accepted_drafts": 48, "spec_proposed_drafts": 150, "spec_accept_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 10, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "8caaaceb5c01459cae365a46aae9db79", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 715, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171046.2403767, "queue_time": 0.0001342296600341797, "reasoning_tokens": 0, "completion_tokens": 57, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 57}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.5570953781716526, "spec_accept_rate": 0.32, "spec_accept_length": 5.7, "spec_num_correct_drafts": 48, "spec_num_proposed_drafts": 150, "spec_verify_ct": 10, "spec_accepted_drafts": 48, "spec_proposed_drafts": 150, "spec_correct_drafts_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 2, 1, 2, 0, 1, 1, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788171043.2511187, "api_server_dispatch_finish_ts": 1788171043.2540686, "request_finished_ts": 1788171047.808214, "decode_throughput": 16821805.047978736, "response_sent_to_client_ts": 1788171047.808872}}
|
||||
{"prompt_id": 29, "output": " The second pair of heels costs 33 * 2 = $<<33*2=66>>66.\nThe heels together cost 66 + 33 = $<<66+33=99>>99.\nThe boots cost $5 more than both pairs of heels together, so the boots cost 99 + 5 = $104.\n#### 104\n\n", "correct": true, "error": null, "completion_tokens": 73, "spec_accept_length": 6.636363636363637, "spec_accept_length_from_histogram": 6.7272727272727275, "spec_accept_rate": 0.38181818181818183, "spec_accepted_drafts": 63, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "da7cb0b778d542c48f4a2d96d6de41fa", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 734, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171046.5084116, "queue_time": 0.00026297569274902344, "reasoning_tokens": 0, "completion_tokens": 73, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 73}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.372211418114603, "spec_accept_rate": 0.38181818181818183, "spec_accept_length": 6.636363636363637, "spec_num_correct_drafts": 63, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 63, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "spec_accept_histogram": [0, 0, 1, 2, 1, 2, 1, 1, 0, 2, 1], "request_received_ts": 1788171043.2466784, "api_server_dispatch_finish_ts": 1788171043.2501311, "request_finished_ts": 1788171048.6188898, "decode_throughput": 88.8612370281484, "response_sent_to_client_ts": 1788171048.6194963}}
|
||||
{"prompt_id": 30, "output": " The total ratio representing their ages is 7+11= <<7+11=18>>18\nSince the fraction of the ratio that represents Allen's age is 11/18, Allen's current age is 11/18*162 = <<11/18*162=99>>99\nIf Allen is currently 99 years old, in 10 years he will be 99+10 = <<99+10=109>>109 years old\n#### 109\n\n", "correct": true, "error": null, "completion_tokens": 98, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.1875, "spec_accept_rate": 0.3458333333333333, "spec_accepted_drafts": 83, "spec_proposed_drafts": 240, "spec_accept_histogram": [0, 0, 3, 3, 2, 3, 0, 0, 3, 0, 1, 1], "spec_verify_ct": 16, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "72d1a93ea65f451685a63ea38b9cc34c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 703, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171046.9418125, "queue_time": 0.0001277923583984375, "reasoning_tokens": 0, "completion_tokens": 98, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 98}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.547496572136879, "spec_accept_rate": 0.3458333333333333, "spec_accept_length": 6.125, "spec_num_correct_drafts": 83, "spec_num_proposed_drafts": 240, "spec_verify_ct": 16, "spec_accepted_drafts": 83, "spec_proposed_drafts": 240, "spec_correct_drafts_histogram": [0, 0, 3, 3, 2, 3, 0, 0, 3, 0, 1, 1], "spec_accept_histogram": [0, 0, 3, 3, 2, 3, 0, 0, 3, 0, 1, 1], "request_received_ts": 1788171043.655841, "api_server_dispatch_finish_ts": 1788171043.6590207, "request_finished_ts": 1788171050.2033377, "decode_throughput": 61.23421026675583, "response_sent_to_client_ts": 1788171050.2038069}}
|
||||
{"prompt_id": 31, "output": " The second person guessed 60 because 20 + (80/2)= <<20+(80/2)=60>>60\nThe second guessed 100 because 80 x 1.25 = <<80*1.25=100>>100\nThe total amount guessed is 240 because 80 + 60 + 100= <<80+60+100=240>>240\nThe average amount guessed is 80 because 240 / 3 = <<240/3=80>>80\n#### 80\n\n", "correct": true, "error": null, "completion_tokens": 107, "spec_accept_length": 5.944444444444445, "spec_accept_length_from_histogram": 5.888888888888889, "spec_accept_rate": 0.32592592592592595, "spec_accepted_drafts": 88, "spec_proposed_drafts": 270, "spec_accept_histogram": [1, 2, 5, 2, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1], "spec_verify_ct": 18, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "20dd7ef70fea4961971b170c798a9200", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171047.9902349, "queue_time": 0.000263214111328125, "reasoning_tokens": 0, "completion_tokens": 107, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 107}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.1681548319756985, "spec_accept_rate": 0.32592592592592595, "spec_accept_length": 5.944444444444445, "spec_num_correct_drafts": 88, "spec_num_proposed_drafts": 270, "spec_verify_ct": 18, "spec_accepted_drafts": 88, "spec_proposed_drafts": 270, "spec_correct_drafts_histogram": [1, 2, 5, 2, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1], "spec_accept_histogram": [1, 2, 5, 2, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1], "request_received_ts": 1788171044.4829614, "api_server_dispatch_finish_ts": 1788171044.4861844, "request_finished_ts": 1788171051.6511161, "decode_throughput": 28803399.555611793, "response_sent_to_client_ts": 1788171051.652002}}
|
||||
{"prompt_id": 32, "output": " He spends 10*.5=<<10*.5=5>>5 hours per day\nThat means he spends 5*7=<<5*7=35>>35 hours per week\n#### 35\n\n", "correct": true, "error": null, "completion_tokens": 45, "spec_accept_length": 6.428571428571429, "spec_accept_length_from_histogram": 6.571428571428571, "spec_accept_rate": 0.37142857142857144, "spec_accepted_drafts": 39, "spec_proposed_drafts": 105, "spec_accept_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "7efc78a27fe8484ebe2802a59cc8711e", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 708, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171047.9902349, "queue_time": 0.0001518726348876953, "reasoning_tokens": 0, "completion_tokens": 45, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 45}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.941901796031743, "spec_accept_rate": 0.37142857142857144, "spec_accept_length": 6.428571428571429, "spec_num_correct_drafts": 39, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 39, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_accept_histogram": [1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "request_received_ts": 1788171044.487551, "api_server_dispatch_finish_ts": 1788171044.4903896, "request_finished_ts": 1788171049.429453, "decode_throughput": 14492220.937423313, "response_sent_to_client_ts": 1788171049.4299169}}
|
||||
{"prompt_id": 33, "output": " Let x be the number of silver coins Gretchen has\nGretchen has x+30 gold coins.\nx+x+30=110\n2*x=80\nx=<<40=40>>40\nGretchen has 40+30=<<40+30=70>>70 gold coins\n#### 70\n\n", "correct": true, "error": null, "completion_tokens": 68, "spec_accept_length": 5.230769230769231, "spec_accept_length_from_histogram": 5.230769230769231, "spec_accept_rate": 0.28205128205128205, "spec_accepted_drafts": 55, "spec_proposed_drafts": 195, "spec_accept_histogram": [1, 3, 2, 2, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c1f9e546172442c2bed3cf0b042ab5f0", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 695, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171048.8013089, "queue_time": 0.0001614093780517578, "reasoning_tokens": 0, "completion_tokens": 68, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 68}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.015332517214119, "spec_accept_rate": 0.28205128205128205, "spec_accept_length": 5.230769230769231, "spec_num_correct_drafts": 55, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 55, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [1, 3, 2, 2, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "spec_accept_histogram": [1, 3, 2, 2, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1], "request_received_ts": 1788171045.4122992, "api_server_dispatch_finish_ts": 1788171045.415459, "request_finished_ts": 1788171051.4276316, "decode_throughput": 13131459.744090537, "response_sent_to_client_ts": 1788171051.428137}}
|
||||
{"prompt_id": 34, "output": " Half of Raymond's jewels is 40/2 = <<40/2=20>>20\nSince Aaron has 5 more jewels than half of Raymond's jewels, he has 20+5 = <<5+20=25>>25 jewels.\nIf Siobhan has 2 fewer jewels than Aaron, he has 25-2 = 23 jewels.\n#### 23\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 6.583333333333333, "spec_accept_length_from_histogram": 6.666666666666667, "spec_accept_rate": 0.37777777777777777, "spec_accepted_drafts": 68, "spec_proposed_drafts": 180, "spec_accept_histogram": [0, 1, 3, 0, 2, 2, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 12, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3ddecfb4a48b4f06950180eb31fe49be", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 708, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171049.0135803, "queue_time": 0.00013375282287597656, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.525186281185597, "spec_accept_rate": 0.37777777777777777, "spec_accept_length": 6.583333333333333, "spec_num_correct_drafts": 68, "spec_num_proposed_drafts": 180, "spec_verify_ct": 12, "spec_accepted_drafts": 68, "spec_proposed_drafts": 180, "spec_correct_drafts_histogram": [0, 1, 3, 0, 2, 2, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 3, 0, 2, 2, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788171046.1263468, "api_server_dispatch_finish_ts": 1788171046.129568, "request_finished_ts": 1788171051.6515331, "decode_throughput": 29324881.74789916, "response_sent_to_client_ts": 1788171051.6529446}}
|
||||
{"prompt_id": 35, "output": " He scored 4*.25=<<4*.25=1>>1 point more in the second 20 minutes\nSo he scored 4+1=<<4+1=5>>5 points in the second 20 minutes\nSo in total, he scored 4+5=<<4+5=9>>9 points in the 40 minutes\n#### 9\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 8.777777777777779, "spec_accept_length_from_histogram": 8.88888888888889, "spec_accept_rate": 0.5259259259259259, "spec_accepted_drafts": 71, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "f53ccf046023492d884a37ed0110adba", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 716, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171050.3849766, "queue_time": 0.0001289844512939453, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.338141418993473, "spec_accept_rate": 0.5259259259259259, "spec_accept_length": 8.777777777777779, "spec_num_correct_drafts": 71, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 71, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 1, 0, 0, 1], "request_received_ts": 1788171047.8256965, "api_server_dispatch_finish_ts": 1788171047.828871, "request_finished_ts": 1788171052.163838, "decode_throughput": 152.32085040947746, "response_sent_to_client_ts": 1788171052.16436}}
|
||||
{"prompt_id": 36, "output": " He eats 2 yogurts a day so over 30 days he will need 2*30 = <<2*30=60>>60 yogurts\nThe yogurt is sold as 4 units for $5.00 and he needs 60 yogurts so that’s 60/4 = <<60/4=15>>15 4-packs\nEach 4-pack is $5.00 and he needs 15 4-packs so he will spend $5*15 = $<<5*15=75.00>>75.00 on yogurt\n#### 75\n\n", "correct": true, "error": null, "completion_tokens": 124, "spec_accept_length": 5.636363636363637, "spec_accept_length_from_histogram": 5.681818181818182, "spec_accept_rate": 0.31212121212121213, "spec_accepted_drafts": 103, "spec_proposed_drafts": 330, "spec_accept_histogram": [0, 2, 4, 3, 3, 2, 2, 1, 4, 0, 0, 1], "spec_verify_ct": 22, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "75b97163a774476081e53fabfe563eb3", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171050.5958166, "queue_time": 0.00014400482177734375, "reasoning_tokens": 0, "completion_tokens": 124, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 124}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.386951171793044, "spec_accept_rate": 0.31212121212121213, "spec_accept_length": 5.636363636363637, "spec_num_correct_drafts": 103, "spec_num_proposed_drafts": 330, "spec_verify_ct": 22, "spec_accepted_drafts": 103, "spec_proposed_drafts": 330, "spec_correct_drafts_histogram": [0, 2, 4, 3, 3, 2, 2, 1, 4, 0, 0, 1], "spec_accept_histogram": [0, 2, 4, 3, 3, 2, 2, 1, 4, 0, 0, 1], "request_received_ts": 1788171048.017575, "api_server_dispatch_finish_ts": 1788171048.0207539, "request_finished_ts": 1788171054.4045262, "decode_throughput": 63.18343324451378, "response_sent_to_client_ts": 1788171054.4049995}}
|
||||
{"prompt_id": 37, "output": " He spent $160 on video games because 8 x 20 = <<8*20=160>>160\nHe earned $165 from his sale because 160 + 5 = <<160+5=165>>165\nHe sold 11 lego sets because 11 x 15 = <<11*15=165>>165\nHe has 2 sets left because 13 - 11 = <<13-11=2>>2\n#### 2\n\n", "correct": true, "error": null, "completion_tokens": 96, "spec_accept_length": 8.0, "spec_accept_length_from_histogram": 8.083333333333332, "spec_accept_rate": 0.4722222222222222, "spec_accepted_drafts": 85, "spec_proposed_drafts": 180, "spec_accept_histogram": [0, 1, 1, 1, 1, 0, 1, 2, 0, 2, 0, 2, 0, 0, 0, 1], "spec_verify_ct": 12, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "677e55f1cd34432c9d391ddbe87fe2cb", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171051.8385348, "queue_time": 0.0001857280731201172, "reasoning_tokens": 0, "completion_tokens": 96, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 96}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.970573452766985, "spec_accept_rate": 0.4722222222222222, "spec_accept_length": 8.0, "spec_num_correct_drafts": 85, "spec_num_proposed_drafts": 180, "spec_verify_ct": 12, "spec_accepted_drafts": 85, "spec_proposed_drafts": 180, "spec_correct_drafts_histogram": [0, 1, 1, 1, 1, 0, 1, 2, 0, 2, 0, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 1, 1, 1, 0, 1, 2, 0, 2, 0, 2, 0, 0, 0, 1], "request_received_ts": 1788171048.6297784, "api_server_dispatch_finish_ts": 1788171048.6329646, "request_finished_ts": 1788171053.6003518, "decode_throughput": 31056621.488811083, "response_sent_to_client_ts": 1788171053.6008267}}
|
||||
{"prompt_id": 38, "output": " He has 7-1=<<7-1=6>>6 possible running days.\nHe runs half of those so he runs 6/2=<<6/2=3>>3 days.\nSo for two days, he runs 3/2=<<3/2=1.5>>1.5 hours.\nSo in total, he runs 3+1.5+1.5=<<3+1.5+1.5=6>>6 hours.\nSo he runs 60/6=<<60/6=10>>10 mph.\n#### 10\n\n", "correct": true, "error": null, "completion_tokens": 124, "spec_accept_length": 6.2, "spec_accept_length_from_histogram": 6.25, "spec_accept_rate": 0.35, "spec_accepted_drafts": 105, "spec_proposed_drafts": 300, "spec_accept_histogram": [0, 3, 1, 2, 3, 0, 4, 3, 1, 1, 2], "spec_verify_ct": 20, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ab7a2157393646e8acc7044b741705d4", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 711, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171052.707009, "queue_time": 0.00020170211791992188, "reasoning_tokens": 0, "completion_tokens": 124, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 124}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.140904388856143, "spec_accept_rate": 0.35, "spec_accept_length": 6.2, "spec_num_correct_drafts": 105, "spec_num_proposed_drafts": 300, "spec_verify_ct": 20, "spec_accepted_drafts": 105, "spec_proposed_drafts": 300, "spec_correct_drafts_histogram": [0, 3, 1, 2, 3, 0, 4, 3, 1, 1, 2], "spec_accept_histogram": [0, 3, 1, 2, 3, 0, 4, 3, 1, 1, 2], "request_received_ts": 1788171049.4399328, "api_server_dispatch_finish_ts": 1788171049.4431252, "request_finished_ts": 1788171056.5808372, "decode_throughput": 31966657.231513977, "response_sent_to_client_ts": 1788171056.5813298}}
|
||||
{"prompt_id": 39, "output": " If Dana can skip at half the speed she can run, then she can run at 3*2=<<3*2=6>>6 miles per hour.\nAnd since she can run at a speed that is 4 times faster than she can walk, this means she can walk at 6/4=1.5 miles per hour.\nIf two-thirds of the time is spent walking, then she walks for 6*(2/3)=<<6*(2/3)=4>>4 hours.\nIf one-third of the time is spent running, then she runs for 6-4=<<6-4=2>>2 hours.\nThus, she runs for 2 hours at 6 miles per hour, or 2*6=<<2*6=12>>12 miles.\nShe walks for 4 hours at 1.5 miles per hour, or 4*1.5=<<4*1.5=6>>6 miles.\nThus, altogether, she travels 12+6=<<12+6=18>>18 miles.\n#### 18\n\n", "correct": true, "error": null, "completion_tokens": 225, "spec_accept_length": 6.818181818181818, "spec_accept_length_from_histogram": 6.848484848484849, "spec_accept_rate": 0.3898989898989899, "spec_accepted_drafts": 193, "spec_proposed_drafts": 495, "spec_accept_histogram": [0, 2, 3, 6, 7, 2, 2, 1, 1, 3, 2, 0, 1, 0, 2, 1], "spec_verify_ct": 33, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "bee58840fd9e4984affa6bfa18af4b9b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 740, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171053.443203, "queue_time": 0.0001366138458251953, "reasoning_tokens": 0, "completion_tokens": 225, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 225}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 10.610384341794997, "spec_accept_rate": 0.3898989898989899, "spec_accept_length": 6.818181818181818, "spec_num_correct_drafts": 193, "spec_num_proposed_drafts": 495, "spec_verify_ct": 33, "spec_accepted_drafts": 193, "spec_proposed_drafts": 495, "spec_correct_drafts_histogram": [0, 2, 3, 6, 7, 2, 2, 1, 1, 3, 2, 0, 1, 0, 2, 1], "spec_accept_histogram": [0, 2, 3, 6, 7, 2, 2, 1, 1, 3, 2, 0, 1, 0, 2, 1], "request_received_ts": 1788171050.2139897, "api_server_dispatch_finish_ts": 1788171050.2172062, "request_finished_ts": 1788171060.824374, "decode_throughput": 290.5402074961987, "response_sent_to_client_ts": 1788171060.8248646}}
|
||||
{"prompt_id": 40, "output": " Ben’s iPhone is 1*2 = <<1*2=2>>2 years old.\nBrandon’s iPhone is 4*2 = <<4*2=8>>8 years old.\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 48, "spec_accept_length": 6.857142857142857, "spec_accept_length_from_histogram": 6.714285714285714, "spec_accept_rate": 0.38095238095238093, "spec_accepted_drafts": 40, "spec_proposed_drafts": 105, "spec_accept_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "spec_verify_ct": 7, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "83533f1dd14b41c88be6a275d02080c1", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 718, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171054.5612996, "queue_time": 0.00014495849609375, "reasoning_tokens": 0, "completion_tokens": 48, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 48}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.596388224978, "spec_accept_rate": 0.38095238095238093, "spec_accept_length": 6.857142857142857, "spec_num_correct_drafts": 40, "spec_num_proposed_drafts": 105, "spec_verify_ct": 7, "spec_accepted_drafts": 40, "spec_proposed_drafts": 105, "spec_correct_drafts_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 2, 1, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1], "request_received_ts": 1788171051.438173, "api_server_dispatch_finish_ts": 1788171051.4416385, "request_finished_ts": 1788171056.0345612, "decode_throughput": 13143863.973954942, "response_sent_to_client_ts": 1788171056.0350468}}
|
||||
{"prompt_id": 41, "output": " With the gemstone, Polly could throw the javelin 3 times farther than 400 feet, for a distance of 3*400=<<3*400=1200>>1200 feet.\n1200 feet is beyond the 1000-foot reach of the dragon's flames by a distance of 1200-1000=<<1200-1000=200>>200 feet.\n#### 200\n\n", "correct": true, "error": null, "completion_tokens": 86, "spec_accept_length": 6.615384615384615, "spec_accept_length_from_histogram": 6.6923076923076925, "spec_accept_rate": 0.37948717948717947, "spec_accepted_drafts": 74, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "47c78529369843bbabd095943e632cb0", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 794, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171054.7348893, "queue_time": 0.00013637542724609375, "reasoning_tokens": 0, "completion_tokens": 86, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 86}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.367660026997328, "spec_accept_rate": 0.37948717948717947, "spec_accept_length": 6.615384615384615, "spec_num_correct_drafts": 74, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 74, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "spec_accept_histogram": [0, 1, 1, 2, 0, 2, 3, 0, 1, 2, 0, 1], "request_received_ts": 1788171051.6692016, "api_server_dispatch_finish_ts": 1788171051.6726334, "request_finished_ts": 1788171058.0368617, "decode_throughput": 28909741.85619259, "response_sent_to_client_ts": 1788171058.0373404}}
|
||||
{"prompt_id": 42, "output": " To start the evening, there were 5 pies, each with 8 pieces, which is 5*8=<<5*8=40>>40 pieces of pie.\nIf only 14 remained, then 40-14=<<40-14=26>>26 pieces of pie had been taken by guests.\n#### 26\n\n", "correct": true, "error": null, "completion_tokens": 70, "spec_accept_length": 7.0, "spec_accept_length_from_histogram": 7.1, "spec_accept_rate": 0.4066666666666667, "spec_accepted_drafts": 61, "spec_proposed_drafts": 150, "spec_accept_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "spec_verify_ct": 10, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a53b456bfc23438e9a2e67b9df953660", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 746, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171054.899614, "queue_time": 0.00020623207092285156, "reasoning_tokens": 0, "completion_tokens": 70, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 70}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.911694198381156, "spec_accept_rate": 0.4066666666666667, "spec_accept_length": 7.0, "spec_num_correct_drafts": 61, "spec_num_proposed_drafts": 150, "spec_verify_ct": 10, "spec_accepted_drafts": 61, "spec_proposed_drafts": 150, "spec_correct_drafts_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "spec_accept_histogram": [0, 0, 1, 2, 1, 0, 1, 2, 0, 2, 0, 1], "request_received_ts": 1788171051.6740005, "api_server_dispatch_finish_ts": 1788171051.6771235, "request_finished_ts": 1788171057.5856948, "decode_throughput": 21574893.959231216, "response_sent_to_client_ts": 1788171057.5862522}}
|
||||
{"prompt_id": 43, "output": " If the total calorie target is 2000 and I have consumed 1800 calories then I have 2000-1800 = <<2000-1800=200>>200 calories left to eat\nIf each serving of chips has 250 calories and I only have 200 calories left to eat, then I can only eat 200/250 of a serving = 4/5 of a serving\nWe also know that a 300g bag of chips has 5 servings, hence each serving has 300g/5 = <<300/5=60>>60 grams\nIf I can only eat 4/5 of a serving, then I can eat only 60g * 4/5 = 240g/5 = 48 grams\n#### 48\n\n", "correct": true, "error": null, "completion_tokens": 161, "spec_accept_length": 6.1923076923076925, "spec_accept_length_from_histogram": 6.230769230769231, "spec_accept_rate": 0.3487179487179487, "spec_accepted_drafts": 136, "spec_proposed_drafts": 390, "spec_accept_histogram": [0, 0, 6, 3, 3, 1, 5, 3, 2, 1, 0, 2], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c324b46d95b64ce485016ba65bfcb773", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 721, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171055.0815876, "queue_time": 0.00018906593322753906, "reasoning_tokens": 0, "completion_tokens": 161, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 161}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.562264379113913, "spec_accept_rate": 0.3487179487179487, "spec_accept_length": 6.1923076923076925, "spec_num_correct_drafts": 136, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 136, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [0, 0, 6, 3, 3, 1, 5, 3, 2, 1, 0, 2], "spec_accept_histogram": [0, 0, 6, 3, 3, 1, 5, 3, 2, 1, 0, 2], "request_received_ts": 1788171052.1747992, "api_server_dispatch_finish_ts": 1788171052.1780963, "request_finished_ts": 1788171061.7370636, "decode_throughput": 52.739358904895894, "response_sent_to_client_ts": 1788171061.7374933}}
|
||||
{"prompt_id": 44, "output": " 1 pound of beeswax and the wick make 10 candles and he wants to make 20 candles so he needs 20/10 = <<20/10=2>>2 pounds of beeswax\n1 pound of beeswax costs $10.00 and he needs 2 pounds so his supplies will cost 10*2 = $<<10*2=20.00>>20.00\nHe wants to sell each candle for $2.00 each and he's selling 20 candles so that's 2*20 = $<<2*20=40.00>>40.00\nIf he makes $40.00 selling candles and his supplies cost $20.00 then his net profit is 40-20 = $<<40-20=20.00>>20.00\n#### 20\n\n", "correct": true, "error": null, "completion_tokens": 168, "spec_accept_length": 6.461538461538462, "spec_accept_length_from_histogram": 6.576923076923077, "spec_accept_rate": 0.3717948717948718, "spec_accepted_drafts": 145, "spec_proposed_drafts": 390, "spec_accept_histogram": [1, 0, 5, 2, 2, 2, 4, 4, 2, 0, 2, 1, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "16aa005c2589494da80b65ec689d851f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 737, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171056.2749515, "queue_time": 0.00014138221740722656, "reasoning_tokens": 0, "completion_tokens": 168, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 168}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 9.147357358131558, "spec_accept_rate": 0.3717948717948718, "spec_accept_length": 6.461538461538462, "spec_num_correct_drafts": 145, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 145, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [1, 0, 5, 2, 2, 2, 4, 4, 2, 0, 2, 1, 1], "spec_accept_histogram": [1, 0, 5, 2, 2, 2, 4, 4, 2, 0, 2, 1, 1], "request_received_ts": 1788171053.611392, "api_server_dispatch_finish_ts": 1788171053.614705, "request_finished_ts": 1788171062.7587492, "decode_throughput": 50390581.59561613, "response_sent_to_client_ts": 1788171062.7595296}}
|
||||
{"prompt_id": 45, "output": " If she wrote five articles on Monday, then on Tuesday she wrote 2/5* 5 = 2 more articles.\nThe total number of articles she wrote on Tuesday is 5+2 = <<5+2=7>>7\nOn Wednesday, the number of articles was twice the number she wrote on Tuesday, so it was 2*7 = <<2*7=14>>14 articles.\nOver the three days, she wrote 5+7 +14 = <<5+7+14=26>>26 articles.\nIf each article takes her 4 hours to research and write about, she spent 4*26 = <<4*26=104>>104 hours on all the articles.\n#### 104\n\n", "correct": true, "error": null, "completion_tokens": 150, "spec_accept_length": 6.25, "spec_accept_length_from_histogram": 6.291666666666667, "spec_accept_rate": 0.3527777777777778, "spec_accepted_drafts": 127, "spec_proposed_drafts": 360, "spec_accept_histogram": [0, 0, 2, 5, 5, 3, 1, 1, 4, 2, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "4eec52121b3c432ba26b586363bbbefa", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 760, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171056.8366196, "queue_time": 0.00013518333435058594, "reasoning_tokens": 0, "completion_tokens": 150, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 150}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 8.343348165974021, "spec_accept_rate": 0.3527777777777778, "spec_accept_length": 6.25, "spec_num_correct_drafts": 127, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 127, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [0, 0, 2, 5, 5, 3, 1, 1, 4, 2, 1], "spec_accept_histogram": [0, 0, 2, 5, 5, 3, 1, 1, 4, 2, 1], "request_received_ts": 1788171054.4157987, "api_server_dispatch_finish_ts": 1788171054.4190912, "request_finished_ts": 1788171062.759147, "decode_throughput": 38.772243314189815, "response_sent_to_client_ts": 1788171062.7604156}}
|
||||
{"prompt_id": 46, "output": " Let's let x be the number of post-it notes in the package that Candice purchased.\nThen, the total number of post-it notes she had when she left the store was 80+x.\nAnd if she used 220 Post-it notes at work and had 23 left, then the total number she had when she left the store was 220+23\nSubtracting 80 from each side of the equation, we see that x=220+23-80=<<220+23-80=163>>163 Post-it notes.\n#### 163\n\n", "correct": true, "error": null, "completion_tokens": 114, "spec_accept_length": 4.956521739130435, "spec_accept_length_from_histogram": 5.0, "spec_accept_rate": 0.26666666666666666, "spec_accepted_drafts": 92, "spec_proposed_drafts": 345, "spec_accept_histogram": [2, 3, 4, 4, 0, 3, 1, 2, 2, 2], "spec_verify_ct": 23, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "210b5cb281f54b84bad2e218aa325340", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 757, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171058.2497754, "queue_time": 0.00016808509826660156, "reasoning_tokens": 0, "completion_tokens": 114, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 114}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.858733362983912, "spec_accept_rate": 0.26666666666666666, "spec_accept_length": 4.956521739130435, "spec_num_correct_drafts": 92, "spec_num_proposed_drafts": 345, "spec_verify_ct": 23, "spec_accepted_drafts": 92, "spec_proposed_drafts": 345, "spec_correct_drafts_histogram": [2, 3, 4, 4, 0, 3, 1, 2, 2, 2], "spec_accept_histogram": [2, 3, 4, 4, 0, 3, 1, 2, 2, 2], "request_received_ts": 1788171056.0451891, "api_server_dispatch_finish_ts": 1788171056.0484638, "request_finished_ts": 1788171063.9039226, "decode_throughput": 37127547.76988984, "response_sent_to_client_ts": 1788171063.9046683}}
|
||||
{"prompt_id": 47, "output": " He bought 200 / 40 = <<200/40=5>>5 blue ties\nSo he bought 5 * 2 = <<5*2=10>>10 red ties\nEach red tie cost $40 * .5 = $<<40*.5=20>>20 more than blue ties\nSo they each cost $40 + $20 = $<<40+20=60>>60\nThus he spent $60 * 10 = $<<60*10=600>>600 on red ties\nSo he spent $200 + $600 = $<<200+600=800>>800 on ties\n#### 800\n\n", "correct": true, "error": null, "completion_tokens": 131, "spec_accept_length": 6.238095238095238, "spec_accept_length_from_histogram": 6.285714285714286, "spec_accept_rate": 0.3523809523809524, "spec_accepted_drafts": 111, "spec_proposed_drafts": 315, "spec_accept_histogram": [0, 0, 3, 5, 3, 2, 2, 1, 2, 2, 0, 0, 0, 0, 0, 1], "spec_verify_ct": 21, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5ab7906969db41c3846c3b6e2507ceb7", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 714, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171059.1328783, "queue_time": 0.00013518333435058594, "reasoning_tokens": 0, "completion_tokens": 131, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 131}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.916867175139487, "spec_accept_rate": 0.3523809523809524, "spec_accept_length": 6.238095238095238, "spec_num_correct_drafts": 111, "spec_num_proposed_drafts": 315, "spec_verify_ct": 21, "spec_accepted_drafts": 111, "spec_proposed_drafts": 315, "spec_correct_drafts_histogram": [0, 0, 3, 5, 3, 2, 2, 1, 2, 2, 0, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 3, 5, 3, 2, 2, 1, 2, 2, 0, 0, 0, 0, 0, 1], "request_received_ts": 1788171056.5916772, "api_server_dispatch_finish_ts": 1788171056.5948532, "request_finished_ts": 1788171064.5085442, "decode_throughput": 35823543.46721417, "response_sent_to_client_ts": 1788171064.5092762}}
|
||||
{"prompt_id": 48, "output": " If Tracy used a piece of wire 4 feet long, since 1 foot is equal to 12 inches, the total length of the wire in inches was 12*4=<<12*4=48>>48 inches.\nSince the wire was cut into pieces 6 inches long, Tracy obtained 48/6=<<48/6=8>>8 pieces\n#### 8\n\n", "correct": true, "error": null, "completion_tokens": 81, "spec_accept_length": 6.230769230769231, "spec_accept_length_from_histogram": 6.3076923076923075, "spec_accept_rate": 0.35384615384615387, "spec_accepted_drafts": 69, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "0fb45f3af2444261b6a5ab75d287f36c", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 704, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171060.28571, "queue_time": 0.0001742839813232422, "reasoning_tokens": 0, "completion_tokens": 81, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 81}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.307476520072669, "spec_accept_rate": 0.35384615384615387, "spec_accept_length": 6.230769230769231, "spec_num_correct_drafts": 69, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 69, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 3, 0, 2, 1, 0, 1, 2, 0, 0, 0, 1], "request_received_ts": 1788171057.5967891, "api_server_dispatch_finish_ts": 1788171057.6000016, "request_finished_ts": 1788171063.9042656, "decode_throughput": 37388181.03155604, "response_sent_to_client_ts": 1788171063.9055488}}
|
||||
{"prompt_id": 49, "output": " The total number of units in the building will be 8 units/floor * 15 floors = <<8*15=120>>120 units.\nIf 3/4 of the building is occupied, then the total number of occupied units is 3/4 * 120 units = <<3/4*120=90>>90 units.\nThe total number of unoccupied units is 120 units - 90 units = <<120-90=30>>30 units.\n#### 30\n\n", "correct": true, "error": null, "completion_tokens": 101, "spec_accept_length": 6.733333333333333, "spec_accept_length_from_histogram": 6.866666666666666, "spec_accept_rate": 0.39111111111111113, "spec_accepted_drafts": 88, "spec_proposed_drafts": 225, "spec_accept_histogram": [0, 2, 1, 2, 1, 1, 3, 2, 0, 1, 0, 0, 0, 1, 0, 1], "spec_verify_ct": 15, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "5c320415d3fa4930b70ba75ab79216c7", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 709, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171060.537147, "queue_time": 0.00012636184692382812, "reasoning_tokens": 0, "completion_tokens": 101, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 101}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.461538522038609, "spec_accept_rate": 0.39111111111111113, "spec_accept_length": 6.733333333333333, "spec_num_correct_drafts": 88, "spec_num_proposed_drafts": 225, "spec_verify_ct": 15, "spec_accepted_drafts": 88, "spec_proposed_drafts": 225, "spec_correct_drafts_histogram": [0, 2, 1, 2, 1, 1, 3, 2, 0, 1, 0, 0, 0, 1, 0, 1], "spec_accept_histogram": [0, 2, 1, 2, 1, 1, 3, 2, 0, 1, 0, 0, 0, 1, 0, 1], "request_received_ts": 1788171058.0473526, "api_server_dispatch_finish_ts": 1788171058.0505698, "request_finished_ts": 1788171064.508891, "decode_throughput": 52569979.14320686, "response_sent_to_client_ts": 1788171064.5101848}}
|
||||
{"prompt_id": 50, "output": " In a week, Lloyd's egg farm produces 252 x 7 = <<252*7=1764>>1764 eggs.\nThere are 1764/12 = <<1764/12=147>>147 dozens of eggs produced in a week.\nSo, Lloyd makes 147 x $2 = $<<147*2=294>>294 on eggs.\n#### 294\n\n", "correct": true, "error": null, "completion_tokens": 79, "spec_accept_length": 5.642857142857143, "spec_accept_length_from_histogram": 5.714285714285714, "spec_accept_rate": 0.3142857142857143, "spec_accepted_drafts": 66, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "baefa51a7dae426aaedf1d221729aa86", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 702, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171062.5055192, "queue_time": 0.0001933574676513672, "reasoning_tokens": 0, "completion_tokens": 79, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 79}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.607200863305479, "spec_accept_rate": 0.3142857142857143, "spec_accept_length": 5.642857142857143, "spec_num_correct_drafts": 66, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 66, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "spec_accept_histogram": [0, 1, 2, 2, 2, 3, 1, 0, 2, 0, 1], "request_received_ts": 1788171060.8354845, "api_server_dispatch_finish_ts": 1788171060.838699, "request_finished_ts": 1788171065.4426854, "decode_throughput": 19978974.778625954, "response_sent_to_client_ts": 1788171065.4431963}}
|
||||
{"prompt_id": 51, "output": " He was travelling at full speed for 4-1=<<4-1=3>>3 hours\nSo he went 3*10=<<3*10=30>>30 miles\nThat means it takes 30/6=<<30/6=5>>5 hours to get back\n#### 5\n\n", "correct": true, "error": null, "completion_tokens": 67, "spec_accept_length": 6.090909090909091, "spec_accept_length_from_histogram": 6.181818181818182, "spec_accept_rate": 0.34545454545454546, "spec_accepted_drafts": 57, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e052601263e647209696704d5bd87cf8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 716, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171063.6937482, "queue_time": 0.00014662742614746094, "reasoning_tokens": 0, "completion_tokens": 67, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 67}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.047629018314183, "spec_accept_rate": 0.34545454545454546, "spec_accept_length": 6.090909090909091, "spec_num_correct_drafts": 57, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 57, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "spec_accept_histogram": [0, 2, 1, 2, 0, 2, 0, 0, 0, 3, 1], "request_received_ts": 1788171061.7477229, "api_server_dispatch_finish_ts": 1788171061.7509506, "request_finished_ts": 1788171065.7953517, "decode_throughput": 15158708.103529412, "response_sent_to_client_ts": 1788171065.795834}}
|
||||
{"prompt_id": 52, "output": " 30 comic books weigh 7.5 pounds because 30 x .25 = <<30*.25=7.5>>7.5\nHe needs to remove 7.5 more pounds because 15 - 7.5 = <<15-7.5=7.5>>7.5\nHe needs to remove 15 toys because 7.5 / .5 = <<15=15>>15\n#### 15\n\n", "correct": true, "error": null, "completion_tokens": 92, "spec_accept_length": 8.363636363636363, "spec_accept_length_from_histogram": 8.454545454545453, "spec_accept_rate": 0.49696969696969695, "spec_accepted_drafts": 82, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 0, 1, 0, 3, 2, 0, 2, 0, 1, 1, 0, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "a5023d117df2456c8376f26f7809898b", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 728, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171065.1119113, "queue_time": 0.00012946128845214844, "reasoning_tokens": 0, "completion_tokens": 92, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 92}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.139327056705952, "spec_accept_rate": 0.49696969696969695, "spec_accept_length": 8.363636363636363, "spec_num_correct_drafts": 82, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 82, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 0, 1, 0, 3, 2, 0, 2, 0, 1, 1, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 0, 1, 0, 3, 2, 0, 2, 0, 1, 1, 0, 0, 0, 1], "request_received_ts": 1788171062.7804968, "api_server_dispatch_finish_ts": 1788171062.7833636, "request_finished_ts": 1788171066.919824, "decode_throughput": 23607273.733752113, "response_sent_to_client_ts": 1788171066.9203212}}
|
||||
{"prompt_id": 53, "output": " On Thursday the mechanic charged a total of 6*60 + 4*40 = $<<6*60+4*40=520>>520.\nOn Friday the mechanic charged a total of 12*40 = $<<12*40=480>>480.\nThus, on Thursday, the day with higher revenue, the mechanic had 520-480 = $<<520-480=40>>40 more revenue than on Friday.\n#### 40\n\n", "correct": true, "error": null, "completion_tokens": 94, "spec_accept_length": 6.714285714285714, "spec_accept_length_from_histogram": 6.785714285714286, "spec_accept_rate": 0.38571428571428573, "spec_accepted_drafts": 81, "spec_proposed_drafts": 210, "spec_accept_histogram": [0, 0, 1, 5, 0, 1, 2, 1, 1, 1, 1, 0, 0, 1], "spec_verify_ct": 14, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e69798cc347147f6ac632826e58eb37f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 764, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171064.8335547, "queue_time": 0.000213623046875, "reasoning_tokens": 0, "completion_tokens": 94, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 94}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.33417454501614, "spec_accept_rate": 0.38571428571428573, "spec_accept_length": 6.714285714285714, "spec_num_correct_drafts": 81, "spec_num_proposed_drafts": 210, "spec_verify_ct": 14, "spec_accepted_drafts": 81, "spec_proposed_drafts": 210, "spec_correct_drafts_histogram": [0, 0, 1, 5, 0, 1, 2, 1, 1, 1, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 5, 0, 1, 2, 1, 1, 1, 1, 0, 0, 1], "request_received_ts": 1788171062.7757843, "api_server_dispatch_finish_ts": 1788171062.779086, "request_finished_ts": 1788171067.109959, "decode_throughput": 37682260.23849057, "response_sent_to_client_ts": 1788171067.1104217}}
|
||||
{"prompt_id": 54, "output": " Patchy has just had 3 * 7 = <<3*7=21>>21 kittens\nSince Trixie's kittens are 12, both cats have 21 +12 =<<21+12=33>>33 kittens.\nCombining the adopted kittens with those from the two cats, the Doubtfire family now has 7 + 33 = <<7+33=40>>40 kittens.\n#### 40\n\n", "correct": true, "error": null, "completion_tokens": 88, "spec_accept_length": 5.5, "spec_accept_length_from_histogram": 5.5625, "spec_accept_rate": 0.30416666666666664, "spec_accepted_drafts": 73, "spec_proposed_drafts": 240, "spec_accept_histogram": [0, 3, 2, 2, 3, 1, 0, 1, 1, 2, 1], "spec_verify_ct": 16, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "2ec25579ebe94842a7a337bbbb515248", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 749, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171066.4627063, "queue_time": 0.00015425682067871094, "reasoning_tokens": 0, "completion_tokens": 88, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 88}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.130107218399644, "spec_accept_rate": 0.30416666666666664, "spec_accept_length": 5.5, "spec_num_correct_drafts": 73, "spec_num_proposed_drafts": 240, "spec_verify_ct": 16, "spec_accepted_drafts": 73, "spec_proposed_drafts": 240, "spec_correct_drafts_histogram": [0, 3, 2, 2, 3, 1, 0, 1, 1, 2, 1], "spec_accept_histogram": [0, 3, 2, 2, 3, 1, 0, 1, 1, 2, 1], "request_received_ts": 1788171063.9261012, "api_server_dispatch_finish_ts": 1788171063.929208, "request_finished_ts": 1788171069.0562084, "decode_throughput": 12615197.66212019, "response_sent_to_client_ts": 1788171069.057062}}
|
||||
{"prompt_id": 55, "output": " Jean has 30 - 2 = <<30-2=28>>28 lollipops\nJean can fill up 28 / 2 = <<28/2=14>>14 bags with lollipops\n#### 14\n\n", "correct": true, "error": null, "completion_tokens": 49, "spec_accept_length": 6.125, "spec_accept_length_from_histogram": 6.25, "spec_accept_rate": 0.35, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_accept_histogram": [0, 0, 3, 1, 1, 0, 0, 0, 0, 1, 2], "spec_verify_ct": 8, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "e9533bf1ff3f4e02a6106e9f1ff7a6f0", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 713, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171066.5961456, "queue_time": 0.0001575946807861328, "reasoning_tokens": 0, "completion_tokens": 49, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 49}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 3.9816475389525294, "spec_accept_rate": 0.35, "spec_accept_length": 6.125, "spec_num_correct_drafts": 42, "spec_num_proposed_drafts": 120, "spec_verify_ct": 8, "spec_accepted_drafts": 42, "spec_proposed_drafts": 120, "spec_correct_drafts_histogram": [0, 0, 3, 1, 1, 0, 0, 0, 0, 1, 2], "spec_accept_histogram": [0, 0, 3, 1, 1, 0, 0, 0, 0, 1, 2], "request_received_ts": 1788171063.9215264, "api_server_dispatch_finish_ts": 1788171063.9247196, "request_finished_ts": 1788171067.903174, "decode_throughput": 11672428.389083909, "response_sent_to_client_ts": 1788171067.903747}}
|
||||
{"prompt_id": 56, "output": " It costs him $14 to go to the movies because 7 + 7 = <<7+7=14>>14\nHe can go to the movies 3 times because 42 / 14 = <<42/14=3>>3\n#### 3\n\n", "correct": true, "error": null, "completion_tokens": 56, "spec_accept_length": 6.222222222222222, "spec_accept_length_from_histogram": 6.333333333333333, "spec_accept_rate": 0.35555555555555557, "spec_accepted_drafts": 48, "spec_proposed_drafts": 135, "spec_accept_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "spec_verify_ct": 9, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "19ee8bce1d2e4b18b22dfe2522c2ec32", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 712, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171067.77493, "queue_time": 0.00015306472778320312, "reasoning_tokens": 0, "completion_tokens": 56, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 56}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 4.934992738999426, "spec_accept_rate": 0.35555555555555557, "spec_accept_length": 6.222222222222222, "spec_num_correct_drafts": 48, "spec_num_proposed_drafts": 135, "spec_verify_ct": 9, "spec_accepted_drafts": 48, "spec_proposed_drafts": 135, "spec_correct_drafts_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 1, 2, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1], "request_received_ts": 1788171064.5249224, "api_server_dispatch_finish_ts": 1788171064.528096, "request_finished_ts": 1788171069.4599152, "decode_throughput": 13895482.428235294, "response_sent_to_client_ts": 1788171069.4603887}}
|
||||
{"prompt_id": 57, "output": " The boxes can weigh up to 5000 pounds - 3755 pounds = <<5000-3755=1245>>1245 pounds in total.\nThere can be 1245 / 15 = <<1245/15=83>>83 boxes loaded onto the truck without exceeding the bridge's weight limit.\n#### 83\n\n", "correct": true, "error": null, "completion_tokens": 69, "spec_accept_length": 6.2727272727272725, "spec_accept_length_from_histogram": 6.363636363636363, "spec_accept_rate": 0.3575757575757576, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "ba31f25880b14c168c40a5a0f5b25dfb", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 740, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171068.0578175, "queue_time": 0.00015425682067871094, "reasoning_tokens": 0, "completion_tokens": 69, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 69}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.419019355904311, "spec_accept_rate": 0.3575757575757576, "spec_accept_length": 6.2727272727272725, "spec_num_correct_drafts": 59, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "spec_accept_histogram": [0, 0, 2, 0, 0, 4, 3, 1, 0, 0, 1], "request_received_ts": 1788171064.5295408, "api_server_dispatch_finish_ts": 1788171064.532471, "request_finished_ts": 1788171069.9485602, "decode_throughput": 19473114.823843177, "response_sent_to_client_ts": 1788171069.9490383}}
|
||||
{"prompt_id": 58, "output": " He spent $40.00 on groceries but they charged him a 25% fee so that's 40*.25 = $10.00\nThere is also a $3.00 delivery fee and a $4.00 tip for an extra $3 + $4 = $<<3+4=7.00>>7.00 in fees\nHis groceries were $40.00, there's a $10.00 fee and another $7.00 in delivery/tip fees for a final total of $40 + $10 + $7 = $<<40+10+7=57.00>>57.00\n#### 57\n\n", "correct": true, "error": null, "completion_tokens": 132, "spec_accept_length": 5.076923076923077, "spec_accept_length_from_histogram": 5.115384615384615, "spec_accept_rate": 0.2743589743589744, "spec_accepted_drafts": 107, "spec_proposed_drafts": 390, "spec_accept_histogram": [0, 5, 5, 4, 1, 3, 2, 4, 0, 1, 0, 0, 1], "spec_verify_ct": 26, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "8dcf1b0f471a4219b7c7121db86e8b6f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 747, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171068.538077, "queue_time": 0.0001659393310546875, "reasoning_tokens": 0, "completion_tokens": 132, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 132}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 8.250277529936284, "spec_accept_rate": 0.2743589743589744, "spec_accept_length": 5.076923076923077, "spec_num_correct_drafts": 107, "spec_num_proposed_drafts": 390, "spec_verify_ct": 26, "spec_accepted_drafts": 107, "spec_proposed_drafts": 390, "spec_correct_drafts_histogram": [0, 5, 5, 4, 1, 3, 2, 4, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 5, 5, 4, 1, 3, 2, 4, 0, 1, 0, 0, 1], "request_received_ts": 1788171065.4539702, "api_server_dispatch_finish_ts": 1788171065.4572413, "request_finished_ts": 1788171073.7042477, "decode_throughput": 44.83700248453688, "response_sent_to_client_ts": 1788171073.7047384}}
|
||||
{"prompt_id": 59, "output": " First find the number of fruit in clusters: 20 fruit/cluster * 6 clusters = <<20*6=120>>120 fruit\nThen add the number of individual fruit to find the total number of fruit: 120 fruit + 67 fruit = <<120+67=187>>187 fruit\n#### 187\n\n", "correct": true, "error": null, "completion_tokens": 66, "spec_accept_length": 5.076923076923077, "spec_accept_length_from_histogram": 5.153846153846154, "spec_accept_rate": 0.27692307692307694, "spec_accepted_drafts": 54, "spec_proposed_drafts": 195, "spec_accept_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "spec_verify_ct": 13, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "3174004539a74cfe922578e48c3051a6", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 697, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171068.873423, "queue_time": 0.00012636184692382812, "reasoning_tokens": 0, "completion_tokens": 66, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 66}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.64719158783555, "spec_accept_rate": 0.27692307692307694, "spec_accept_length": 5.076923076923077, "spec_num_correct_drafts": 54, "spec_num_proposed_drafts": 195, "spec_verify_ct": 13, "spec_accepted_drafts": 54, "spec_proposed_drafts": 195, "spec_correct_drafts_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "spec_accept_histogram": [0, 2, 1, 4, 1, 3, 0, 0, 1, 0, 0, 1], "request_received_ts": 1788171065.8073256, "api_server_dispatch_finish_ts": 1788171065.810489, "request_finished_ts": 1788171071.4545171, "decode_throughput": 145.5518742832151, "response_sent_to_client_ts": 1788171071.455095}}
|
||||
{"prompt_id": 60, "output": " 20% of 25 oranges is (20/100)*25 = <<(20/100)*25=5>>5 oranges\nThe total number of bad, unripe and sour oranges is therefore 1+5+2 = <<1+5+2=8>>8 oranges\nThe rest which is 25-8 = <<25-8=17>>17 oranges are good\n#### 17\n\n", "correct": true, "error": null, "completion_tokens": 86, "spec_accept_length": 5.733333333333333, "spec_accept_length_from_histogram": 5.866666666666666, "spec_accept_rate": 0.3244444444444444, "spec_accepted_drafts": 73, "spec_proposed_drafts": 225, "spec_accept_histogram": [0, 2, 2, 2, 2, 3, 0, 0, 1, 2, 0, 0, 1], "spec_verify_ct": 15, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "c20ad8afbdbc4950b349fd2c5a9a3d6d", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 704, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171070.0984845, "queue_time": 0.00012946128845214844, "reasoning_tokens": 0, "completion_tokens": 86, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 86}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 6.288965605199337, "spec_accept_rate": 0.3244444444444444, "spec_accept_length": 5.733333333333333, "spec_num_correct_drafts": 73, "spec_num_proposed_drafts": 225, "spec_verify_ct": 15, "spec_accepted_drafts": 73, "spec_proposed_drafts": 225, "spec_correct_drafts_histogram": [0, 2, 2, 2, 2, 3, 0, 0, 1, 2, 0, 0, 1], "spec_accept_histogram": [0, 2, 2, 2, 2, 3, 0, 0, 1, 2, 0, 0, 1], "request_received_ts": 1788171066.930689, "api_server_dispatch_finish_ts": 1788171066.9339535, "request_finished_ts": 1788171073.2196548, "decode_throughput": 25633493.90254178, "response_sent_to_client_ts": 1788171073.2200944}}
|
||||
{"prompt_id": 61, "output": " The total cost of the broch was 500+800=$<<500+800=1300>>1300\nThe insurance was 1300*.1=$<<1300*.1=130>>130\nSo she pays 1300+130=$<<1300+130=1430>>1430\n#### 1430\n\n", "correct": true, "error": null, "completion_tokens": 69, "spec_accept_length": 6.2727272727272725, "spec_accept_length_from_histogram": 6.363636363636363, "spec_accept_rate": 0.3575757575757576, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_accept_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "spec_verify_ct": 11, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "1fced9f4ebbd44cfba5ac345da9ddd98", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 724, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171070.2662716, "queue_time": 0.00012826919555664062, "reasoning_tokens": 0, "completion_tokens": 69, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 69}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 5.699625255074352, "spec_accept_rate": 0.3575757575757576, "spec_accept_length": 6.2727272727272725, "spec_num_correct_drafts": 59, "spec_num_proposed_drafts": 165, "spec_verify_ct": 11, "spec_accepted_drafts": 59, "spec_proposed_drafts": 165, "spec_correct_drafts_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "spec_accept_histogram": [0, 0, 1, 1, 2, 3, 2, 1, 0, 0, 0, 0, 1], "request_received_ts": 1788171067.1207526, "api_server_dispatch_finish_ts": 1788171067.1240063, "request_finished_ts": 1788171072.8203778, "decode_throughput": 19946576.705914494, "response_sent_to_client_ts": 1788171072.820937}}
|
||||
{"prompt_id": 62, "output": " First find how many years Marcy works after 20 years: 30 years - 20 years = <<30-20=10>>10 years\nThen multiply that number by the amount of her pension she gets per year: 10 years * 5% = 50%\nThen multiply that percentage by the total value of the pension to find how much she gets: $50,000 * 50% = $<<50000*50*.01=25000>>25,000\n#### 25000\n\n", "correct": true, "error": null, "completion_tokens": 106, "spec_accept_length": 4.416666666666667, "spec_accept_length_from_histogram": 4.458333333333334, "spec_accept_rate": 0.23055555555555557, "spec_accepted_drafts": 83, "spec_proposed_drafts": 360, "spec_accept_histogram": [1, 3, 4, 5, 5, 3, 0, 2, 1], "spec_verify_ct": 24, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "b45c310e65904dadbc38a55907b94f8f", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 731, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171070.6017368, "queue_time": 0.00012564659118652344, "reasoning_tokens": 0, "completion_tokens": 106, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 106}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.041785742621869, "spec_accept_rate": 0.23055555555555557, "spec_accept_length": 4.416666666666667, "spec_num_correct_drafts": 83, "spec_num_proposed_drafts": 360, "spec_verify_ct": 24, "spec_accepted_drafts": 83, "spec_proposed_drafts": 360, "spec_correct_drafts_histogram": [1, 3, 4, 5, 5, 3, 0, 2, 1], "spec_accept_histogram": [1, 3, 4, 5, 5, 3, 0, 2, 1], "request_received_ts": 1788171067.9173245, "api_server_dispatch_finish_ts": 1788171067.9230635, "request_finished_ts": 1788171074.9591103, "decode_throughput": 24305894.474506844, "response_sent_to_client_ts": 1788171074.9597068}}
|
||||
{"prompt_id": 63, "output": " A year has twelve months, so a half of the year is 12/2 = <<12/2=6>>6 months,\nFor the first six months, the streaming service charged her a total of 6*140 = $<<6*140=840>>840\nFor the next half of the year, the streaming service company charged her 10/100*140 = $<<10/100*140=14>>14 less\nShe paid $140-$14 = $<<140-14=126>>126 each month for the remaining half of the year,\nIn total, she paid 126*6 = $<<126*6=756>>756 for the other half of the year for the streaming service.\nThe total amount of money she paid at the end of the year is $840+$756 = $<<840+756=1596>>1596\n#### 1596\n\n", "correct": true, "error": null, "completion_tokens": 185, "spec_accept_length": 5.606060606060606, "spec_accept_length_from_histogram": 5.636363636363637, "spec_accept_rate": 0.3090909090909091, "spec_accepted_drafts": 153, "spec_proposed_drafts": 495, "spec_accept_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "spec_verify_ct": 33, "finish_reason": {"type": "stop", "matched": "Question"}, "meta_info": {"id": "9e1861ca22984ac48fba15cc0ae91ba8", "finish_reason": {"type": "stop", "matched": "Question"}, "prompt_tokens": 734, "weight_version": "default", "num_retractions": 0, "forward_entry_time": 1788171071.670827, "queue_time": 0.00013375282287597656, "reasoning_tokens": 0, "completion_tokens": 185, "cached_tokens": 0, "weight_versions": [{"version": "default", "start": 0, "end": 185}], "cached_tokens_details": null, "dp_rank": null, "e2e_latency": 7.202895070891827, "spec_accept_rate": 0.3090909090909091, "spec_accept_length": 5.606060606060606, "spec_num_correct_drafts": 153, "spec_num_proposed_drafts": 495, "spec_verify_ct": 33, "spec_accepted_drafts": 153, "spec_proposed_drafts": 495, "spec_correct_drafts_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "spec_accept_histogram": [0, 4, 7, 5, 0, 5, 2, 2, 5, 1, 2], "request_received_ts": 1788171069.0688148, "api_server_dispatch_finish_ts": 1788171069.0722227, "request_finished_ts": 1788171076.27171, "decode_throughput": 140.2172170198619, "response_sent_to_client_ts": 1788171076.2722104}}
|
||||
@ -0,0 +1,28 @@
|
||||
gsm8k_c8 174.1.60.1: docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/dataset/gsm8k.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /dataset/gsm8k.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 8 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c8.jsonl --summary-file /results/bench/gsm8k_c8_summary.json
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/awq/awq.py:53: UserWarning: Only CUDA, HIP and XPU support AWQ currently.
|
||||
warnings.warn(f"Only CUDA, HIP and XPU support AWQ currently.")
|
||||
/opt/kimi-dflash/python/sglang/srt/layers/quantization/gguf.py:69: UserWarning: Only CUDA, MUSA and NPU support GGUF quantization currently.
|
||||
warnings.warn(f"Only CUDA, MUSA and NPU support GGUF quantization currently.")
|
||||
0%| | 0/64 [00:00<?, ?it/s]
2%|▏ | 1/64 [00:04<04:21, 4.16s/it]
3%|▎ | 2/64 [00:05<02:20, 2.27s/it]
5%|▍ | 3/64 [00:05<01:24, 1.39s/it]
6%|▋ | 4/64 [00:06<01:07, 1.13s/it]
8%|▊ | 5/64 [00:06<00:57, 1.03it/s]
9%|▉ | 6/64 [00:07<00:56, 1.04it/s]
11%|█ | 7/64 [00:09<01:06, 1.16s/it]
12%|█▎ | 8/64 [00:10<01:02, 1.11s/it]
14%|█▍ | 9/64 [00:12<01:11, 1.30s/it]
16%|█▌ | 10/64 [00:13<01:08, 1.27s/it]
17%|█▋ | 11/64 [00:13<00:51, 1.03it/s]
19%|█▉ | 12/64 [00:14<00:44, 1.18it/s]
20%|██ | 13/64 [00:14<00:40, 1.25it/s]
22%|██▏ | 14/64 [00:16<00:59, 1.19s/it]
25%|██▌ | 16/64 [00:17<00:33, 1.43it/s]
27%|██▋ | 17/64 [00:18<00:41, 1.13it/s]
28%|██▊ | 18/64 [00:20<00:53, 1.15s/it]
30%|██▉ | 19/64 [00:20<00:42, 1.06it/s]
31%|███▏ | 20/64 [00:22<00:43, 1.02it/s]
33%|███▎ | 21/64 [00:22<00:35, 1.20it/s]
36%|███▌ | 23/64 [00:22<00:22, 1.81it/s]
38%|███▊ | 24/64 [00:23<00:24, 1.62it/s]
41%|████ | 26/64 [00:24<00:21, 1.80it/s]
42%|████▏ | 27/64 [00:25<00:21, 1.69it/s]
44%|████▍ | 28/64 [00:27<00:31, 1.16it/s]
45%|████▌ | 29/64 [00:27<00:24, 1.45it/s]
47%|████▋ | 30/64 [00:27<00:22, 1.50it/s]
48%|████▊ | 31/64 [00:28<00:23, 1.41it/s]
50%|█████ | 32/64 [00:29<00:23, 1.38it/s]
52%|█████▏ | 33/64 [00:30<00:26, 1.15it/s]
53%|█████▎ | 34/64 [00:30<00:20, 1.47it/s]
56%|█████▋ | 36/64 [00:31<00:13, 2.05it/s]
58%|█████▊ | 37/64 [00:32<00:19, 1.39it/s]
59%|█████▉ | 38/64 [00:33<00:19, 1.35it/s]
61%|██████ | 39/64 [00:35<00:24, 1.02it/s]
62%|██████▎ | 40/64 [00:35<00:20, 1.16it/s]
64%|██████▍ | 41/64 [00:36<00:20, 1.11it/s]
66%|██████▌ | 42/64 [00:37<00:16, 1.30it/s]
67%|██████▋ | 43/64 [00:40<00:28, 1.36s/it]
69%|██████▉ | 44/64 [00:41<00:24, 1.23s/it]
70%|███████ | 45/64 [00:42<00:22, 1.17s/it]
73%|███████▎ | 47/64 [00:43<00:15, 1.12it/s]
77%|███████▋ | 49/64 [00:43<00:09, 1.51it/s]
80%|███████▉ | 51/64 [00:44<00:07, 1.69it/s]
81%|████████▏ | 52/64 [00:45<00:06, 1.85it/s]
83%|████████▎ | 53/64 [00:46<00:07, 1.48it/s]
84%|████████▍ | 54/64 [00:46<00:05, 1.80it/s]
86%|████████▌ | 55/64 [00:47<00:05, 1.62it/s]
88%|████████▊ | 56/64 [00:48<00:06, 1.31it/s]
89%|████████▉ | 57/64 [00:48<00:04, 1.51it/s]
91%|█████████ | 58/64 [00:49<00:03, 1.63it/s]
92%|█████████▏| 59/64 [00:50<00:04, 1.15it/s]
94%|█████████▍| 60/64 [00:52<00:04, 1.02s/it]
95%|█████████▌| 61/64 [00:52<00:02, 1.20it/s]
97%|█████████▋| 62/64 [00:52<00:01, 1.37it/s]
98%|█████████▊| 63/64 [00:54<00:00, 1.13it/s]
100%|██████████| 64/64 [00:55<00:00, 1.01s/it]
100%|██████████| 64/64 [00:55<00:00, 1.15it/s]
|
||||
{
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 8,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 7.007018111850178,
|
||||
"median_accept_length_per_question": 6.35064935064935,
|
||||
"min_accept_length_per_question": 4.416666666666667,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 55.545577608048916,
|
||||
"output_throughput": 117.56111433529787
|
||||
}
|
||||
@ -0,0 +1,22 @@
|
||||
{
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 8,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 7.007018111850178,
|
||||
"median_accept_length_per_question": 6.35064935064935,
|
||||
"min_accept_length_per_question": 4.416666666666667,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 55.545577608048916,
|
||||
"output_throughput": 117.56111433529787
|
||||
}
|
||||
@ -0,0 +1 @@
|
||||
{"text":" 5\n\nQuestion: A train travels 60 miles in 1 hour. How far in 3 hours?\nAnswer: 180\n\nQuestion: What is 15% of 200?\nAnswer: 30\n\nQuestion: If x + 5 = 12, what is x?\nAnswer: 7\n\nQuestion: Sarah is twice as old as Tom. Tom is 8. How old is Sarah?\nAnswer: 16\n\nQuestion: A rectangle has length 5 and width 3. What is its area?\nAnswer: 15\n\nQuestion: What comes next: 2, 4, 6, ","output_ids":[220,20,382,29328,25,401,9010,41333,220,2205,12034,306,220,16,9185,13,3653,4972,306,220,18,5469,1820,33853,25,220,7924,382,29328,25,5071,387,220,1308,4,318,220,1208,1820,33853,25,220,1341,382,29328,25,1892,1288,715,220,20,327,220,1042,11,1573,387,1288,1820,33853,25,220,22,382,29328,25,32221,387,16471,559,3410,559,14184,13,14184,387,220,23,13,3653,3410,387,32221,1820,33853,25,220,1285,382,29328,25,401,40864,924,5315,220,20,316,4260,220,18,13,5071,387,1682,3602,1820,33853,25,220,1308,382,29328,25,5071,5123,2897,25,220,17,11,220,19,11,220,21,11,220],"meta_info":{"id":"5c401e6bf6534cd5a4e291700ad5aa27","finish_reason":{"type":"length","length":128},"prompt_tokens":22,"weight_version":"default","num_retractions":0,"forward_entry_time":1788170122.105658,"queue_time":0.00035190582275390625,"reasoning_tokens":0,"completion_tokens":128,"cached_tokens":0,"weight_versions":[{"version":"default","start":0,"end":128}],"cached_tokens_details":null,"dp_rank":null,"e2e_latency":5.7867260607890785,"spec_accept_rate":0.17714285714285713,"spec_accept_length":3.657142857142857,"spec_num_correct_drafts":93,"spec_num_proposed_drafts":525,"spec_verify_ct":35,"spec_accepted_drafts":93,"spec_proposed_drafts":525,"spec_correct_drafts_histogram":[8,7,7,3,3,1,1,2,2,0,1],"spec_accept_histogram":[8,7,7,3,3,1,1,2,2,0,1],"request_received_ts":1788170119.6363459,"api_server_dispatch_finish_ts":1788170119.6378887,"request_finished_ts":1788170125.4230719,"decode_throughput":22702940.422542244,"response_sent_to_client_ts":1788170125.4237604}}
|
||||
@ -0,0 +1 @@
|
||||
0
|
||||
@ -0,0 +1 @@
|
||||
0
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_d_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.5 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_d_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.6 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_d_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.7 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_d_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.8 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/dataset/gsm8k.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /dataset/gsm8k.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 1 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c1.jsonl --summary-file /results/bench/gsm8k_c1_summary.json
|
||||
@ -0,0 +1 @@
|
||||
docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 1 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c1.jsonl --summary-file /results/bench/gsm8k_c1_summary.json
|
||||
@ -0,0 +1 @@
|
||||
docker run --rm --network host -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/../../../datasets/gsm8k/test.jsonl:/dataset/gsm8k.jsonl:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/bench_gsm8k_acceptance.py:/bench.py:ro -v /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_validation/results/pd-dflash-kvbounds1-20260831-1740:/results --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 /bench.py --data-path /dataset/gsm8k.jsonl --num-questions 64 --num-shots 5 --max-new-tokens 512 --temperature 0 --top-p 1 --parallel 8 --speculative-algorithm DFLASH --host 174.1.60.1 --port 31000 --backend srt --output-file /results/bench/gsm8k_c8.jsonl --summary-file /results/bench/gsm8k_c8_summary.json
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_p_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --speculative-draft-kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.88 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 --enable-metrics --host 0.0.0.0 --port 30000
|
||||
@ -0,0 +1 @@
|
||||
docker run -d --name kimi3_pro6000_pd_dflash_validation_router_0 --network host --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang_router.launch_router --pd-disaggregation --mini-lb --prefill http://174.1.60.1:30000 28800 --decode http://174.1.60.5:30000 --host 0.0.0.0 --port 31000
|
||||
@ -0,0 +1 @@
|
||||
1
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@ -0,0 +1,144 @@
|
||||
[33mMiniLB is only for debugging purposes, it only supports random policy![0m
|
||||
[MiniLB] Overriding policy to random
|
||||
INFO: Started server process [1]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:31000 (Press CTRL+C to quit)
|
||||
INFO: 174.1.60.1:59540 - "GET /health HTTP/1.1" 200 OK
|
||||
/opt/sglang/lib/python3.12/site-packages/sglang_router/mini_lb.py:396: FastAPIDeprecationWarning: ORJSONResponse is deprecated, FastAPI now serializes data directly to JSON bytes via Pydantic when a return type or response model is set, which is faster and doesn't need a custom response class. Read more in the FastAPI docs: https://fastapi.tiangolo.com/advanced/custom-response/#orjson-or-response-model and https://fastapi.tiangolo.com/tutorial/response-model/
|
||||
return await lb.generate(
|
||||
INFO: 174.1.60.1:59548 - "POST /generate HTTP/1.1" 200 OK
|
||||
/opt/sglang/lib/python3.12/site-packages/sglang_router/mini_lb.py:359: FastAPIDeprecationWarning: ORJSONResponse is deprecated, FastAPI now serializes data directly to JSON bytes via Pydantic when a return type or response model is set, which is faster and doesn't need a custom response class. Read more in the FastAPI docs: https://fastapi.tiangolo.com/advanced/custom-response/#orjson-or-response-model and https://fastapi.tiangolo.com/tutorial/response-model/
|
||||
return await _get_model_info_impl()
|
||||
INFO: 174.1.60.1:49872 - "GET /get_model_info HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:56534 - "GET /health HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:34528 - "GET /get_model_info HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:34534 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48724 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48734 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48750 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48764 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48780 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:33686 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:33696 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:50890 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:50902 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:50908 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:60410 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:60424 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:60428 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42190 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42196 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42204 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:40220 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:40236 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:40238 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:59726 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:59742 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:59748 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:44262 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:44274 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:44282 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:44296 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:38590 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:38596 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:38602 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:38618 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42080 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42094 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42102 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:53214 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:53226 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:53228 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:53230 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:46432 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:46434 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58776 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58778 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58790 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58792 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48536 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48548 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58506 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58516 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:58528 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48998 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49000 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49014 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49020 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48172 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48176 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48188 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:48190 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39632 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39646 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39654 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:35784 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:35798 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:35812 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:37916 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45072 - "GET /get_model_info HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45078 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45082 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45088 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45102 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45086 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45112 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45098 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:45114 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39922 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39952 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39946 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39934 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39962 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39978 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39986 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42528 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42542 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42568 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42556 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42596 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42606 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42612 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42590 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42618 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42578 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42978 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42968 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43004 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:42992 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43014 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43042 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43022 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43058 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43026 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43062 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43068 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43100 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43084 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49478 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43104 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49490 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49482 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:43120 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49502 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49514 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49524 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49530 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49558 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49542 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49574 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:49580 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39796 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39820 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39812 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39836 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39822 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39850 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39862 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39872 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39898 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39882 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39866 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39906 - "POST /generate HTTP/1.1" 200 OK
|
||||
INFO: 174.1.60.1:39908 - "POST /generate HTTP/1.1" 200 OK
|
||||
@ -0,0 +1 @@
|
||||
DATASET_OK 1319 3730d312f6e3440559ace48831e51066acaca737f6eabec99bccb9e4b3c39d14
|
||||
@ -0,0 +1,80 @@
|
||||
|
||||
174.1.60.1 2026-08-31T18:08:19+08:00
|
||||
0, 53361 MiB, 85651 MiB, 0 %
|
||||
1, 53361 MiB, 85651 MiB, 0 %
|
||||
2, 53361 MiB, 85651 MiB, 0 %
|
||||
3, 53361 MiB, 85651 MiB, 0 %
|
||||
4, 58677 MiB, 85651 MiB, 0 %
|
||||
5, 58669 MiB, 85651 MiB, 0 %
|
||||
6, 58677 MiB, 85651 MiB, 0 %
|
||||
7, 58669 MiB, 85651 MiB, 0 %
|
||||
|
||||
174.1.60.2 2026-08-31T18:08:20+08:00
|
||||
0, 58695 MiB, 85651 MiB, 0 %
|
||||
1, 58675 MiB, 85651 MiB, 0 %
|
||||
2, 58695 MiB, 85651 MiB, 0 %
|
||||
3, 58675 MiB, 85651 MiB, 0 %
|
||||
4, 63201 MiB, 85651 MiB, 0 %
|
||||
5, 63201 MiB, 85651 MiB, 0 %
|
||||
6, 63201 MiB, 85651 MiB, 0 %
|
||||
7, 63201 MiB, 85651 MiB, 0 %
|
||||
|
||||
174.1.60.3 2026-08-31T18:08:21+08:00
|
||||
0, 63367 MiB, 85651 MiB, 94 %
|
||||
1, 63367 MiB, 85651 MiB, 98 %
|
||||
2, 63367 MiB, 85651 MiB, 0 %
|
||||
3, 63367 MiB, 85651 MiB, 14 %
|
||||
4, 63489 MiB, 85651 MiB, 51 %
|
||||
5, 63469 MiB, 85651 MiB, 96 %
|
||||
6, 63469 MiB, 85651 MiB, 0 %
|
||||
7, 63469 MiB, 85651 MiB, 7 %
|
||||
|
||||
174.1.60.4 2026-08-31T18:08:22+08:00
|
||||
0, 63633 MiB, 85651 MiB, 0 %
|
||||
1, 63633 MiB, 85651 MiB, 0 %
|
||||
2, 63633 MiB, 85651 MiB, 0 %
|
||||
3, 63633 MiB, 85651 MiB, 0 %
|
||||
4, 76579 MiB, 85651 MiB, 0 %
|
||||
5, 76579 MiB, 85651 MiB, 0 %
|
||||
6, 76579 MiB, 85651 MiB, 0 %
|
||||
7, 76579 MiB, 85651 MiB, 0 %
|
||||
|
||||
174.1.60.5 2026-08-31T18:08:23+08:00
|
||||
0, 79931 MiB, 85651 MiB, 0 %
|
||||
1, 79933 MiB, 85651 MiB, 0 %
|
||||
2, 79917 MiB, 85651 MiB, 0 %
|
||||
3, 79897 MiB, 85651 MiB, 39 %
|
||||
4, 79889 MiB, 85651 MiB, 0 %
|
||||
5, 79933 MiB, 85651 MiB, 0 %
|
||||
6, 79917 MiB, 85651 MiB, 0 %
|
||||
7, 80019 MiB, 85651 MiB, 0 %
|
||||
|
||||
174.1.60.6 2026-08-31T18:08:25+08:00
|
||||
0, 79971 MiB, 85651 MiB, 98 %
|
||||
1, 79933 MiB, 85651 MiB, 100 %
|
||||
2, 79917 MiB, 85651 MiB, 100 %
|
||||
3, 79937 MiB, 85651 MiB, 100 %
|
||||
4, 79889 MiB, 85651 MiB, 100 %
|
||||
5, 79933 MiB, 85651 MiB, 100 %
|
||||
6, 79917 MiB, 85651 MiB, 92 %
|
||||
7, 80019 MiB, 85651 MiB, 96 %
|
||||
|
||||
174.1.60.7 2026-08-31T18:08:26+08:00
|
||||
0, 79971 MiB, 85651 MiB, 99 %
|
||||
1, 79933 MiB, 85651 MiB, 100 %
|
||||
2, 79917 MiB, 85651 MiB, 100 %
|
||||
3, 79937 MiB, 85651 MiB, 100 %
|
||||
4, 79889 MiB, 85651 MiB, 100 %
|
||||
5, 79933 MiB, 85651 MiB, 100 %
|
||||
6, 79917 MiB, 85651 MiB, 100 %
|
||||
7, 80019 MiB, 85651 MiB, 99 %
|
||||
|
||||
174.1.60.8 2026-08-31T18:08:27+08:00
|
||||
0, 79971 MiB, 85651 MiB, 0 %
|
||||
1, 79933 MiB, 85651 MiB, 0 %
|
||||
2, 79917 MiB, 85651 MiB, 0 %
|
||||
3, 79937 MiB, 85651 MiB, 0 %
|
||||
4, 79849 MiB, 85651 MiB, 0 %
|
||||
5, 79933 MiB, 85651 MiB, 0 %
|
||||
6, 79917 MiB, 85651 MiB, 0 %
|
||||
7, 79979 MiB, 85651 MiB, 0 %
|
||||
@ -0,0 +1,80 @@
|
||||
|
||||
NODE 174.1.60.1
|
||||
0, 3 MiB, 0 %
|
||||
1, 3 MiB, 0 %
|
||||
2, 3 MiB, 0 %
|
||||
3, 3 MiB, 0 %
|
||||
4, 3 MiB, 0 %
|
||||
5, 3 MiB, 0 %
|
||||
6, 3 MiB, 0 %
|
||||
7, 3 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.2
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.3
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.4
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.5
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.6
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.7
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
|
||||
NODE 174.1.60.8
|
||||
0, 0 MiB, 0 %
|
||||
1, 0 MiB, 0 %
|
||||
2, 0 MiB, 0 %
|
||||
3, 0 MiB, 0 %
|
||||
4, 0 MiB, 0 %
|
||||
5, 0 MiB, 0 %
|
||||
6, 0 MiB, 0 %
|
||||
7, 0 MiB, 0 %
|
||||
@ -0,0 +1,8 @@
|
||||
174.1.60.1 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.2 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.3 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.4 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.5 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.6 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.7 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
174.1.60.8 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
|
||||
@ -0,0 +1,246 @@
|
||||
{
|
||||
"run_id": "pd-dflash-kvbounds1-20260831-1740",
|
||||
"dataset_sha256": "3730d312f6e3440559ace48831e51066acaca737f6eabec99bccb9e4b3c39d14",
|
||||
"cases": {
|
||||
"1": {
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 1,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 6.943340536821033,
|
||||
"median_accept_length_per_question": 6.2727272727272725,
|
||||
"min_accept_length_per_question": 4.076923076923077,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 195.41776519827545,
|
||||
"output_throughput": 33.379769712245,
|
||||
"totals": {
|
||||
"output_tokens": 6523,
|
||||
"verify_steps": 1026,
|
||||
"accepted_drafts": 5562,
|
||||
"proposed_drafts": 15390,
|
||||
"prompt_tokens": 46392
|
||||
},
|
||||
"independent_answer_reparse_correct": 64,
|
||||
"correct_excluding_five_shot_overlap": 59,
|
||||
"questions_excluding_five_shot_overlap": 59,
|
||||
"answers_with_final_marker": 64,
|
||||
"api_accept_length_step_weighted": 6.3576998050682265,
|
||||
"histogram_accept_length_step_weighted": 6.421052631578948,
|
||||
"histogram_accept_length_equal_question": 7.03653796026238,
|
||||
"strict_draft_accept_rate": 0.36140350877192984,
|
||||
"accept_histogram": [
|
||||
9,
|
||||
82,
|
||||
139,
|
||||
168,
|
||||
101,
|
||||
116,
|
||||
77,
|
||||
75,
|
||||
64,
|
||||
56,
|
||||
38,
|
||||
24,
|
||||
22,
|
||||
7,
|
||||
8,
|
||||
40
|
||||
],
|
||||
"prompt_tokens_min": 694,
|
||||
"prompt_tokens_max": 794,
|
||||
"outputs_max": 225,
|
||||
"finish_reasons": {
|
||||
"stop": 64
|
||||
},
|
||||
"cached_tokens_total": 0,
|
||||
"retractions_total": 0
|
||||
},
|
||||
"8": {
|
||||
"questions": 64,
|
||||
"successful_requests": 64,
|
||||
"failed_requests": 0,
|
||||
"failed_prompt_ids": [],
|
||||
"num_shots": 5,
|
||||
"max_new_tokens": 512,
|
||||
"parallel": 8,
|
||||
"temperature": 0.0,
|
||||
"top_p": 1.0,
|
||||
"prompt_format": "sglang_official_raw_five_shot",
|
||||
"speculative_algorithm": "DFLASH",
|
||||
"length_limited_requests": 0,
|
||||
"accuracy_all_questions": 1.0,
|
||||
"accuracy_successful_requests": 1.0,
|
||||
"mean_accept_length_equal_weight_per_question": 7.007018111850178,
|
||||
"median_accept_length_per_question": 6.35064935064935,
|
||||
"min_accept_length_per_question": 4.416666666666667,
|
||||
"max_accept_length_per_question": 16.333333333333332,
|
||||
"duration_s": 55.545577608048916,
|
||||
"output_throughput": 117.56111433529787,
|
||||
"totals": {
|
||||
"output_tokens": 6530,
|
||||
"verify_steps": 1012,
|
||||
"accepted_drafts": 5582,
|
||||
"proposed_drafts": 15180,
|
||||
"prompt_tokens": 46392
|
||||
},
|
||||
"independent_answer_reparse_correct": 64,
|
||||
"correct_excluding_five_shot_overlap": 59,
|
||||
"questions_excluding_five_shot_overlap": 59,
|
||||
"answers_with_final_marker": 64,
|
||||
"api_accept_length_step_weighted": 6.452569169960475,
|
||||
"histogram_accept_length_step_weighted": 6.515810276679842,
|
||||
"histogram_accept_length_equal_question": 7.102034009014687,
|
||||
"strict_draft_accept_rate": 0.36772068511198946,
|
||||
"accept_histogram": [
|
||||
10,
|
||||
67,
|
||||
140,
|
||||
169,
|
||||
93,
|
||||
117,
|
||||
76,
|
||||
72,
|
||||
67,
|
||||
60,
|
||||
42,
|
||||
22,
|
||||
20,
|
||||
8,
|
||||
6,
|
||||
43
|
||||
],
|
||||
"prompt_tokens_min": 694,
|
||||
"prompt_tokens_max": 794,
|
||||
"outputs_max": 225,
|
||||
"finish_reasons": {
|
||||
"stop": 64
|
||||
},
|
||||
"cached_tokens_total": 0,
|
||||
"retractions_total": 0
|
||||
}
|
||||
},
|
||||
"log_scan": {
|
||||
"d_0.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 69,
|
||||
"rdma_ready_ack": 946,
|
||||
"graph_decode_batches": 34
|
||||
},
|
||||
"d_1.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 0,
|
||||
"rdma_ready_ack": 976,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"d_2.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 0,
|
||||
"rdma_ready_ack": 964,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"d_3.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 0,
|
||||
"rdma_ready_ack": 952,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"p_0.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 17,
|
||||
"rdma_ready_ack": 0,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"p_1.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 17,
|
||||
"rdma_ready_ack": 0,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"p_2.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 18,
|
||||
"rdma_ready_ack": 0,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"p_3.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 80,
|
||||
"kda_fused_decode_fallback": 17,
|
||||
"rdma_ready_ack": 0,
|
||||
"graph_decode_batches": 0
|
||||
},
|
||||
"router.log": {
|
||||
"fatal_signature": 0,
|
||||
"numa_lookup_warning": 0,
|
||||
"kda_fused_decode_fallback": 0,
|
||||
"rdma_ready_ack": 0,
|
||||
"graph_decode_batches": 0
|
||||
}
|
||||
},
|
||||
"metric_definitions": {
|
||||
"api_accept_length": "completion_tokens / spec_verify_ct, averaged equally over requests",
|
||||
"histogram_accept_length_weighted": "1 + total accepted draft tokens / total verify steps",
|
||||
"output_tps": "total completion_tokens / client measured duration_s"
|
||||
},
|
||||
"scope": "fixed 64 GSM8K deployment regression; five-shot examples overlap first 5 questions; not independent held-out accuracy",
|
||||
"exit_codes": {
|
||||
"exit_code": 1,
|
||||
"bench_resume_exit_code": 0,
|
||||
"cleanup_exit_code": 0
|
||||
},
|
||||
"evidence_sha256": {
|
||||
"bench/gsm8k_c1.dataset_path_failure.log": "84d949fb574407d533b7ffbd9350f417f837af9790492db00e93fb5b0129e29d",
|
||||
"bench/gsm8k_c1.jsonl": "0929e5a8f07aa28955d7a59a9e1685c8d71d81c92ba43d4a61f189e6d4d57b34",
|
||||
"bench/gsm8k_c1.log": "cc8488e67e2950e02dca977e195977beff21d092fd9f9ca06d240d194b97fa28",
|
||||
"bench/gsm8k_c1_summary.json": "1831996f6a0c7953118265db03f947720b4072edb963e430fff915258284ccc6",
|
||||
"bench/gsm8k_c8.jsonl": "11e7be10c8cbcf0dde37d831a434e6d7581fd0574d728b361895396d772de6ee",
|
||||
"bench/gsm8k_c8.log": "3edfd639473b8009825a0aa19c77b57567af91292a4b106235eb28323c7d9ede",
|
||||
"bench/gsm8k_c8_summary.json": "2853ecbab21fe9ca0735b54ad470d1038d4681fa954108d68094e18ad68ce109",
|
||||
"bench/smoke.json": "c32b0f6df0c20565748fb09c579bd80eac9c668800968e3bede315d6a6f51b1b",
|
||||
"bench_resume_exit_code": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa",
|
||||
"cleanup_exit_code": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa",
|
||||
"commands/d_0.cmd.txt": "6f651da18473b3865a83e3bbb29ffc543c4a96ff34488c08fb75f0c84e18fc3e",
|
||||
"commands/d_1.cmd.txt": "d280afb5a90ba751a1e03277d6b31dcfedcff405397ad176513dad43b31663d7",
|
||||
"commands/d_2.cmd.txt": "2663f478cd91344d5ff5ae3bd51a49ab9ea71fc70a56b6d07859f2e95ab717d8",
|
||||
"commands/d_3.cmd.txt": "2bb81aa835ba5c8a18be7eb77811aa4487da0713a7990c913ce025d49e05cd85",
|
||||
"commands/gsm8k_c1.cmd.txt": "fe4569485e104ae13035064941934d65e9a705d8b260637627b97db8e6e22e30",
|
||||
"commands/gsm8k_c1.dataset_path_failure.cmd.txt": "0c1aa55b8660afa372cb3290c2377fd7920e374d765bc4d663d3f4626aa8ff8c",
|
||||
"commands/gsm8k_c8.cmd.txt": "7aa31e99060d18301bb51f9adfa1d62c11f5993caaf1681e9b13a15583de6d8c",
|
||||
"commands/p_0.cmd.txt": "1147c8d0d1de1925e9f48d38251371c071c92da67e768ac6519b916b803eeb6e",
|
||||
"commands/p_1.cmd.txt": "261da7b41f0f8c865ee4bdee89710ebf903a4578b6c756a014229e7a7bec8ee4",
|
||||
"commands/p_2.cmd.txt": "9cf2e4ed7bc337e0d42bd90f7ed8fc03ef3e020a85c4c75bbafc29257ac6ea6a",
|
||||
"commands/p_3.cmd.txt": "2313aaaadcb1c80652f33a7e0246ace1ab622d4d7e9ecd87b3603247acbdb784",
|
||||
"commands/router.cmd.txt": "8af366cace8b33785bee4378a9e5a75636858aee249bc16e2dc4f345fab4cfb3",
|
||||
"exit_code": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865",
|
||||
"logs/d_0.log": "c846da72e60bcdd4bbf802af7ec7317cb3f5f7c9a4fbff9c7f1584d47e190733",
|
||||
"logs/d_1.log": "9ec54b0d03d2484c810f72be13efe1e7242a1b077e904ffbfba80f14740d24be",
|
||||
"logs/d_2.log": "2b71d2b6267914d0caacf82dea1be316030170e7e3f12ecc1484d1d790fea6cb",
|
||||
"logs/d_3.log": "5dff02037677cdb2f9b9271591e8085e532073224e6b3e9fde8bc893f669a714",
|
||||
"logs/p_0.log": "30f94f283fa515e7f2929ae32a24134dbb5897ef542a34e7c8c19e170f677f86",
|
||||
"logs/p_1.log": "bb82b6e4428bc76a1c0e93bd4af6e72b9f22a7647194f4fb807ae08530a7229c",
|
||||
"logs/p_2.log": "2883931b8784036941f5c05035f4430fd4a722205db82017bcb3fe23dc43adc3",
|
||||
"logs/p_3.log": "be0548b79556d890fd93d161a97daa67a37cd185413bb107c4f38b6fe5dbe890",
|
||||
"logs/router.log": "aabfbd56eac2c7d401c707124dcddbb6edc2dae65a1ce9311955d53aacf0e9ce",
|
||||
"metadata/dataset_mount_validation.txt": "8e33ed2e323b5d1980bae1ad8486ef2a90229f50c5b655a9b96b5cbe489148a3",
|
||||
"metadata/gpu_during_gsm8k.txt": "5faa28fe4ccd6416f36394853d384f3369fd731c837b836a2e25e5845d9899bd",
|
||||
"metadata/post_run_resources.txt": "de0d555e14e669190577a076e9288e46e718475b24a3f186832423106becd685",
|
||||
"metadata/preflight.txt": "eb603bf2006e1cc2f84e5c6d1d6d90a8936b7ea38cdae8c60a1c5170f618574a"
|
||||
}
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user