rewrite README, add new platform onboarding guide, fix broken scripts/common paths
- rewrite README with project purpose, standard workflow, corrected index - add docs/NEW_PLATFORM_GUIDE.md (new GPU onboarding SOP, GLM5.2 reuse) - fix ../../scripts/common -> ../../../scripts/common in 42 experiment scripts - refresh stale docs (EXPERIMENT_GUIDE, H200_QUICKSTART, ADAPTIVE_CONCURRENCY_USAGE, BENCHMARK_WORKFLOW) - remove dead code (dp_proxy.py) and .bak leftovers - add p800 adaptive results (tp4_dp2/tp8_dp1 metrics + summary) - gitignore envs/charts and .tmp_charts
This commit is contained in:
yy-fighting
parent
90f0257ae0
commit
8652a685e6
4
.gitignore
vendored
4
.gitignore
vendored
@ -5,6 +5,10 @@ envs/modelscope-upload/
|
|||||||
envs/sglang/
|
envs/sglang/
|
||||||
envs/sglang-src/
|
envs/sglang-src/
|
||||||
envs/vllm/
|
envs/vllm/
|
||||||
|
envs/charts/
|
||||||
|
# 其他一次性工具环境/临时产物
|
||||||
|
.venv-charts/
|
||||||
|
.tmp_charts/
|
||||||
# 其他通用忽略
|
# 其他通用忽略
|
||||||
deps/
|
deps/
|
||||||
tmp/
|
tmp/
|
||||||
|
|||||||
@ -3,16 +3,20 @@
|
|||||||
## Directory Layout
|
## Directory Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
/data/user1/yy/
|
<repo-root>/ # 本仓库根目录（如 /data1/yy/sskj，按机器调整）
|
||||||
├── platforms/ # chip/accelerator platform configs
|
├── platforms/ # chip/accelerator platform configs
|
||||||
│ ├── kunlun_p800.env
|
│ ├── kunlun_p800.env
|
||||||
|
│ ├── nvidia_h20.env
|
||||||
│ ├── nvidia_h200.env
|
│ ├── nvidia_h200.env
|
||||||
|
│ ├── nvidia_rtx6000d.env
|
||||||
│ └── patches/kunlun_p800/ # runtime patches required by some images
|
│ └── patches/kunlun_p800/ # runtime patches required by some images
|
||||||
├── scripts/ # shared benchmark/orchestrator/utility scripts
|
├── scripts/
|
||||||
│ ├── common/ # reusable components (lib.sh, platform.sh, parse_backend.py, ...)
|
│ └── common/ # reusable components (lib.sh, platform.sh, parse_backend.py, adaptive_bench_lib.sh, ...)
|
||||||
│ ├── benchmark_dspark_0707/ # legacy DSpark benchmark suite (仍被 experiments/dsv4_h200_dspark/ 引用)
|
├── docs/
|
||||||
│ └── SLO_STANDARDS.md # 推理服务 SLO 标准
|
│ └── SLO_STANDARDS.md # 推理服务 SLO 标准
|
||||||
├── experiments/ # experiment-centric directories (preferred)
|
├── experiments/ # experiment-centric directories (preferred): experiments/<platform>/<experiment>/
|
||||||
|
│ ├── TEMPLATE/ # template for fixed-scenario sglang-vs-vllm experiments
|
||||||
|
│ └── <platform>/ # h20 / h200 / p800 / pro6000
|
||||||
│ └── dsv4_p800_sglang/
|
│ └── dsv4_p800_sglang/
|
||||||
│ ├── README.md
|
│ ├── README.md
|
||||||
│ ├── config.env # experiment-level configuration
|
│ ├── config.env # experiment-level configuration
|
||||||
@ -24,28 +28,26 @@
|
|||||||
│ ├── report.md
|
│ ├── report.md
|
||||||
│ ├── results.json
|
│ ├── results.json
|
||||||
│ └── logs/
|
│ └── logs/
|
||||||
├── bench_results/ # legacy benchmark outputs (read-only history)
|
|
||||||
│ ├── dspark_grid_20260707-132641/
|
|
||||||
│ └── ...
|
|
||||||
├── logs/ # server logs (stdout/stderr from start scripts)
|
|
||||||
├── datasets/ # benchmark datasets
|
├── datasets/ # benchmark datasets
|
||||||
└── envs/ # Python virtual environments
|
└── envs/ # Python virtual environments
|
||||||
```
|
```
|
||||||
|
|
||||||
|
注：legacy 的 `scripts/benchmark_dspark_0707/` 已删除；顶层 `bench_results/` 与 `logs/` 已不在仓库中（历史结果已归档迁移至仓库外）。
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
1. **Experiments are the primary organization unit.**
|
1. **Experiments are the primary organization unit.**
|
||||||
- Each experiment lives under `experiments/<experiment_name>/` and contains its scripts, configuration, and results.
|
- Each experiment lives under `experiments/<platform>/<experiment_name>/` (platform ∈ {h20, h200, p800, pro6000}) and contains its scripts, configuration, and results.
|
||||||
- Shared orchestration code lives in `scripts/common/`; do not copy server start / health check logic into every experiment.
|
- Shared orchestration code lives in `scripts/common/`; do not copy server start / health check logic into every experiment.
|
||||||
- 新实验必须放在 `experiments/<name>/` 下；`scripts/` 下仅保留 `scripts/common/` 和被现有实验引用的 legacy 脚本。
|
- 新实验必须放在 `experiments/<platform>/<name>/` 下；`scripts/` 下仅保留 `scripts/common/`（原 legacy 套件 `scripts/benchmark_dspark_0707/` 已删除）。
|
||||||
|
|
||||||
2. **Benchmark outputs live with their experiment.**
|
2. **Benchmark outputs live with their experiment.**
|
||||||
- For `experiments/<name>/`, results go in `experiments/<name>/results/<RUN_ID>/`.
|
- For `experiments/<platform>/<name>/`, results go in `experiments/<platform>/<name>/results/<RUN_ID>/`.
|
||||||
- Each run directory must contain `report.md` (human-readable) and `results.json` (structured data).
|
- Each run directory must contain `report.md` (human-readable) and `results.json` (structured data).
|
||||||
- Raw outputs go in `raw_outputs/`; logs go in `logs/`.
|
- Raw outputs go in `raw_outputs/`; logs go in `logs/`.
|
||||||
- Legacy `bench_results/<experiment>_<timestamp>/` directories remain valid for archived runs.
|
- Legacy `bench_results/<experiment>_<timestamp>/` 目录已从仓库移除，历史结果已归档迁移至仓库外。
|
||||||
|
|
||||||
3. **Each `bench_results/<run>/` directory must contain two final artifacts.**
|
3. **Each experiment run directory (`results/<run>/`) must contain two final artifacts.**
|
||||||
- A Markdown report for human reading (e.g. `report.md`, `comparison_report.md`).
|
- A Markdown report for human reading (e.g. `report.md`, `comparison_report.md`).
|
||||||
- A JSON file with the complete structured result data for programmatic analysis (e.g. `results.json`).
|
- A JSON file with the complete structured result data for programmatic analysis (e.g. `results.json`).
|
||||||
- The directory may also contain a `README.md` documenting provenance if the report alone does not cover it.
|
- The directory may also contain a `README.md` documenting provenance if the report alone does not cover it.
|
||||||
@ -59,8 +61,7 @@
|
|||||||
- See [Final JSON Schema](#final-json-schema) below for the recommended structure.
|
- See [Final JSON Schema](#final-json-schema) below for the recommended structure.
|
||||||
|
|
||||||
5. **Scripts should default `RESULT_ROOT` to the experiment's results directory.**
|
5. **Scripts should default `RESULT_ROOT` to the experiment's results directory.**
|
||||||
- For `experiments/<name>/run_bench.sh`, default to `experiments/<name>/results/${RUN_ID}/`.
|
- For `experiments/<platform>/<name>/run_bench.sh`, default to `experiments/<platform>/<name>/results/${RUN_ID}/`.
|
||||||
- For legacy scripts, default to `bench_results/<experiment>_${RUN_ID}/`.
|
|
||||||
- Allow override via `RESULT_ROOT` env var.
|
- Allow override via `RESULT_ROOT` env var.
|
||||||
- Use `RUN_ID=$(date '+%Y%m%d-%H%M%S')` unless specified.
|
- Use `RUN_ID=$(date '+%Y%m%d-%H%M%S')` unless specified.
|
||||||
|
|
||||||
@ -96,30 +97,23 @@
|
|||||||
For experiment-centric layout:
|
For experiment-centric layout:
|
||||||
|
|
||||||
```
|
```
|
||||||
experiments/<experiment>/results/<YYYYMMDD-HHMMSS>/
|
experiments/<platform>/<experiment>/results/<YYYYMMDD-HHMMSS>/
|
||||||
```
|
```
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
|
|
||||||
- `experiments/dsv4_p800_sglang/results/20260708-120000/`
|
- `experiments/p800/dsv4_p800_sglang/results/20260708-120000/`
|
||||||
- `experiments/dspark_grid/results/20260707-132641/`
|
- `experiments/h200/dsv4_h200_sglang_vs_vllm/results/20260707-132641/`
|
||||||
|
|
||||||
The `results.json` metadata already records `chip`/`accelerator` and `engine`, so the directory path does not need to encode them. If a single experiment must distinguish across platforms in its directory tree, use:
|
The `results.json` metadata already records `chip`/`accelerator` and `engine`, so the directory path does not need to encode them. If a single experiment must distinguish across platforms in its directory tree, use:
|
||||||
|
|
||||||
```
|
```
|
||||||
experiments/<experiment>/results/<chip>_<engine>_<YYYYMMDD-HHMMSS>/
|
experiments/<platform>/<experiment>/results/<chip>_<engine>_<YYYYMMDD-HHMMSS>/
|
||||||
```
|
```
|
||||||
|
|
||||||
### Legacy result directories
|
### Legacy result directories (removed)
|
||||||
|
|
||||||
```
|
`bench_results/<experiment>_<YYYYMMDD-HHMMSS>/` 与 `experiments/legacy_bench_results/` 均已从仓库移除，历史结果已归档迁移至仓库外；保留此节仅说明历史命名格式。
|
||||||
bench_results/<experiment>_<YYYYMMDD-HHMMSS>/
|
|
||||||
```
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
|
|
||||||
- `experiments/legacy_bench_results/dspark_grid_20260707-132641/` (legacy result moved under `experiments/`)
|
|
||||||
- `experiments/legacy_bench_results/dsv4_backend_comparison_20260707/` (legacy result moved under `experiments/`)
|
|
||||||
|
|
||||||
### Raw output files
|
### Raw output files
|
||||||
|
|
||||||
@ -146,14 +140,14 @@ logs/<experiment>_orchestrator_YYYYMMDD_HHMMSS.log
|
|||||||
|
|
||||||
## Final JSON Schema
|
## Final JSON Schema
|
||||||
|
|
||||||
The JSON file inside each `bench_results/<run>/` directory should follow a stable schema so that downstream Python scripts can load every experiment the same way. The file is usually named `results.json`.
|
The JSON file inside each experiment run directory (`experiments/<platform>/<experiment>/results/<run>/`) should follow a stable schema so that downstream Python scripts can load every experiment the same way. The file is usually named `results.json`.
|
||||||
|
|
||||||
### Required top-level fields
|
### Required top-level fields
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"metadata": {
|
"metadata": {
|
||||||
"experiment": "dspark_grid",
|
"experiment": "dsv4_h200_dspark",
|
||||||
"run_id": "20260707-132641",
|
"run_id": "20260707-132641",
|
||||||
"timestamp": "2026-07-07T13:26:41+08:00",
|
"timestamp": "2026-07-07T13:26:41+08:00",
|
||||||
"model": "/data/models/DeepSeek-V4-Flash-DSpark",
|
"model": "/data/models/DeepSeek-V4-Flash-DSpark",
|
||||||
@ -162,8 +156,8 @@ The JSON file inside each `bench_results/<run>/` directory should follow a stabl
|
|||||||
"hardware": "8x NVIDIA H200 143GB",
|
"hardware": "8x NVIDIA H200 143GB",
|
||||||
"accelerator": "NVIDIA H200",
|
"accelerator": "NVIDIA H200",
|
||||||
"chip": "NVIDIA H200",
|
"chip": "NVIDIA H200",
|
||||||
"script": "scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh",
|
"script": "experiments/h200/dsv4_h200_dspark/run_bench.sh",
|
||||||
"env": "/data/user1/yy/envs/vllm-dspark",
|
"env": "/path/to/envs/vllm-dspark",
|
||||||
"git_commit": "optional git sha",
|
"git_commit": "optional git sha",
|
||||||
"description": "optional free-text note"
|
"description": "optional free-text note"
|
||||||
},
|
},
|
||||||
@ -241,37 +235,27 @@ The JSON file inside each `bench_results/<run>/` directory should follow a stabl
|
|||||||
### Run P800 SGLang benchmark (Kunlun P800, Docker)
|
### Run P800 SGLang benchmark (Kunlun P800, Docker)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash experiments/dsv4_p800_sglang/run_bench.sh
|
bash experiments/p800/dsv4_p800_sglang/run_bench.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run H200 DSpark benchmark (NVIDIA H200, native venv)
|
### Run H200 DSpark benchmark (NVIDIA H200, native venv)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash experiments/dsv4_h200_dspark/run_bench.sh
|
bash experiments/h200/dsv4_h200_dspark/run_bench.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run legacy DSpark grid benchmark
|
Note: the wrapper's default `LEGACY_GRID_SCRIPT` / `SERVER_START_SCRIPT` point at the removed legacy scripts; override them before running (see `docs/H200_QUICKSTART.md`).
|
||||||
|
|
||||||
```bash
|
### Legacy DSpark scripts (removed)
|
||||||
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run DSpark spec-tokens comparison
|
The legacy suite `scripts/benchmark_dspark_0707/` (grid benchmark, spec-tokens comparison, etc.) has been removed from this repo.
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Parse results
|
### Parse results
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Experiment-centric layout
|
# Experiment-centric layout
|
||||||
python3 experiments/dsv4_p800_sglang/parse_results.py \
|
python3 experiments/p800/dsv4_p800_sglang/parse_results.py \
|
||||||
experiments/dsv4_p800_sglang/results/<run_id>
|
experiments/p800/dsv4_p800_sglang/results/<run_id>
|
||||||
|
|
||||||
# Legacy layout
|
|
||||||
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
|
|
||||||
/data/user1/yy/experiments/legacy_bench_results/dspark_grid_<run_id>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Adding a New Platform or Experiment
|
## Adding a New Platform or Experiment
|
||||||
@ -297,7 +281,7 @@ For host-native platforms, set the relevant venv paths (see
|
|||||||
### 2. Create an experiment directory
|
### 2. Create an experiment directory
|
||||||
|
|
||||||
```
|
```
|
||||||
experiments/<name>/
|
experiments/<platform>/<name>/
|
||||||
├── README.md # Purpose and usage
|
├── README.md # Purpose and usage
|
||||||
├── config.env # Model, port, scenarios, engine overrides
|
├── config.env # Model, port, scenarios, engine overrides
|
||||||
├── start_server.sh # (optional) platform-specific server launch
|
├── start_server.sh # (optional) platform-specific server launch
|
||||||
@ -309,7 +293,7 @@ At minimum, `run_bench.sh` should:
|
|||||||
|
|
||||||
1. Source `scripts/common/lib.sh` and `scripts/common/platform.sh`.
|
1. Source `scripts/common/lib.sh` and `scripts/common/platform.sh`.
|
||||||
2. Read `config.env`.
|
2. Read `config.env`.
|
||||||
3. Create `experiments/<name>/results/<RUN_ID>/`.
|
3. Create `experiments/<platform>/<name>/results/<RUN_ID>/`.
|
||||||
4. Call `write_metadata_json` to create `results.json`.
|
4. Call `write_metadata_json` to create `results.json`.
|
||||||
5. Record the exact server launch command/args in `results.json` `config.server_args` (or `phaseN_server_args`).
|
5. Record the exact server launch command/args in `results.json` `config.server_args` (or `phaseN_server_args`).
|
||||||
6. Run the benchmark scenarios.
|
6. Run the benchmark scenarios.
|
||||||
@ -327,16 +311,15 @@ Docker/XPU 平台可以在 `scripts/common/` 下新增自己的 helper，但不
|
|||||||
|
|
||||||
### 4. Example experiments
|
### 4. Example experiments
|
||||||
|
|
||||||
- Docker / XPU: `experiments/dsv4_p800_sglang/`
|
- Docker / XPU: `experiments/p800/dsv4_p800_sglang/`
|
||||||
- Native / H200: `experiments/dsv4_h200_dspark/`
|
- Native / H200: `experiments/h200/dsv4_h200_dspark/`
|
||||||
|
|
||||||
See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough.
|
See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough.
|
||||||
|
|
||||||
## Checklist Before Committing / Archiving
|
## Checklist Before Committing / Archiving
|
||||||
|
|
||||||
- [ ] No `.jsonl`, `.json`, `.log`, or `.md` files left in the project root.
|
- [ ] No `.jsonl`, `.json`, `.log`, or `.md` files left in the project root.
|
||||||
- [ ] For `experiments/<name>/`, outputs live in `experiments/<name>/results/<timestamp>/`.
|
- [ ] For `experiments/<platform>/<name>/`, outputs live in `experiments/<platform>/<name>/results/<timestamp>/`.
|
||||||
- [ ] For legacy runs, outputs live in `bench_results/<experiment>_<timestamp>/`.
|
|
||||||
- [ ] `<run>/report.md` (or equivalent human-readable `.md`) exists.
|
- [ ] `<run>/report.md` (or equivalent human-readable `.md`) exists.
|
||||||
- [ ] `<run>/results.json` exists and follows the [Final JSON Schema](#final-json-schema).
|
- [ ] `<run>/results.json` exists and follows the [Final JSON Schema](#final-json-schema).
|
||||||
- [ ] `<run>/results.json` metadata records the `chip`/`accelerator` and `engine`/`backend` used.
|
- [ ] `<run>/results.json` metadata records the `chip`/`accelerator` and `engine`/`backend` used.
|
||||||
@ -344,6 +327,6 @@ See `docs/H200_QUICKSTART.md` for a concrete H200 porting walkthrough.
|
|||||||
- [ ] `<run>/results.json` each scenario records `slo_status` (overall pass/fail/partial against the relevant SLO).
|
- [ ] `<run>/results.json` each scenario records `slo_status` (overall pass/fail/partial against the relevant SLO).
|
||||||
- [ ] Raw output filenames include the chip/accelerator and engine when cross-platform runs may collide.
|
- [ ] Raw output filenames include the chip/accelerator and engine when cross-platform runs may collide.
|
||||||
- [ ] `<run>/README.md` exists and documents provenance (or the report itself covers provenance).
|
- [ ] `<run>/README.md` exists and documents provenance (or the report itself covers provenance).
|
||||||
- [ ] Scripts either live under `experiments/<name>/` or in `scripts/` (or `scripts/<group>/`).
|
- [ ] Scripts either live under `experiments/<platform>/<name>/` or in `scripts/common/`.
|
||||||
- [ ] Script path references updated after moving.
|
- [ ] Script path references updated after moving.
|
||||||
- [ ] Commits include the experiment code plus final `results.json` and `.md` reports; logs/raw outputs are left ignored.
|
- [ ] Commits include the experiment code plus final `results.json` and `.md` reports; logs/raw outputs are left ignored.
|
||||||
|
|||||||
266
README.md
266
README.md
@ -1,190 +1,124 @@
|
|||||||
# DSV4 / DeepSeek-V4-Flash 推理测速项目
|
# sskj — 多平台大模型推理性能基准测试项目
|
||||||
|
|
||||||
> 记录 vllm-dspark、SGLang、vLLM 等后端在 DeepSeek-V4-Flash 上的 benchmark 脚本与最终结果。
|
> **项目目的**：当新显卡（GPU/NPU）到货时，用最短时间完成大模型在该平台上的推理性能评估与部署配置选型。
|
||||||
> H200 快速上手指南见 [`docs/H200_QUICKSTART.md`](docs/H200_QUICKSTART.md)。
|
> 当前模型：DeepSeek-V4-Flash（FP8 / INT8）；后续接入 GLM5.2，**完全复用**本项目的实验与报告流程。
|
||||||
|
> 新平台接入 SOP：[`docs/NEW_PLATFORM_GUIDE.md`](docs/NEW_PLATFORM_GUIDE.md)。
|
||||||
|
|
||||||
## 目录说明
|
## 这个项目解决什么问题
|
||||||
|
|
||||||
| 目录/文件 | 说明 |
|
1. **新卡快速评估**：新平台到货后，按 SOP 复制实验目录、改少量配置，即可跑起与既有平台口径一致的 benchmark，横向可比。
|
||||||
|
2. **部署配置选型**：通过 TP×DP 并行矩阵 + 自适应并发搜索，回答"该平台上哪种并行配置在什么并发下满足 SLO、吞吐最高"。
|
||||||
|
3. **数据资产化**：每轮实验产出结构化结果（jsonl/csv/manifest），汇总成飞书性能报告并回填多维表格，形成可检索的历史数据库。
|
||||||
|
|
||||||
|
## 标准工作流（全流程）
|
||||||
|
|
||||||
|
```
|
||||||
|
① 跑实验 experiments/<platform>/<experiment>/run_adaptive_concurrency*.sh
|
||||||
|
└─ 公共搜索逻辑: scripts/common/adaptive_bench_lib.sh
|
||||||
|
② 落盘 adaptive_results/<run_id>/
|
||||||
|
└─ adaptive_points.jsonl / adaptive_shapes.jsonl / adaptive_summary.* / run_manifest.json
|
||||||
|
③ 分析 用 Python 从 jsonl 实算（禁止目测），SLO 口径见 docs/SLO_STANDARDS.md
|
||||||
|
④ 飞书报告 按飞书 wiki「性能报告编写指南（README）」写到「显卡性能报告/<平台>」节点下
|
||||||
|
⑤ 回填多维表格 飞书「模型推理适配 Bench 迭代跟踪」：每成功探测点一行
|
||||||
|
```
|
||||||
|
|
||||||
|
- SLO 口径（S2 层，DSV4-Flash / GLM5.2 均适用）：**TTFT P95 < 3s，TPOT P95 < 50ms**，严格小于。详见 [`docs/SLO_STANDARDS.md`](docs/SLO_STANDARDS.md)。
|
||||||
|
- 报告编写规范（章节骨架、分析口径、发布流程）以飞书 wiki「显卡性能报告 / 性能报告编写指南（README）」为准。
|
||||||
|
- 多维表格每行 = 一个成功探测点（config、Concurrency、Throughput、Input/Output TPS、TTFT/TPOT/E2E P95、SLO达标状态、开发文档链接）。
|
||||||
|
|
||||||
|
## 目录结构
|
||||||
|
|
||||||
|
```
|
||||||
|
platforms/ # 平台配置（每平台一个 .env）+ 平台补丁（patches/）
|
||||||
|
scripts/common/ # 跨实验复用的编排组件（见下）
|
||||||
|
experiments/ # 实验目录（三层结构）
|
||||||
|
├── TEMPLATE/ # 老式固定场景对比实验模板（sglang vs vllm）
|
||||||
|
├── h20/ # NVIDIA H20 96GB
|
||||||
|
├── h200/ # NVIDIA H200 143GB
|
||||||
|
├── p800/ # Kunlun P800 XPU
|
||||||
|
└── pro6000/ # NVIDIA RTX 6000D
|
||||||
|
docs/ # 规范与指南（SLO、实验规范、新卡接入）
|
||||||
|
envs/ # 环境搭建文档（具体 venv 不提交，见 .gitignore）
|
||||||
|
datasets/ # benchmark 数据集（不提交）
|
||||||
|
BENCHMARK_WORKFLOW.md # 结果目录与命名规范、results.json schema
|
||||||
|
```
|
||||||
|
|
||||||
|
### scripts/common/ 组件
|
||||||
|
|
||||||
|
| 文件 | 职责 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `platforms/` | 芯片/加速器平台配置（`*.env`），如 P800、H200、RTX 6000D |
|
| `lib.sh` | 日志、health check、metadata JSON 生成 |
|
||||||
| `scripts/common/` | 跨实验复用的 orchestration 组件（server 启停、health check、元数据生成、结果解析） |
|
| `platform.sh` | 平台自动探测并加载 `platforms/<chip>.env` |
|
||||||
| `scripts/` | 仅保留仍在使用的 legacy DSpark 脚本与 SLO 标准 |
|
| `adaptive_bench_lib.sh` | **自适应并发搜索核心库**（引擎无关，靠 5 个 `engine_*` 回调注入平台逻辑） |
|
||||||
| `experiments/` | 以实验为单位的自包含目录（脚本 + 配置 + 结果） |
|
| `adaptive_concurrency.py` | 上者的 Python 助手（shapes 生成、结果解析、汇总） |
|
||||||
| `experiments/legacy_bench_results/` | 历史归档的实验产物（只读），从原 `bench_results/` 迁移而来 |
|
| `parse_backend.py` | 固定并发实验：raw jsonl → results.json + report.md |
|
||||||
| `BENCHMARK_WORKFLOW.md` | benchmark 目录与命名规范 |
|
| `compare.py` | SGLang vs vLLM 横向对比表（老式实验用） |
|
||||||
| `docs/SLO_STANDARDS.md` | 推理服务 SLO 标准（TTFT/TPOT） |
|
| `warmup.py` | 服务预热 |
|
||||||
|
| `server_docker.sh` / `bench_client_docker.sh` | P800 专用 Docker 服务端/客户端生命周期 |
|
||||||
> 注：`.gitignore` 已排除 `envs/`、`deps/`、`tmp/`、`datasets/`、`logs/`、`*.zip`、原始 `raw_outputs/*.jsonl` 等大文件/中间文件。
|
| `adaptive_heartbeat.sh` | 长跑实验的心跳监控（手动独立运行） |
|
||||||
|
|
||||||
## 实验索引
|
## 实验索引
|
||||||
|
|
||||||
### 新结构（experiments/）
|
### TP/DP matrix + 自适应并发（当前主流形态）
|
||||||
|
|
||||||
| 实验 | 脚本 | 说明 |
|
| 实验 | 说明 |
|
||||||
|---|---|---|
|
|---|---|
|
||||||
| DSV4 P800 SGLang | `experiments/dsv4_p800_sglang/run_bench.sh` | Kunlun P800 + SGLang + DeepSeek-V4-Flash-INT8 |
|
| `experiments/h20/dsv4_h20_vllm_tp_dp_matrix/` | H20 + vLLM，TP2/DP4、TP4/DP2、TP8/DP1 |
|
||||||
| DSV4 H200 DSpark | `experiments/dsv4_h200_dspark/run_bench.sh` | NVIDIA H200 + vllm-dspark + DeepSeek-V4-Flash-DSpark |
|
| `experiments/h20/dsv4_h20_sglang_tp_dp_matrix/` | H20 + SGLang，同上 |
|
||||||
| DSV4 H200 vLLM baseline | `experiments/dsv4_h200_vllm/run_bench.sh` | NVIDIA H200 + vLLM + DeepSeek-V4-Flash |
|
| `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` | H200 + vLLM |
|
||||||
| DSV4 H200 SGLang baseline | `experiments/dsv4_h200_sglang/run_bench.sh` | NVIDIA H200 + native SGLang + DeepSeek-V4-Flash |
|
| `experiments/h200/dsv4_h200_sglang_tp_dp_matrix/` | H200 + SGLang |
|
||||||
| DSV4 H200 SGLang vs vLLM | `experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh` | NVIDIA H200 上 SGLang 与 vLLM 控制变量对比（TP=8，最长 200k 上下文） |
|
| `experiments/p800/dsv4_p800_sglang_tp_dp_matrix/` | P800 + SGLang（INT8；TP2/DP4 启动 OOM 无数据，见 config.env 注释） |
|
||||||
| DSV4 H200 vLLM DSpark vs default | `experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh` | vLLM 开启 DSpark 投机解码 vs 默认配置，验证 TTFT 差异 |
|
| `experiments/pro6000/dsv4_pro6000_vllm_tp_dp_matrix/` | RTX 6000D + vLLM |
|
||||||
| DSV4 H200 max context length | `experiments/dsv4_h200_max_context_length/run_bench.sh` | 探索 SGLang / vLLM 在 H200 上能支持的最大输入长度 |
|
| `experiments/pro6000/dsv4_pro6000_sglang_tp_dp_matrix/` | RTX 6000D + SGLang |
|
||||||
| DSV4 H200 SGLang TP/DP Matrix | `experiments/dsv4_h200_sglang_tp_dp_matrix/run_bench.sh` | H200 上 SGLang 不同 TP×DP 配置的固定并发矩阵测试 |
|
|
||||||
| DSV4 H200 SGLang TP/DP Adaptive | `experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh` | H200 上 SGLang 不同 TP×DP 配置的自适应并发饱和点搜索 |
|
|
||||||
| DSV4 H200 vLLM TP/DP Matrix | `experiments/dsv4_h200_vllm_tp_dp_matrix/run_bench.sh` | H200 上 vLLM 不同 TP×DP 配置的固定并发矩阵测试 |
|
|
||||||
| DSV4 H200 vLLM TP/DP Adaptive | `experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh` | H200 上 vLLM 不同 TP×DP 配置的自适应并发饱和点搜索 |
|
|
||||||
| DSV4 H200 64k SGLang vs vLLM | `experiments/dsv4_h200_64k_sglang_vs_vllm/run_bench.sh` | H200 上 64k 上下文 SGLang 与 vLLM 对比 |
|
|
||||||
| DSV4 H200 256k 4k Probe | `experiments/dsv4_h200_256k_4k_probe/run_bench.sh` | H200 上 256k 输入/4k 输出长上下文探测 |
|
|
||||||
| DSV4 H200 vLLM TP2 Custom Bench | `experiments/dsv4_h200_vllm_tp2_custom_bench/run_bench.sh` | H200 上 vLLM TP=2 自定义压测客户端（多服务负载均衡） |
|
|
||||||
| DSV4 H200 vLLM TP4 Custom Bench | `experiments/dsv4_h200_vllm_tp4_custom_bench/run_bench.sh` | H200 上 vLLM TP=4 自定义压测客户端 |
|
|
||||||
| DSV4 H200 vLLM TP8 Custom Bench | `experiments/dsv4_h200_vllm_tp8_custom_bench/run_bench.sh` | H200 上 vLLM TP=8 自定义压测客户端 |
|
|
||||||
| DSV4 H200 vLLM MTP vs Default | `experiments/dsv4_h200_vllm_mtp_vs_default/run_bench.sh` | H200 上 vLLM MTP（Multi-Token Prediction）vs 默认配置对比 |
|
|
||||||
| DSV4 H200 Long Context Matrix | `experiments/dsv4_h200_long_context_matrix/run_bench.sh` | H200 上长上下文矩阵测试（多后端对比） |
|
|
||||||
| **DSV4 RTX 6000D SGLang TP/DP Adaptive** | `experiments/dsv4_pro6000_sglang_tp_dp_matrix/run_adaptive_concurrency.sh` | **RTX 6000D 上 SGLang 自适应并发饱和点搜索** |
|
|
||||||
| **DSV4 RTX 6000D vLLM TP/DP Adaptive** | `experiments/dsv4_pro6000_vllm_tp_dp_matrix/run_adaptive_concurrency.sh` | **RTX 6000D 上 vLLM 自适应并发饱和点搜索** |
|
|
||||||
|
|
||||||
### 旧结构（scripts/ + bench_results/）
|
每个目录内：`run_bench.sh` 跑固定并发矩阵；`run_adaptive_concurrency.sh` 从 C=1 指数倍增搜饱和点；`run_adaptive_concurrency_add16.sh` 从 C=16 线性 +16 步进、带 TTFT SLO 停止与回退（当前主力用法，见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`）。
|
||||||
|
|
||||||
仅保留还在被 `experiments/dsv4_h200_dspark/` 引用的 legacy DSpark 脚本，其余旧脚本已删除。
|
### 其他实验（H200 老形态 & 专项）
|
||||||
|
|
||||||
| 实验 | 脚本 | 最终报告 | 说明 |
|
| 实验 | 说明 |
|
||||||
|---|---|---|---|
|
|---|---|
|
||||||
| DSpark grid benchmark | `scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh` | `experiments/legacy_bench_results/dspark_grid_20260707-132641/report.md` | P1/P2/P3 全量网格 |
|
| `experiments/h200/dsv4_h200_sglang/` `…/dsv4_h200_vllm/` | H200 单引擎 baseline |
|
||||||
| DSpark spec-tokens 对比 | `scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh` | `experiments/legacy_bench_results/dspark_st_comparison_20260707-150649/comparison_report.md` | `--spec-tokens 3` vs `5` |
|
| `experiments/h200/dsv4_h200_sglang_vs_vllm/` | H200 引擎控制变量对比 |
|
||||||
|
| `experiments/h200/dsv4_h200_dspark/` `…/dsv4_h200_vllm_dspark_vs_default/` | DSpark 投机解码相关 |
|
||||||
|
| `experiments/h200/dsv4_h200_vllm_mtp_vs_default/` | vLLM MTP 对比 |
|
||||||
|
| `experiments/h200/dsv4_h200_vllm_tp{2,4,8}_custom_bench/` | 自定义压测客户端（多服务负载均衡） |
|
||||||
|
| `experiments/h200/dsv4_h200_max_context_length/` `…/dsv4_h200_long_context_matrix/` `…/dsv4_h200_256k_4k_probe/` `…/dsv4_h200_64k_sglang_vs_vllm/` | 长上下文专项 |
|
||||||
|
| `experiments/p800/dsv4_p800_sglang/` `…/dsv4_p800_max_context_length/` `…/dsv4_p800_long_context_matrix/` `…/dsv4_p800_256k_4k_probe/` | P800 baseline 与长上下文专项 |
|
||||||
|
| `experiments/TEMPLATE/` | 老式固定场景实验模板 |
|
||||||
|
|
||||||
## 快速复现
|
## 快速复现
|
||||||
|
|
||||||
### P800 SGLang
|
以 H20 vLLM 自适应并发搜索为例（其他平台同理，换目录即可）：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash experiments/dsv4_p800_sglang/run_bench.sh
|
# 1. dry-run：只打印搜索计划，不加载模型
|
||||||
|
cd experiments/h20/dsv4_h20_vllm_tp_dp_matrix
|
||||||
|
DRY_RUN=1 bash run_adaptive_concurrency_add16.sh
|
||||||
|
|
||||||
|
# 2. 单组合冒烟：只测 TP=8、1K/128，并发上限 8
|
||||||
|
RUN_ID=smoke-$(date +%Y%m%d-%H%M%S) \
|
||||||
|
TP_LIST="8" ISL_LIST="1024" OSL_LIST="128" GRID_LIMIT=1 SEARCH_MAX_CONCURRENCY=8 \
|
||||||
|
bash run_adaptive_concurrency_add16.sh
|
||||||
|
|
||||||
|
# 3. 正式跑（放 tmux）
|
||||||
|
tmux new-session -d -s h20-vllm-adaptive \
|
||||||
|
"cd $(pwd) && bash run_adaptive_concurrency_add16.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
### H200 DSpark
|
更多用法（断点续跑 RESUME_RUN_ID、常用覆盖参数）见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`。
|
||||||
|
|
||||||
```bash
|
## 新平台 / 新模型接入
|
||||||
bash experiments/dsv4_h200_dspark/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 SGLang baseline
|
- **新显卡**：按 [`docs/NEW_PLATFORM_GUIDE.md`](docs/NEW_PLATFORM_GUIDE.md) 执行，核心动作是复制 `experiments/h20/dsv4_h20_<engine>_tp_dp_matrix`、改 `config.env` / `adaptive_config.env` / `matrix.json` 三件套，再加 `platforms/<chip>.env`。
|
||||||
|
- **新模型（GLM5.2）**：复用同一实验目录结构，新增 `experiments/<platform>/glm52_<platform>_<engine>_tp_dp_matrix/`，改模型路径、量化与引擎启动参数即可；SLO 口径、报告流程、多维表格结构不变。
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_sglang/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 SGLang vs vLLM 对比
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_sglang_vs_vllm/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 vLLM DSpark vs default
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_vllm_dspark_vs_default/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 SGLang TP/DP Matrix
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_sglang_tp_dp_matrix/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 SGLang TP/DP 自适应并发搜索
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 vLLM TP/DP Matrix
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_vllm_tp_dp_matrix/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 vLLM TP/DP 自适应并发搜索
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 64k SGLang vs vLLM
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_64k_sglang_vs_vllm/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 256k 4k Probe
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_256k_4k_probe/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 vLLM TP2/TP4/TP8 Custom Bench
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_vllm_tp2_custom_bench/run_bench.sh
|
|
||||||
bash experiments/dsv4_h200_vllm_tp4_custom_bench/run_bench.sh
|
|
||||||
bash experiments/dsv4_h200_vllm_tp8_custom_bench/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 vLLM MTP vs Default
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_vllm_mtp_vs_default/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### H200 Long Context Matrix
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_h200_long_context_matrix/run_bench.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### RTX 6000D SGLang TP/DP 自适应并发搜索
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_pro6000_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### RTX 6000D vLLM TP/DP 自适应并发搜索
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash experiments/dsv4_pro6000_vllm_tp_dp_matrix/run_adaptive_concurrency.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### DSpark grid / spec-tokens（旧结构）
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
||||||
bash scripts/benchmark_dspark_0707/run_dspark_st_comparison.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### 解析已有结果
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 新结构
|
|
||||||
python3 experiments/dsv4_h200_dspark/parse_results.py \
|
|
||||||
experiments/dsv4_h200_dspark/results/<run_id>
|
|
||||||
|
|
||||||
# 旧结构
|
|
||||||
/data/user1/yy/envs/sglang/bin/python scripts/benchmark_dspark_0707/parse_results.py \
|
|
||||||
/data/user1/yy/experiments/legacy_bench_results/dspark_grid_20260707-132641
|
|
||||||
```
|
|
||||||
|
|
||||||
## SLO 参考
|
|
||||||
|
|
||||||
详见 `docs/SLO_STANDARDS.md`。
|
|
||||||
|
|
||||||
主要关注指标：
|
|
||||||
|
|
||||||
- **TTFT P95**：S2（DeepSeek-V4-Flash 所在层）目标 `< 3s`
|
|
||||||
- **TPOT**：S2 目标 `< 50ms`
|
|
||||||
|
|
||||||
## 环境要求
|
## 环境要求
|
||||||
|
|
||||||
- **NVIDIA H200**：Python env `/data/user1/yy/envs/vllm-dspark`（服务端）、`/data/user1/yy/envs/sglang`（压测客户端）；模型 `/data/models/DeepSeek-V4-Flash`、`/data/models/DeepSeek-V4-Flash-DSpark`。
|
- 公共：bash、Python 3、`sglang.bench_serving` 压测客户端（Docker 或 venv）；绘图分析用 `envs/charts/`（matplotlib）。
|
||||||
- **NVIDIA RTX 6000D**：Python env `/root/.miniconda3/envs/sglang`（压测客户端）；模型 `/data/hf_models/DeepSeek-V4-Flash`；Docker 镜像 `lmsysorg/sglang:latest`、`vllm/vllm-openai:latest`。
|
- 各平台的镜像 / venv / 模型路径见对应实验目录的 `config.env` 与 `platforms/<chip>.env`；**模型与数据集路径是机器相关的**，换机器时按实际路径调整。
|
||||||
- **Kunlun P800**：Docker 镜像 `iregistry.baidu-int.com/xpu/sglang-p800-pd-disagg-0510:20260511_4202`；模型 `/data1/models/DeepSeek-V4-Flash-INT8`；压测客户端在容器内运行。
|
- 环境搭建规范见 `envs/README.md` 与 `envs/UV_ENV_SETUP.md`。
|
||||||
|
|
||||||
平台相关常量见 `platforms/*.env`。
|
## 注意事项
|
||||||
|
|
||||||
|
- 提交内容：实验代码 + 最终产物（results.json / report.md / adaptive 的 jsonl 与 summary）；日志、raw_outputs、gpu_logs 不入库（`.gitignore` 已排除）。
|
||||||
|
- 仓库会clone到多台机器，实验脚本内引用公共组件一律用 `${SCRIPT_DIR}/../../../scripts/common`（TEMPLATE 为 `../../`），不要写绝对路径。
|
||||||
|
|||||||
@ -4,8 +4,10 @@
|
|||||||
|
|
||||||
## 1. 目录结构
|
## 1. 目录结构
|
||||||
|
|
||||||
|
实验目录为三层结构：`experiments/<platform>/<experiment_name>/`，其中 platform ∈ {h20, h200, p800, pro6000}；另有 `experiments/TEMPLATE/` 作为新实验模板。
|
||||||
|
|
||||||
```
|
```
|
||||||
experiments/<experiment_name>/
|
experiments/<platform>/<experiment_name>/
|
||||||
├── config.env # 实验参数唯一来源
|
├── config.env # 实验参数唯一来源
|
||||||
├── run_bench.sh # 编排入口
|
├── run_bench.sh # 编排入口
|
||||||
├── start_sglang.sh # SGLang server 启动脚本
|
├── start_sglang.sh # SGLang server 启动脚本
|
||||||
@ -24,22 +26,27 @@ experiments/<experiment_name>/
|
|||||||
└── logs/ # .gitignore 忽略
|
└── logs/ # .gitignore 忽略
|
||||||
```
|
```
|
||||||
|
|
||||||
通用工具集中放在 `scripts/common/`，不要复制到每个实验：
|
上述是 TEMPLATE 式的固定场景对比实验布局（sglang vs vllm，老形态，仍可用）。当前主流实验形态是 TP/DP matrix + 自适应并发搜索：实验目录内由 `matrix.json` + `generate_scenarios.py` 生成场景，`adaptive_config.env` 配置搜索参数，`run_adaptive_concurrency.sh` 驱动，结果落盘 `adaptive_results/<run_id>/`（`adaptive_points.jsonl`、`adaptive_summary.{jsonl,md,csv}`、`run_manifest.json`、`tp*_dp*/` 等），用法见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`。
|
||||||
|
|
||||||
|
通用工具集中放在 `scripts/common/`，不要复制到每个实验。三层实验目录以 `../../../scripts/common` 引用（`experiments/TEMPLATE/` 为两层，用 `../../scripts/common`）：
|
||||||
|
|
||||||
- `scripts/common/lib.sh`：日志、metadata、JSON 工具
|
- `scripts/common/lib.sh`：日志、metadata、JSON 工具
|
||||||
- `scripts/common/platform.sh`：平台自动检测
|
- `scripts/common/platform.sh`：平台自动检测
|
||||||
- `scripts/common/warmup.py`：server 预热
|
- `scripts/common/warmup.py`：server 预热
|
||||||
- `scripts/common/parse_backend.py`：解析 raw jsonl -> results.json + report.md
|
- `scripts/common/parse_backend.py`：解析 raw jsonl -> results.json + report.md
|
||||||
- `scripts/common/compare.py`：生成 SGLang vs vLLM 对比表
|
- `scripts/common/compare.py`：生成 SGLang vs vLLM 对比表
|
||||||
|
- `scripts/common/adaptive_bench_lib.sh` / `adaptive_concurrency.py`：自适应并发搜索
|
||||||
|
- `scripts/common/server_docker.sh` / `bench_client_docker.sh`：Docker server / client 管理
|
||||||
|
- `scripts/common/adaptive_heartbeat.sh`：自适应测试心跳保活
|
||||||
|
|
||||||
## 2. 新增一个实验
|
## 2. 新增一个实验
|
||||||
|
|
||||||
最快方式：
|
最快方式：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp -r experiments/TEMPLATE experiments/<your_experiment_name>
|
cp -r experiments/TEMPLATE experiments/<platform>/<your_experiment_name>
|
||||||
# 修改 config.env、start_*.sh
|
# 修改 config.env、start_*.sh
|
||||||
bash experiments/<your_experiment_name>/run_bench.sh
|
bash experiments/<platform>/<your_experiment_name>/run_bench.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2.1 config.env 必备字段
|
### 2.1 config.env 必备字段
|
||||||
@ -52,8 +59,9 @@ MODEL_PATH="/data/models/DeepSeek-V4-Flash"
|
|||||||
SGLANG_PORT="${SGLANG_PORT:-30006}"
|
SGLANG_PORT="${SGLANG_PORT:-30006}"
|
||||||
VLLM_PORT="${VLLM_PORT:-30005}"
|
VLLM_PORT="${VLLM_PORT:-30005}"
|
||||||
|
|
||||||
VENV_SGLANG="${VENV_SGLANG:-/data/user1/yy/envs/sglang}"
|
# venv 路径按机器实际配置调整
|
||||||
VENV_VLLM="${VENV_VLLM:-/data/user1/yy/envs/vllm}"
|
VENV_SGLANG="${VENV_SGLANG:-/path/to/envs/sglang}"
|
||||||
|
VENV_VLLM="${VENV_VLLM:-/path/to/envs/vllm}"
|
||||||
|
|
||||||
export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}"
|
export CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}"
|
||||||
TP="${TP:-8}"
|
TP="${TP:-8}"
|
||||||
@ -89,9 +97,11 @@ VLLM_START_SCRIPT="${SCRIPT_DIR:-.}/start_vllm.sh"
|
|||||||
|
|
||||||
### 3.1 芯片级默认配置
|
### 3.1 芯片级默认配置
|
||||||
|
|
||||||
芯片相关默认值放到 `platforms/<chip>.env`，例如：
|
芯片相关默认值放到 `platforms/<chip>.env`，现有：
|
||||||
|
|
||||||
|
- `platforms/nvidia_h20.env`
|
||||||
- `platforms/nvidia_h200.env`
|
- `platforms/nvidia_h200.env`
|
||||||
|
- `platforms/nvidia_rtx6000d.env`
|
||||||
- `platforms/kunlun_p800.env`
|
- `platforms/kunlun_p800.env`
|
||||||
|
|
||||||
实验级覆盖通过 `config.env` 实现。
|
实验级覆盖通过 `config.env` 实现。
|
||||||
@ -110,7 +120,7 @@ VLLM_START_SCRIPT="${SCRIPT_DIR:-.}/start_vllm.sh"
|
|||||||
|
|
||||||
## 4. SLO 标准
|
## 4. SLO 标准
|
||||||
|
|
||||||
默认使用 S2 层级：
|
默认使用 S2 层级（完整分级定义见 `docs/SLO_STANDARDS.md`）：
|
||||||
|
|
||||||
- TTFT P95 < 3000 ms
|
- TTFT P95 < 3000 ms
|
||||||
- TPOT mean < 50 ms
|
- TPOT mean < 50 ms
|
||||||
@ -142,14 +152,14 @@ VLLM_START_SCRIPT="${SCRIPT_DIR:-.}/start_vllm.sh"
|
|||||||
### 6.3 提交示例
|
### 6.3 提交示例
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git add experiments/<name>/config.env \
|
git add experiments/<platform>/<name>/config.env \
|
||||||
experiments/<name>/run_bench.sh \
|
experiments/<platform>/<name>/run_bench.sh \
|
||||||
experiments/<name>/start_*.sh \
|
experiments/<platform>/<name>/start_*.sh \
|
||||||
experiments/<name>/results/<run_id>/comparison.md \
|
experiments/<platform>/<name>/results/<run_id>/comparison.md \
|
||||||
experiments/<name>/results/<run_id>/sglang/results.json \
|
experiments/<platform>/<name>/results/<run_id>/sglang/results.json \
|
||||||
experiments/<name>/results/<run_id>/sglang/report.md \
|
experiments/<platform>/<name>/results/<run_id>/sglang/report.md \
|
||||||
experiments/<name>/results/<run_id>/vllm/results.json \
|
experiments/<platform>/<name>/results/<run_id>/vllm/results.json \
|
||||||
experiments/<name>/results/<run_id>/vllm/report.md
|
experiments/<platform>/<name>/results/<run_id>/vllm/report.md
|
||||||
|
|
||||||
git commit -m "results: <experiment_name> run <run_id>"
|
git commit -m "results: <experiment_name> run <run_id>"
|
||||||
git push origin main
|
git push origin main
|
||||||
@ -174,7 +184,7 @@ declare -a SCENARIOS=(
|
|||||||
|
|
||||||
## 8. 命名约定
|
## 8. 命名约定
|
||||||
|
|
||||||
- 实验目录：`{model}_{chip}_{scenario}_{backend_vs_backend}`，例如 `dsv4_h200_sglang_vs_vllm`
|
- 实验目录：`experiments/<platform>/{model}_{chip}_{scenario}_{backend_vs_backend}`，例如 `experiments/h200/dsv4_h200_sglang_vs_vllm`
|
||||||
- scenario 名称：`c{concurrency}_i{input_len}_o{output_len}`
|
- scenario 名称：`c{concurrency}_i{input_len}_o{output_len}`
|
||||||
- raw 输出：`{backend}_{label}_{MMDD}_{concurrency}_{input_len}_{output_len}.jsonl`
|
- raw 输出：`{backend}_{label}_{MMDD}_{concurrency}_{input_len}_{output_len}.jsonl`
|
||||||
- run_id：`YYYYMMDD-HHMMSS`
|
- run_id：`YYYYMMDD-HHMMSS`
|
||||||
|
|||||||
@ -4,7 +4,7 @@ This guide covers how to use this benchmark repository on an NVIDIA H200 machine
|
|||||||
|
|
||||||
## Current state
|
## Current state
|
||||||
|
|
||||||
The H200 path is currently based on **native host virtual environments** (not Docker):
|
The older H200 experiments (e.g. `experiments/h200/dsv4_h200_dspark/`) are based on **native host virtual environments** (not Docker):
|
||||||
|
|
||||||
- Server engine: `vllm-dspark`
|
- Server engine: `vllm-dspark`
|
||||||
- Server env: `envs/vllm-dspark`
|
- Server env: `envs/vllm-dspark`
|
||||||
@ -12,13 +12,15 @@ The H200 path is currently based on **native host virtual environments** (not Do
|
|||||||
- Default model: `/data/models/DeepSeek-V4-Flash-DSpark`
|
- Default model: `/data/models/DeepSeek-V4-Flash-DSpark`
|
||||||
- Default port: `30004`
|
- Default port: `30004`
|
||||||
|
|
||||||
The legacy benchmark scripts live under `scripts/benchmark_dspark_0707/`. A migration wrapper experiment `experiments/dsv4_h200_dspark/` is also provided so the results follow the new `experiments/<name>/results/<RUN_ID>/` layout.
|
Note: the current mainstream H200 experiments — the TP/DP matrix adaptive-concurrency suites `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` and `experiments/h200/dsv4_h200_sglang_tp_dp_matrix/` — are Docker-based (`USE_DOCKER=1` for the server, benchmark client via `DOCKER_CLIENT_IMAGE`); see `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`.
|
||||||
|
|
||||||
|
The legacy benchmark suite `scripts/benchmark_dspark_0707/` has been removed from this repo. The migration wrapper experiment `experiments/h200/dsv4_h200_dspark/` still exists and produces the `experiments/h200/<name>/results/<RUN_ID>/` layout, but its default `LEGACY_GRID_SCRIPT` / `SERVER_START_SCRIPT` point at the removed scripts and must be overridden to run.
|
||||||
|
|
||||||
## 1. Pull and verify
|
## 1. Pull and verify
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone <repo-url> /data/user1/yy
|
git clone <repo-url> sskj # clone path: adjust to your machine
|
||||||
cd /data/user1/yy
|
cd sskj
|
||||||
|
|
||||||
# Platform should auto-detect as nvidia_h200
|
# Platform should auto-detect as nvidia_h200
|
||||||
source scripts/common/platform.sh
|
source scripts/common/platform.sh
|
||||||
@ -30,36 +32,28 @@ If auto-detection fails, set it explicitly:
|
|||||||
PLATFORM=nvidia_h200 source scripts/common/platform.sh
|
PLATFORM=nvidia_h200 source scripts/common/platform.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
## 2. Run the existing DSpark benchmark grid
|
## 2. Legacy DSpark benchmark grid (removed)
|
||||||
|
|
||||||
This uses the legacy scripts and writes results to `bench_results/dspark_grid_<RUN_ID>/`.
|
The legacy suite `scripts/benchmark_dspark_0707/` has been removed from this repo, along with its `bench_results/dspark_grid_<RUN_ID>/` output location (historical results were archived elsewhere). Use the wrapper experiment below, or the current tp_dp matrix experiments (`experiments/h200/dsv4_h200_{vllm,sglang}_tp_dp_matrix/`), instead.
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
To reuse an already-running server:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
SKIP_MANAGE_SERVER=1 bash scripts/benchmark_dspark_0707/run_dspark_benchmark_grid.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
## 3. Run the migration wrapper experiment
|
## 3. Run the migration wrapper experiment
|
||||||
|
|
||||||
This produces results in the new experiment-centric layout:
|
This produces results in the experiment-centric layout:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash experiments/dsv4_h200_dspark/run_bench.sh
|
bash experiments/h200/dsv4_h200_dspark/run_bench.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Results land in `experiments/dsv4_h200_dspark/results/<RUN_ID>/`.
|
Results land in `experiments/h200/dsv4_h200_dspark/results/<RUN_ID>/`.
|
||||||
|
|
||||||
|
Caveat: the wrapper's default `LEGACY_GRID_SCRIPT` and `SERVER_START_SCRIPT` refer to the removed `scripts/benchmark_dspark_0707/` suite and `scripts/start_dsv4_dspark_8card.sh`; set both explicitly (or restore equivalent scripts) before running.
|
||||||
|
|
||||||
## 4. Create a new H200 experiment
|
## 4. Create a new H200 experiment
|
||||||
|
|
||||||
To add a new H200 benchmark (for example a different model or engine), create:
|
To add a new H200 benchmark (for example a different model or engine), create:
|
||||||
|
|
||||||
```
|
```
|
||||||
experiments/<your_name>/
|
experiments/h200/<your_name>/
|
||||||
├── README.md # What this experiment measures
|
├── README.md # What this experiment measures
|
||||||
├── config.env # Model, port, scenarios, venv paths
|
├── config.env # Model, port, scenarios, venv paths
|
||||||
├── start_server.sh # (optional) native server launch
|
├── start_server.sh # (optional) native server launch
|
||||||
@ -77,9 +71,9 @@ PORT="${PORT:-30004}"
|
|||||||
BACKEND="${BACKEND:-vllm}"
|
BACKEND="${BACKEND:-vllm}"
|
||||||
ENGINE="${ENGINE:-vllm-dspark}"
|
ENGINE="${ENGINE:-vllm-dspark}"
|
||||||
|
|
||||||
# Native venvs
|
# Native venvs (paths are per-machine; adjust to yours)
|
||||||
VENV_SERVER="${VENV_SERVER:-/data/user1/yy/envs/vllm-dspark}"
|
VENV_SERVER="${VENV_SERVER:-/path/to/envs/vllm-dspark}"
|
||||||
VENV_CLIENT="${VENV_CLIENT:-/data/user1/yy/envs/sglang}"
|
VENV_CLIENT="${VENV_CLIENT:-/path/to/envs/sglang}"
|
||||||
|
|
||||||
SCENARIOS=(
|
SCENARIOS=(
|
||||||
"32 512 256"
|
"32 512 256"
|
||||||
@ -89,11 +83,11 @@ SCENARIOS=(
|
|||||||
Then source the platform loader and shared helpers at the top of `run_bench.sh`:
|
Then source the platform loader and shared helpers at the top of `run_bench.sh`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
For a concrete example, see `experiments/dsv4_h200_dspark/`.
|
For a concrete example, see `experiments/h200/dsv4_h200_dspark/`. For the current mainstream form (TP/DP matrix + adaptive concurrency search, Docker-based), copy `experiments/h200/dsv4_h200_vllm_tp_dp_matrix/` instead; usage in `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`.
|
||||||
|
|
||||||
## 5. Cross-platform comparison
|
## 5. Cross-platform comparison
|
||||||
|
|
||||||
@ -101,5 +95,5 @@ For a concrete example, see `experiments/dsv4_h200_dspark/`.
|
|||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- `platforms/nvidia_h200.env` assumes the legacy directory layout under `/data/user1/yy`. If your H200 machine uses different paths, override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT`.
|
- `platforms/nvidia_h200.env` still carries venv defaults under `/data/user1/yy` (paths from another machine). Override `VENV_VLLM_DSPARK`, `VENV_SGLANG`, `MODEL_ROOT`, or `SERVER_START_SCRIPT` to match your machine.
|
||||||
- Native server management helpers are not yet as mature as the Docker helpers in `scripts/common/server_docker.sh`. The H200 wrapper currently delegates to the existing `scripts/start_dsv4_dspark_8card.sh`.
|
- Native server management helpers are not as mature as the Docker helpers in `scripts/common/server_docker.sh`. The wrapper's default `SERVER_START_SCRIPT` (`scripts/start_dsv4_dspark_8card.sh`) no longer exists in this repo and must be overridden.
|
||||||
|
|||||||
139
docs/NEW_PLATFORM_GUIDE.md
Normal file
139
docs/NEW_PLATFORM_GUIDE.md
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
# 新平台接入指南（New Platform Onboarding SOP）
|
||||||
|
|
||||||
|
> 目标：新显卡（GPU/NPU）到货后，用最短时间跑起与既有平台口径一致的推理性能实验。
|
||||||
|
> 读者：执行接入的人或 Agent。带 🤖 的步骤 Agent 可自动完成；带 🧑 的步骤需要人确认或提供信息。
|
||||||
|
|
||||||
|
## 0. 前置准备 🧑
|
||||||
|
|
||||||
|
接入前确认以下信息就绪，缺一项都会卡住后续步骤：
|
||||||
|
|
||||||
|
| 项目 | 说明 | 示例 |
|
||||||
|
|---|---|---|
|
||||||
|
| 机器与驱动 | 8 卡整机，驱动/固件可用，`nvidia-smi` 或厂商等价工具（`xpu-smi`）能看到全部卡 | — |
|
||||||
|
| 推理镜像 | 该平台可用的 vLLM / SGLang 镜像或 native 环境 | `vllm/vllm-openai:latest`、`iregistry.baidu-int.com/xpu/sglang-p800-...` |
|
||||||
|
| 模型权重 | 模型在本机的路径与精度格式 | `/data1/hf_models/DeepSeek-V4-Flash`（FP8）、`/data1/models/DeepSeek-V4-Flash-INT8` |
|
||||||
|
| 上下文上限 | 该平台显存能支撑的 max context | H20：1M；P800 INT8：140k |
|
||||||
|
| 数据集 | ShareGPT 种子文件（random workload 用） | `datasets/ShareGPT_V3_unfiltered_cleaned_split.json` |
|
||||||
|
| 显存约束 | 单卡显存，预判 TP/DP 可行性（见 §4.2） | P800 96 GiB/卡 → TP2 放不下 INT8 专家权重 |
|
||||||
|
|
||||||
|
## 1. 平台配置 🤖
|
||||||
|
|
||||||
|
1. 新建 `platforms/<chip>.env`（参考 `platforms/nvidia_h20.env` / `platforms/kunlun_p800.env`），至少包含 `CHIP`、`ACCELERATOR`、`HARDWARE`、`ENGINE`、`DEFAULT_PORT`、`MODEL_ROOT`；Docker 平台再加 `DOCKER_IMAGE`、`CONTAINER_NAME`；需要运行时补丁的加 `PATCH_ROOT` 并把补丁放到 `platforms/patches/<chip>/`。
|
||||||
|
2. 在 `scripts/common/platform.sh` 的自动探测分支里加上新平台（当前只识别 P800 / H20 / 其他 NVIDIA→H200，**新 NVIDIA 卡不改会被误判成 H200**）。
|
||||||
|
3. 在 `platforms/README.md` 的平台表中加一行。
|
||||||
|
|
||||||
|
## 2. 复制实验目录 🤖
|
||||||
|
|
||||||
|
**NVIDIA 平台**（有官方 vLLM/SGLang 镜像）直接复制 H20 目录——它是路径正确、维护最新的范本，且 NVIDIA 平台间 `start_*_docker.sh` 零改动可复用：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp -r experiments/h20/dsv4_h20_vllm_tp_dp_matrix experiments/<chip>/dsv4_<chip>_vllm_tp_dp_matrix
|
||||||
|
cp -r experiments/h20/dsv4_h20_sglang_tp_dp_matrix experiments/<chip>/dsv4_<chip>_sglang_tp_dp_matrix
|
||||||
|
# 清理复制过来的结果与缓存
|
||||||
|
rm -rf experiments/<chip>/*/adaptive_results experiments/<chip>/*/results experiments/<chip>/*/__pycache__
|
||||||
|
```
|
||||||
|
|
||||||
|
**非 NVIDIA 平台**（XPU 等）同样从 H20 目录复制，但需按 §4 重写启动脚本与平台回调，以 `experiments/p800/dsv4_p800_sglang_tp_dp_matrix` 为范例。
|
||||||
|
|
||||||
|
> 不要抄 `experiments/h200/` 或 `experiments/pro6000/` 的目录结构组织方式以外的内容：那里存在 `dsl`（应为 `osl`）的批量误改名残留，新目录以 H20 版为准。
|
||||||
|
|
||||||
|
## 3. 修改三件套配置 🤖（🧑 确认关键参数）
|
||||||
|
|
||||||
|
### 3.1 `config.env`（核心参数面板）
|
||||||
|
|
||||||
|
| 必改项 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `EXPERIMENT` | 实验名，与目录名一致 |
|
||||||
|
| `MODEL_NAME` / `MODEL_PATH` / `SERVED_MODEL_NAME` | 本机模型路径（机器相关） |
|
||||||
|
| `DOCKER_IMAGE` / `CONTAINER_NAME` | 平台镜像；容器名专用化避免冲突 |
|
||||||
|
| `VLLM_PORT` / `SGLANG_PORT` | 避开已占用端口 |
|
||||||
|
| `CUDA_VISIBLE_DEVICES` / `XPU_VISIBLE_DEVICES` | 设备选择 |
|
||||||
|
| `MAX_MODEL_LEN` / `CONTEXT_LENGTH` | 按平台显存与需求设定 |
|
||||||
|
| `GPU_MEMORY_UTILIZATION` / `MEM_FRACTION_STATIC` | 显存水位（H20 0.9 / P800 0.8） |
|
||||||
|
| `KV_CACHE_DTYPE`、`BLOCK_SIZE` / `PAGE_SIZE` | KV cache 精度与块大小 |
|
||||||
|
| `PARALLEL_CONFIGS` | 按显存裁剪（见 §4.2） |
|
||||||
|
| `DATASET_PATH` | 本机数据集路径 |
|
||||||
|
|
||||||
|
### 3.2 `adaptive_config.env`
|
||||||
|
|
||||||
|
一般保持默认即可（起步 C=16、×2 或 +16 步进、TTFT_SLO_MS=4000、TTFT_GROUP_SKIP_MS=8000、num_prompts=5×C）。显存小或单请求 prefill 慢的平台，可适当降低 `SEARCH_MAX_CONCURRENCY`。
|
||||||
|
|
||||||
|
### 3.3 `matrix.json`
|
||||||
|
|
||||||
|
按 `CONTEXT_LENGTH` 和显存砍 shape：超出上下文上限的 ISL 删掉，显存放不下的组合标 `N`。参照 H20（1M 上下文，7 个 ISL 档）与 P800（140k 上下文，5 个 ISL 档）两版的取舍。
|
||||||
|
|
||||||
|
## 4. 非 NVIDIA 平台的额外工作（以 P800 为范例）🧑🤖
|
||||||
|
|
||||||
|
### 4.1 需要重写的地方
|
||||||
|
|
||||||
|
- **`start_*_docker.sh`**：设备映射（如 `/dev/xpu*`）、厂商 env 变量、引擎 launch args（attention backend、量化、kv-cache dtype、cuda graph 等价物开关）、补丁挂载（如 P800 挂 `bench_serving.py` 补丁补 P95 指标）、容器内 bootstrap。
|
||||||
|
- **`run_adaptive_concurrency*.sh` 的 5 个 `engine_*` 回调**：`engine_start_server` / `engine_stop_server`（pid kill 还是 `docker rm -f`）、`engine_run_bench`（独立客户端容器还是 `docker exec` 进服务端容器）、`engine_detect_oom`（日志文件还是 `docker logs`）、`engine_build_server_args`。
|
||||||
|
- **GPU 监控**：公共库的 `adaptive_start_gpu_monitor` 写死 `nvidia-smi`，非 NVIDIA 平台在 run 脚本里用厂商工具重写该函数（P800 即用 `xpu-smi` 覆盖）。
|
||||||
|
- **warmup 能力差异**：部分厂商镜像的 bench_serving 不支持 `--warmup-requests`，需设 `BENCH_WARMUP_MAX_REQUESTS=0` 并在 adaptive_config.env 里注明。
|
||||||
|
|
||||||
|
### 4.2 显存可行性预判（避免白跑）
|
||||||
|
|
||||||
|
MoE 模型的专家权重在未开 EP 时按 TP 组切分、DP 副本间不共享：每卡专家权重 ≈ 专家总权重 / TP。实例：DSV4-Flash-INT8 路由专家约 264 GiB，P800 单卡 96 GiB → TP=2 每卡需 137 GiB，启动即 OOM，TP≥4 才可行。接入时先算一遍，把不可行的配置从 `PARALLEL_CONFIGS` 去掉，并把根因写进 config.env 注释（照 P800 的做法）。
|
||||||
|
|
||||||
|
## 5. 冒烟 🤖
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd experiments/<chip>/dsv4_<chip>_<engine>_tp_dp_matrix
|
||||||
|
|
||||||
|
# 1. dry-run：只打印计划，不起服务
|
||||||
|
DRY_RUN=1 bash run_adaptive_concurrency_add16.sh
|
||||||
|
|
||||||
|
# 2. 单 shape 小并发：验证起服务、发压测、出指标全链路
|
||||||
|
RUN_ID=smoke-$(date +%Y%m%d-%H%M%S) \
|
||||||
|
TP_LIST="8" ISL_LIST="1024" OSL_LIST="128" GRID_LIMIT=1 SEARCH_MAX_CONCURRENCY=8 \
|
||||||
|
bash run_adaptive_concurrency_add16.sh
|
||||||
|
|
||||||
|
# 3. 检查 smoke 产物：adaptive_results/smoke-*/adaptive_points.jsonl 应有 COMPLETED 点
|
||||||
|
```
|
||||||
|
|
||||||
|
冒烟失败的常见原因：镜像 launch args 不被支持（查服务端日志）、端口冲突、数据集路径不存在、显存不足（回 §4.2）。
|
||||||
|
|
||||||
|
## 6. 正式跑 🤖
|
||||||
|
|
||||||
|
```bash
|
||||||
|
tmux new-session -d -s <chip>-<engine>-adaptive \
|
||||||
|
"cd $(pwd) && bash run_adaptive_concurrency_add16.sh"
|
||||||
|
```
|
||||||
|
|
||||||
|
- 心跳监控：`scripts/common/adaptive_heartbeat.sh` 或 `tail -f adaptive_results/<run_id>/logs/orchestrator.log`。
|
||||||
|
- 中断了用 `RESUME_RUN_ID=<run_id>` 断点续跑（按已终态 shape 去重）。
|
||||||
|
- 完整参数说明见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`。
|
||||||
|
|
||||||
|
## 7. 分析、报告与归档 🤖（🧑 确认发布）
|
||||||
|
|
||||||
|
1. **分析**：用 Python 从 `adaptive_points.jsonl` / `adaptive_shapes.jsonl` 实算（SLO：TTFT P95 < 3s 且 TPOT P95 < 50ms，严格小于），禁止目测估算。
|
||||||
|
2. **飞书报告**：按飞书 wiki「显卡性能报告 / 性能报告编写指南（README）」的骨架写到「显卡性能报告/<平台>」节点下；图表用 `envs/charts/bin/python` + matplotlib 生成。
|
||||||
|
3. **多维表格**：在「模型推理适配 Bench 迭代跟踪」的对应平台表（没有则按现有 schema 新建）按"每成功探测点一行"回填；`SLO达标状态` 按上述口径判定。
|
||||||
|
4. **提交**：实验代码 + `adaptive_results/<run_id>/` 的 jsonl/summary/manifest 入库；日志与 raw_outputs 不入库。
|
||||||
|
|
||||||
|
## 8. 完成检查清单
|
||||||
|
|
||||||
|
- [ ] `platforms/<chip>.env` 已建，`platform.sh` 探测分支已加，`platforms/README.md` 已更新
|
||||||
|
- [ ] `config.env` / `adaptive_config.env` / `matrix.json` 三件套已按平台调整
|
||||||
|
- [ ] smoke 通过（有 COMPLETED 探测点）
|
||||||
|
- [ ] 正式 run 的 `run_manifest.json` 参数正确
|
||||||
|
- [ ] 飞书报告已发布到对应平台节点下
|
||||||
|
- [ ] 多维表格已回填
|
||||||
|
- [ ] git 提交不含日志、raw_outputs、gpu_logs
|
||||||
|
|
||||||
|
## 9. 已知坑
|
||||||
|
|
||||||
|
1. **公共库路径层级**：`experiments/<platform>/<experiment>/` 下的脚本引用公共组件必须是 `${SCRIPT_DIR}/../../../scripts/common`；`experiments/TEMPLATE/` 下是 `../../`。写错会在 source 阶段直接失败。
|
||||||
|
2. **机器相关绝对路径**：模型、数据集、venv 路径随机器不同（`/data1/...`、`/data3/...`），换机器先改 config.env，不要在脚本里写死。
|
||||||
|
3. **TTFT SLO 停止 ≠ 饱和**：add16 流派的搜索在 P95 TTFT 超 4000ms 即停，绝大多数 shape 不会到达 TPS 平台期，报告中的"最佳 TPS"是 TTFT 边界内最高值，分析时不要当成吞吐上限。
|
||||||
|
4. **`dsl` 命名坑**：h200/pro6000 目录部分文件把 `osl` 误改名成 `dsl`（含 CSV 表头），复制时以 h20 目录为准。
|
||||||
|
5. **结果结构以 jsonl 为准**：`.gitignore` 排除 csv/logs，跨机器汇总数据时用 `adaptive_points.jsonl`、`adaptive_shapes.jsonl`、`adaptive_summary.*`、`run_manifest.json`。
|
||||||
|
|
||||||
|
## 10. 新模型接入（GLM5.2）
|
||||||
|
|
||||||
|
GLM5.2 **完全复用**本 SOP 的实验与报告流程，差异只在配置层：
|
||||||
|
|
||||||
|
1. 新目录命名 `experiments/<platform>/glm52_<platform>_<engine>_tp_dp_matrix/`；
|
||||||
|
2. `config.env` 改模型路径、`SERVED_MODEL_NAME`、量化方式（按发布精度）、`CONTEXT_LENGTH`（按模型实际上限）、引擎 launch args（attention backend / rope / 量化 flag 等按模型要求）；
|
||||||
|
3. `matrix.json` 按模型上下文上限调整 ISL 档；
|
||||||
|
4. SLO 口径、报告骨架、多维表格结构不变；多维表格的 `模型名称` 选/新增 GLM5.2 对应选项。
|
||||||
@ -40,3 +40,4 @@ envs/
|
|||||||
- `sglang/`
|
- `sglang/`
|
||||||
- `sglang-src/`
|
- `sglang-src/`
|
||||||
- `vllm/`
|
- `vllm/`
|
||||||
|
- `charts/`（matplotlib 绘图环境，用于生成性能报告图表）
|
||||||
|
|||||||
@ -6,16 +6,18 @@
|
|||||||
|
|
||||||
## 1. 进入目录
|
## 1. 进入目录
|
||||||
|
|
||||||
|
以下路径均相对仓库根目录（如 H200 机器上的 `experiments/h200/...`）。
|
||||||
|
|
||||||
vLLM：
|
vLLM：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /data3/yy/sskj/experiments/dsv4_h200_vllm_tp_dp_matrix
|
cd experiments/h200/dsv4_h200_vllm_tp_dp_matrix
|
||||||
```
|
```
|
||||||
|
|
||||||
SGLang：
|
SGLang：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /data3/yy/sskj/experiments/dsv4_h200_sglang_tp_dp_matrix
|
cd experiments/h200/dsv4_h200_sglang_tp_dp_matrix
|
||||||
```
|
```
|
||||||
|
|
||||||
两个目录的入口都是：
|
两个目录的入口都是：
|
||||||
@ -43,19 +45,19 @@ ISL_LIST="1024" \
|
|||||||
OSL_LIST="128" \
|
OSL_LIST="128" \
|
||||||
GRID_LIMIT=1 \
|
GRID_LIMIT=1 \
|
||||||
SEARCH_MAX_CONCURRENCY=8 \
|
SEARCH_MAX_CONCURRENCY=8 \
|
||||||
RESULT_BASE=/data3/yy/sskj_adaptive_results/smoke \
|
RESULT_BASE="$PWD/adaptive_results/smoke" \
|
||||||
bash run_adaptive_concurrency.sh
|
bash run_adaptive_concurrency.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
## 4. 正式运行
|
## 4. 正式运行
|
||||||
|
|
||||||
长任务放进 tmux。vLLM 示例：
|
长任务放进 tmux。以下命令假设在仓库根目录下执行。vLLM 示例：
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tmux new-session -d -s vllm-adaptive \
|
tmux new-session -d -s vllm-adaptive \
|
||||||
"cd /data3/yy/sskj/experiments/dsv4_h200_vllm_tp_dp_matrix && \
|
"cd experiments/h200/dsv4_h200_vllm_tp_dp_matrix && \
|
||||||
RUN_ID=vllm-\$(date +%Y%m%d-%H%M%S) \
|
RUN_ID=vllm-\$(date +%Y%m%d-%H%M%S) \
|
||||||
RESULT_BASE=/data3/yy/sskj_adaptive_results/vllm \
|
RESULT_BASE=\$PWD/adaptive_results/vllm \
|
||||||
bash run_adaptive_concurrency.sh"
|
bash run_adaptive_concurrency.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -63,9 +65,9 @@ SGLang 示例：
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
tmux new-session -d -s sglang-adaptive \
|
tmux new-session -d -s sglang-adaptive \
|
||||||
"cd /data3/yy/sskj/experiments/dsv4_h200_sglang_tp_dp_matrix && \
|
"cd experiments/h200/dsv4_h200_sglang_tp_dp_matrix && \
|
||||||
RUN_ID=sglang-\$(date +%Y%m%d-%H%M%S) \
|
RUN_ID=sglang-\$(date +%Y%m%d-%H%M%S) \
|
||||||
RESULT_BASE=/data3/yy/sskj_adaptive_results/sglang \
|
RESULT_BASE=\$PWD/adaptive_results/sglang \
|
||||||
bash run_adaptive_concurrency.sh"
|
bash run_adaptive_concurrency.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="sglang"
|
ENGINE="sglang"
|
||||||
ENGINE_PORT="$SGLANG_PORT"
|
ENGINE_PORT="$SGLANG_PORT"
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="sglang"
|
ENGINE="sglang"
|
||||||
ENGINE_PORT="$SGLANG_PORT"
|
ENGINE_PORT="$SGLANG_PORT"
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -524,7 +524,7 @@ run_parallel_config() {
|
|||||||
|
|
||||||
# Parse results.
|
# Parse results.
|
||||||
log "parsing ${config_label} results"
|
log "parsing ${config_label} results"
|
||||||
"$PYTHON" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend sglang \
|
"$PYTHON" "${SCRIPT_DIR}/../../../../scripts/common/parse_backend.py" "$result_root" --backend sglang \
|
||||||
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="vllm"
|
ENGINE="vllm"
|
||||||
ENGINE_PORT="$VLLM_PORT"
|
ENGINE_PORT="$VLLM_PORT"
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="vllm"
|
ENGINE="vllm"
|
||||||
ENGINE_PORT="$VLLM_PORT"
|
ENGINE_PORT="$VLLM_PORT"
|
||||||
|
|||||||
@ -1,167 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Find the Total-TPS saturation concurrency for each vLLM TP/DP/ISL/OSL shape.
|
|
||||||
set -Eeuo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
|
||||||
# shellcheck source=/dev/null
|
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
|
||||||
# shellcheck source=/dev/null
|
|
||||||
source "${SCRIPT_DIR}/config.env"
|
|
||||||
# shellcheck source=/dev/null
|
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
|
||||||
# shellcheck source=/dev/null
|
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
|
||||||
|
|
||||||
ENGINE="vllm"
|
|
||||||
ENGINE_PORT="$VLLM_PORT"
|
|
||||||
RESULT_BASE="${RESULT_BASE:-${SCRIPT_DIR}/adaptive_results}"
|
|
||||||
ACTIVE_ENGINE_SERVER_LOG=""
|
|
||||||
|
|
||||||
if [[ -x "${VENV_CLIENT}/bin/python" ]]; then
|
|
||||||
PYTHON="${VENV_CLIENT}/bin/python"
|
|
||||||
else
|
|
||||||
PYTHON="$(command -v python3)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
DOCKER_IMAGE="${DOCKER_IMAGE:-vllm/vllm-openai:latest}"
|
|
||||||
DOCKER_CLIENT_IMAGE="${DOCKER_CLIENT_IMAGE:-lmsysorg/sglang:latest}"
|
|
||||||
|
|
||||||
engine_is_healthy() {
|
|
||||||
curl --fail --silent --show-error --max-time 5 \
|
|
||||||
"http://127.0.0.1:${ENGINE_PORT}/health" >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
engine_stop_server() {
|
|
||||||
local tp="$1"
|
|
||||||
local dp="$2"
|
|
||||||
local pid_file="${RUNTIME_BASE}/${EXPERIMENT}_vllm_tp${tp}_dp${dp}.pid"
|
|
||||||
|
|
||||||
if [[ -f "$pid_file" ]]; then
|
|
||||||
local pid
|
|
||||||
pid="$(cat "$pid_file")"
|
|
||||||
if kill -0 "$pid" 2>/dev/null; then
|
|
||||||
log "stopping vllm server pid=${pid} tp=${tp} dp=${dp}"
|
|
||||||
kill "$pid" 2>/dev/null || true
|
|
||||||
sleep 5
|
|
||||||
kill -9 "$pid" 2>/dev/null || true
|
|
||||||
fi
|
|
||||||
rm -f "$pid_file"
|
|
||||||
fi
|
|
||||||
docker rm -f "${EXPERIMENT}_vllm_tp${tp}_dp${dp}" >/dev/null 2>&1 || true
|
|
||||||
ACTIVE_ENGINE_SERVER_LOG=""
|
|
||||||
sleep 2
|
|
||||||
}
|
|
||||||
|
|
||||||
engine_build_server_args() {
|
|
||||||
local tp="$1"
|
|
||||||
local dp="$2"
|
|
||||||
local -a args=(
|
|
||||||
vllm serve "$MODEL_PATH"
|
|
||||||
--trust-remote-code
|
|
||||||
--kv-cache-dtype "$KV_CACHE_DTYPE"
|
|
||||||
--block-size "$BLOCK_SIZE"
|
|
||||||
--tensor-parallel-size "$tp"
|
|
||||||
--gpu-memory-utilization "$GPU_MEMORY_UTILIZATION"
|
|
||||||
--max-model-len "$MAX_MODEL_LEN"
|
|
||||||
--max-num-seqs "$MAX_NUM_SEQS"
|
|
||||||
--no-enable-flashinfer-autotune
|
|
||||||
--host 0.0.0.0
|
|
||||||
--port "$ENGINE_PORT"
|
|
||||||
)
|
|
||||||
if (( dp > 1 )); then
|
|
||||||
args+=(--data-parallel-size "$dp")
|
|
||||||
fi
|
|
||||||
printf '%q ' "${args[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
engine_start_server() {
|
|
||||||
local tp="$1"
|
|
||||||
local dp="$2"
|
|
||||||
local outer_log="${ADAPTIVE_LOG_DIR}/vllm_tp${tp}_dp${dp}.server.outer.log"
|
|
||||||
log "starting vllm server tp=${tp} dp=${dp}"
|
|
||||||
bash "${SCRIPT_DIR}/start_vllm_dp.sh" "$tp" "$dp" >> "$outer_log" 2>&1
|
|
||||||
if ! engine_is_healthy; then
|
|
||||||
log "ERROR: vllm health check failed tp=${tp} dp=${dp}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
ACTIVE_ENGINE_SERVER_LOG="$(
|
|
||||||
find "${RUNTIME_BASE}/logs" -maxdepth 1 -type f \
|
|
||||||
-name "${EXPERIMENT}_vllm*tp${tp}_dp${dp}_*.log" \
|
|
||||||
-printf '%T@ %p\n' 2>/dev/null | sort -nr | head -n 1 | cut -d' ' -f2-
|
|
||||||
)"
|
|
||||||
log "vllm server healthy tp=${tp} dp=${dp} log=${ACTIVE_ENGINE_SERVER_LOG:-unknown}"
|
|
||||||
}
|
|
||||||
|
|
||||||
engine_detect_oom() {
|
|
||||||
local detail_log="$1"
|
|
||||||
local tp="$2"
|
|
||||||
local dp="$3"
|
|
||||||
local pattern='CUDA out of memory|torch\.OutOfMemoryError|OutOfMemory|out of memory|OOM|RESOURCE_EXHAUSTED|Failed to allocate memory'
|
|
||||||
local outer_log="${ADAPTIVE_LOG_DIR}/vllm_tp${tp}_dp${dp}.server.outer.log"
|
|
||||||
local -a logs=("$detail_log" "$outer_log")
|
|
||||||
if [[ -n "$ACTIVE_ENGINE_SERVER_LOG" ]]; then
|
|
||||||
logs+=("$ACTIVE_ENGINE_SERVER_LOG")
|
|
||||||
fi
|
|
||||||
grep -Eiq "$pattern" "${logs[@]}" 2>/dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
engine_run_bench() {
|
|
||||||
local isl="$1"
|
|
||||||
local osl="$2"
|
|
||||||
local concurrency="$3"
|
|
||||||
local num_prompts="$4"
|
|
||||||
local output_file="$5"
|
|
||||||
local warmup_requests
|
|
||||||
warmup_requests="$(adaptive_warmup_request_count "$concurrency")"
|
|
||||||
local -a bench_args=(
|
|
||||||
--backend vllm
|
|
||||||
--host 127.0.0.1
|
|
||||||
--port "$ENGINE_PORT"
|
|
||||||
--dataset-name "$BENCH_DATASET_NAME"
|
|
||||||
--random-input-len "$isl"
|
|
||||||
--random-output-len "$osl"
|
|
||||||
--random-range-ratio "$RANDOM_RANGE_RATIO"
|
|
||||||
--num-prompts "$num_prompts"
|
|
||||||
--max-concurrency "$concurrency"
|
|
||||||
--request-rate 10000
|
|
||||||
--warmup-requests "$warmup_requests"
|
|
||||||
--output-file "$output_file"
|
|
||||||
--output-details
|
|
||||||
--disable-tqdm
|
|
||||||
)
|
|
||||||
if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
|
|
||||||
bench_args+=(--dataset-path "$DATASET_PATH")
|
|
||||||
else
|
|
||||||
bench_args+=(--tokenize-prompt)
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ "$USE_DOCKER_CLIENT" == "1" ]]; then
|
|
||||||
local -a volume_args=(-v "${MODEL_PATH}:${MODEL_PATH}:ro" -v "${RESULT_BASE}:${RESULT_BASE}")
|
|
||||||
if [[ "$BENCH_DATASET_NAME" == "random" ]]; then
|
|
||||||
volume_args+=(-v "${DATASET_PATH}:${DATASET_PATH}:ro")
|
|
||||||
fi
|
|
||||||
docker run --rm \
|
|
||||||
--network host \
|
|
||||||
"${volume_args[@]}" \
|
|
||||||
-e PYTHONUNBUFFERED=1 \
|
|
||||||
-e HF_HUB_OFFLINE=1 \
|
|
||||||
-e TRANSFORMERS_OFFLINE=1 \
|
|
||||||
"$DOCKER_CLIENT_IMAGE" \
|
|
||||||
python -m sglang.bench_serving "${bench_args[@]}"
|
|
||||||
else
|
|
||||||
"$PYTHON" -m sglang.bench_serving "${bench_args[@]}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
export -f engine_run_bench
|
|
||||||
export ENGINE_PORT MODEL_PATH RESULT_BASE DOCKER_CLIENT_IMAGE USE_DOCKER_CLIENT
|
|
||||||
export BENCH_DATASET_NAME DATASET_PATH RANDOM_RANGE_RATIO BENCH_WARMUP_MAX_REQUESTS PYTHON
|
|
||||||
|
|
||||||
export SEARCH_START_CONCURRENCY=16
|
|
||||||
export SEARCH_ADDEND=16
|
|
||||||
|
|
||||||
adaptive_main "$@"
|
|
||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -512,7 +512,7 @@ run_parallel_config() {
|
|||||||
|
|
||||||
# Parse results.
|
# Parse results.
|
||||||
log "parsing ${config_label} results"
|
log "parsing ${config_label} results"
|
||||||
"$PYTHON" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
"$PYTHON" "${SCRIPT_DIR}/../../../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
||||||
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -95,7 +95,7 @@ run_warmup() {
|
|||||||
num=1
|
num=1
|
||||||
|
|
||||||
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=${num})"
|
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=${num})"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/warmup.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/warmup.py" \
|
||||||
--backend "$backend" \
|
--backend "$backend" \
|
||||||
--port "$port" \
|
--port "$port" \
|
||||||
--input-len "$input_len" \
|
--input-len "$input_len" \
|
||||||
@ -189,7 +189,7 @@ parse_backend() {
|
|||||||
local backend="$1"
|
local backend="$1"
|
||||||
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
||||||
log "parsing ${backend} results in ${result_root}"
|
log "parsing ${backend} results in ${result_root}"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
||||||
>> "${result_root}/logs/parse.log" 2>&1 || {
|
>> "${result_root}/logs/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
||||||
}
|
}
|
||||||
@ -264,7 +264,7 @@ parse_backend vllm
|
|||||||
|
|
||||||
# Generate comparison.
|
# Generate comparison.
|
||||||
log "generating comparison report"
|
log "generating comparison report"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/compare.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/compare.py" \
|
||||||
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
||||||
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
||||||
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -95,7 +95,7 @@ run_warmup() {
|
|||||||
num=1
|
num=1
|
||||||
|
|
||||||
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=${num})"
|
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=${num})"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/warmup.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/warmup.py" \
|
||||||
--backend "$backend" \
|
--backend "$backend" \
|
||||||
--port "$port" \
|
--port "$port" \
|
||||||
--input-len "$input_len" \
|
--input-len "$input_len" \
|
||||||
@ -189,7 +189,7 @@ parse_backend() {
|
|||||||
local backend="$1"
|
local backend="$1"
|
||||||
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
||||||
log "parsing ${backend} results in ${result_root}"
|
log "parsing ${backend} results in ${result_root}"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
||||||
>> "${result_root}/logs/parse.log" 2>&1 || {
|
>> "${result_root}/logs/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
||||||
}
|
}
|
||||||
@ -264,7 +264,7 @@ parse_backend vllm
|
|||||||
|
|
||||||
# Generate comparison.
|
# Generate comparison.
|
||||||
log "generating comparison report"
|
log "generating comparison report"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/compare.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/compare.py" \
|
||||||
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
||||||
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
||||||
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
||||||
|
|||||||
@ -9,9 +9,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -107,7 +107,7 @@ run_warmup() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=1)"
|
log "warming up ${backend} (input=${input_len}, output=${output_len}, num=1)"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/warmup.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/warmup.py" \
|
||||||
--backend vllm \
|
--backend vllm \
|
||||||
--port "$port" \
|
--port "$port" \
|
||||||
--input-len "$input_len" \
|
--input-len "$input_len" \
|
||||||
@ -235,7 +235,7 @@ parse_backend() {
|
|||||||
parser_backend="vllm"
|
parser_backend="vllm"
|
||||||
fi
|
fi
|
||||||
log "parsing ${backend} results in ${result_root} (parser_backend=${parser_backend})"
|
log "parsing ${backend} results in ${result_root} (parser_backend=${parser_backend})"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend "$parser_backend" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend "$parser_backend" \
|
||||||
>> "${result_root}/logs/parse.log" 2>&1 || {
|
>> "${result_root}/logs/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="sglang"
|
ENGINE="sglang"
|
||||||
ENGINE_PORT="$SGLANG_PORT"
|
ENGINE_PORT="$SGLANG_PORT"
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="sglang"
|
ENGINE="sglang"
|
||||||
ENGINE_PORT="$SGLANG_PORT"
|
ENGINE_PORT="$SGLANG_PORT"
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -522,7 +522,7 @@ run_parallel_config() {
|
|||||||
|
|
||||||
# Parse results.
|
# Parse results.
|
||||||
log "parsing ${config_label} results"
|
log "parsing ${config_label} results"
|
||||||
"$PYTHON" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend sglang \
|
"$PYTHON" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend sglang \
|
||||||
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -100,7 +100,7 @@ run_warmup() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
log "warming up ${backend} (phase=${phase}, input=${input_len}, output=${output_len}, num=${num})"
|
log "warming up ${backend} (phase=${phase}, input=${input_len}, output=${output_len}, num=${num})"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/warmup.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/warmup.py" \
|
||||||
--backend "$backend" \
|
--backend "$backend" \
|
||||||
--port "$port" \
|
--port "$port" \
|
||||||
--input-len "$input_len" \
|
--input-len "$input_len" \
|
||||||
@ -264,7 +264,7 @@ parse_backend() {
|
|||||||
local backend="$1"
|
local backend="$1"
|
||||||
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
local result_root="${RESULT_BASE}/${RUN_ID}/${backend}"
|
||||||
log "parsing ${backend} results in ${result_root}"
|
log "parsing ${backend} results in ${result_root}"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend "$backend" \
|
||||||
>> "${result_root}/logs/parse.log" 2>&1 || {
|
>> "${result_root}/logs/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
log "WARNING: parser failed for ${backend}; see ${result_root}/logs/parse.log"
|
||||||
}
|
}
|
||||||
@ -348,7 +348,7 @@ parse_backend vllm
|
|||||||
|
|
||||||
# Generate comparison.
|
# Generate comparison.
|
||||||
log "generating comparison report"
|
log "generating comparison report"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/compare.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/compare.py" \
|
||||||
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
--sglang "${RESULT_BASE}/${RUN_ID}/sglang" \
|
||||||
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
--vllm "${RESULT_BASE}/${RUN_ID}/vllm" \
|
||||||
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
--output "${RESULT_BASE}/${RUN_ID}/comparison.md" \
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -7,9 +7,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -68,7 +68,7 @@ start_server() {
|
|||||||
|
|
||||||
run_warmup() {
|
run_warmup() {
|
||||||
log "warming up mtp (input=4000, output=512, num=2)"
|
log "warming up mtp (input=4000, output=512, num=2)"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/warmup.py" \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/warmup.py" \
|
||||||
--backend vllm \
|
--backend vllm \
|
||||||
--port "$MTP_PORT" \
|
--port "$MTP_PORT" \
|
||||||
--input-len 4000 \
|
--input-len 4000 \
|
||||||
@ -153,7 +153,7 @@ run_benchmark() {
|
|||||||
parse_mtp() {
|
parse_mtp() {
|
||||||
local result_root="${RESULT_BASE}/${RUN_ID}/mtp"
|
local result_root="${RESULT_BASE}/${RUN_ID}/mtp"
|
||||||
log "parsing mtp results in ${result_root}"
|
log "parsing mtp results in ${result_root}"
|
||||||
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
"${VENV_CLIENT}/bin/python" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
||||||
>> "${result_root}/logs/parse.log" 2>&1 || {
|
>> "${result_root}/logs/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for mtp; see ${result_root}/logs/parse.log"
|
log "WARNING: parser failed for mtp; see ${result_root}/logs/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -15,9 +15,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -15,9 +15,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -14,9 +14,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="vllm"
|
ENGINE="vllm"
|
||||||
ENGINE_PORT="$VLLM_PORT"
|
ENGINE_PORT="$VLLM_PORT"
|
||||||
|
|||||||
@ -6,15 +6,15 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/adaptive_config.env"
|
source "${SCRIPT_DIR}/adaptive_config.env"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/adaptive_bench_lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/adaptive_bench_lib.sh"
|
||||||
|
|
||||||
ENGINE="vllm"
|
ENGINE="vllm"
|
||||||
ENGINE_PORT="$VLLM_PORT"
|
ENGINE_PORT="$VLLM_PORT"
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -512,7 +512,7 @@ run_parallel_config() {
|
|||||||
|
|
||||||
# Parse results.
|
# Parse results.
|
||||||
log "parsing ${config_label} results"
|
log "parsing ${config_label} results"
|
||||||
"$PYTHON" "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
"$PYTHON" "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$result_root" --backend vllm \
|
||||||
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
>> "${phase_log_dir}/parse.log" 2>&1 || {
|
||||||
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
log "WARNING: parser failed for ${config_label}; see ${phase_log_dir}/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -5,13 +5,13 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/server_docker.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/server_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/bench_client_docker.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/bench_client_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -5,7 +5,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../../scripts/common/server_docker.sh"
|
source "${SCRIPT_DIR}/../../../../scripts/common/server_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
@ -217,7 +217,7 @@ with open('${host_output_file}') as f:
|
|||||||
|
|
||||||
parse_backend() {
|
parse_backend() {
|
||||||
log "parsing results in ${RESULT_ROOT}"
|
log "parsing results in ${RESULT_ROOT}"
|
||||||
python3 "${SCRIPT_DIR}/../../scripts/common/parse_backend.py" "$RESULT_ROOT" --backend sglang \
|
python3 "${SCRIPT_DIR}/../../../scripts/common/parse_backend.py" "$RESULT_ROOT" --backend sglang \
|
||||||
>> "${LOG_DIR}/parse.log" 2>&1 || {
|
>> "${LOG_DIR}/parse.log" 2>&1 || {
|
||||||
log "WARNING: parse_backend.py failed; see ${LOG_DIR}/parse.log"
|
log "WARNING: parse_backend.py failed; see ${LOG_DIR}/parse.log"
|
||||||
}
|
}
|
||||||
|
|||||||
@ -17,9 +17,9 @@ fi
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -6,9 +6,9 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -10,9 +10,9 @@ fi
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -5,13 +5,13 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/lib.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/lib.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/platform.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/platform.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/server_docker.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/server_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/bench_client_docker.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/bench_client_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -5,7 +5,7 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|||||||
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
||||||
|
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/../../scripts/common/server_docker.sh"
|
source "${SCRIPT_DIR}/../../../scripts/common/server_docker.sh"
|
||||||
# shellcheck source=/dev/null
|
# shellcheck source=/dev/null
|
||||||
source "${SCRIPT_DIR}/config.env"
|
source "${SCRIPT_DIR}/config.env"
|
||||||
|
|
||||||
|
|||||||
@ -1 +1,121 @@
|
|||||||
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":41.2688269931823,"request_tps":1.9385091806272121,"input_tps":1985.0334009622652,"output_tps":248.12917512028315,"total_tps":2233.1625760825486,"mean_input_tokens":1024.0,"mean_output_tokens":128.0,"ttft_p50_ms":1839.1637965105474,"ttft_p95_ms":5156.00185089279,"ttft_p99_ms":5158.878551644739,"tpot_p50_ms":47.26517358684809,"tpot_p95_ms":54.75502148450271,"tpot_p99_ms":55.868444815663054,"e2e_p50_ms":7647.177290520631,"e2e_p95_ms":11683.607439068146,"e2e_p99_ms":11687.237907222006,"itl_p50_ms":38.27845409978181,"itl_p95_ms":61.400388937909156,"itl_p99_ms":83.85505519807339,"validation_errors":[],"timestamp":"2026-07-16T07:48:44+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o128_a1.log"}
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":41.2688269931823,"request_tps":1.9385091806272121,"input_tps":1985.0334009622652,"output_tps":248.12917512028315,"total_tps":2233.1625760825486,"mean_input_tokens":1024.0,"mean_output_tokens":128.0,"ttft_p50_ms":1839.1637965105474,"ttft_p95_ms":5156.00185089279,"ttft_p99_ms":5158.878551644739,"tpot_p50_ms":47.26517358684809,"tpot_p95_ms":54.75502148450271,"tpot_p99_ms":55.868444815663054,"e2e_p50_ms":7647.177290520631,"e2e_p95_ms":11683.607439068146,"e2e_p99_ms":11687.237907222006,"itl_p50_ms":38.27845409978181,"itl_p95_ms":61.400388937909156,"itl_p99_ms":83.85505519807339,"validation_errors":[],"timestamp":"2026-07-16T07:48:44+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":67.59083261387423,"request_tps":1.1835924622650464,"input_tps":1211.9986813594076,"output_tps":302.9996703398519,"total_tps":1514.9983516992595,"mean_input_tokens":1024.0,"mean_output_tokens":256.0,"ttft_p50_ms":1088.1021856330335,"ttft_p95_ms":1940.051195200067,"ttft_p99_ms":1945.7215382088907,"tpot_p50_ms":48.00720819381668,"tpot_p95_ms":51.906457407692194,"tpot_p99_ms":52.662985878384795,"e2e_p50_ms":13323.23444657959,"e2e_p95_ms":14546.717717882711,"e2e_p99_ms":14597.288672290742,"itl_p50_ms":38.109477958641946,"itl_p95_ms":62.32210020534694,"itl_p99_ms":91.81069683981994,"validation_errors":[],"timestamp":"2026-07-16T07:50:13+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":85.51100934692658,"request_tps":1.8711040978462112,"input_tps":1916.0105961945203,"output_tps":479.0026490486301,"total_tps":2395.0132452431503,"mean_input_tokens":1024.0,"mean_output_tokens":256.0,"ttft_p50_ms":1161.2384765176103,"ttft_p95_ms":3756.2352412380274,"ttft_p99_ms":3780.965418519918,"tpot_p50_ms":61.98459189412568,"tpot_p95_ms":66.44078092480663,"tpot_p99_ms":66.8051291848259,"e2e_p50_ms":16901.625977130607,"e2e_p95_ms":18837.460162944622,"e2e_p99_ms":18985.462763826363,"itl_p50_ms":44.66060339473188,"itl_p95_ms":68.57059714384377,"itl_p99_ms":449.1709415311925,"validation_errors":[],"timestamp":"2026-07-16T07:52:00+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":256,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":58.086855,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c32_i1024_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c32_i1024_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":240,"failed":0,"duration_s":102.57387904892676,"request_tps":2.3397769707580456,"input_tps":2395.9316180562387,"output_tps":598.9829045140597,"total_tps":2994.9145225702982,"mean_input_tokens":1024.0,"mean_output_tokens":256.0,"ttft_p50_ms":4857.287564082071,"ttft_p95_ms":5687.151023850311,"ttft_p99_ms":5947.1146802348085,"tpot_p50_ms":60.30845692353871,"tpot_p95_ms":72.36586667291418,"tpot_p99_ms":78.81875480012964,"e2e_p50_ms":20432.19355738256,"e2e_p95_ms":21489.17214354733,"e2e_p99_ms":22516.328958119266,"itl_p50_ms":51.60140746738762,"itl_p95_ms":73.69143945397809,"itl_p99_ms":89.00051706237718,"validation_errors":[],"timestamp":"2026-07-16T07:54:03+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":256,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":1,"gain_pct":25.047932,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c48_i1024_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c48_i1024_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":123.35641185496934,"request_tps":0.6485272941795385,"input_tps":664.0919492398474,"output_tps":332.0459746199237,"total_tps":996.137923859771,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":1815.1043795514852,"ttft_p95_ms":2051.647847087588,"ttft_p99_ms":2054.4600446219556,"tpot_p50_ms":44.89284182270832,"tpot_p95_ms":47.29539701357892,"tpot_p99_ms":47.49259979092314,"e2e_p50_ms":24692.195138079114,"e2e_p95_ms":25352.763180364855,"e2e_p99_ms":25618.341889507137,"itl_p50_ms":37.906087934970856,"itl_p95_ms":61.653098394162946,"itl_p99_ms":66.99016668368131,"validation_errors":[],"timestamp":"2026-07-16T07:56:28+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":151.8317316111643,"request_tps":1.0537981639421354,"input_tps":1079.0893198767467,"output_tps":539.5446599383733,"total_tps":1618.63397981512,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":1056.6700529307127,"ttft_p95_ms":3730.0430950359428,"ttft_p99_ms":3803.3011273038574,"tpot_p50_ms":55.7243038511538,"tpot_p95_ms":60.350505806735306,"tpot_p99_ms":61.641385694806246,"e2e_p50_ms":29571.674760896713,"e2e_p95_ms":32270.35216266522,"e2e_p99_ms":32279.17558920337,"itl_p50_ms":45.009728986769915,"itl_p95_ms":67.9298322298564,"itl_p99_ms":236.92634512903186,"validation_errors":[],"timestamp":"2026-07-16T07:59:21+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":512,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":62.49095,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c32_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c32_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":240,"failed":0,"duration_s":194.6086190820206,"request_tps":1.2332444530570794,"input_tps":1262.8423199304493,"output_tps":631.4211599652247,"total_tps":1894.263479895674,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":5219.216491095722,"ttft_p95_ms":5779.679291823413,"ttft_p99_ms":6389.283453426324,"tpot_p50_ms":58.08764942142558,"tpot_p95_ms":62.69429164324429,"tpot_p99_ms":91.06430042089896,"e2e_p50_ms":34896.657894947566,"e2e_p95_ms":35910.429730673786,"e2e_p99_ms":49593.8836463075,"itl_p50_ms":51.571647985838354,"itl_p95_ms":73.75171015737578,"itl_p99_ms":77.6099856174551,"validation_errors":[],"timestamp":"2026-07-16T08:02:57+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":512,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":1,"gain_pct":17.028526,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c48_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c48_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":239.2404650580138,"request_tps":0.3343915920770371,"input_tps":342.416990286886,"output_tps":342.416990286886,"total_tps":684.833980573772,"mean_input_tokens":1024.0,"mean_output_tokens":1024.0,"ttft_p50_ms":635.5562334647402,"ttft_p95_ms":1903.8669666741043,"ttft_p99_ms":1906.7863759305328,"tpot_p50_ms":45.73389088913875,"tpot_p95_ms":46.81248970833659,"tpot_p99_ms":47.236905306400615,"e2e_p50_ms":47589.559288928285,"e2e_p95_ms":48763.24818015564,"e2e_p99_ms":49025.48799880082,"itl_p50_ms":38.22040546219796,"itl_p95_ms":62.13192297145724,"itl_p99_ms":69.20204504393043,"validation_errors":[],"timestamp":"2026-07-16T08:07:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":283.1469709521625,"request_tps":0.5650775618822773,"input_tps":578.639423367452,"output_tps":578.639423367452,"total_tps":1157.278846734904,"mean_input_tokens":1024.0,"mean_output_tokens":1024.0,"ttft_p50_ms":1039.5908689824864,"ttft_p95_ms":3708.7073334376323,"ttft_p99_ms":3736.5557909896597,"tpot_p50_ms":53.18489079777869,"tpot_p95_ms":56.531733996098275,"tpot_p99_ms":56.90662554211692,"e2e_p50_ms":55562.747771968134,"e2e_p95_ms":58622.54847800359,"e2e_p99_ms":58864.768436276354,"itl_p50_ms":44.60177046712488,"itl_p95_ms":68.54137406917289,"itl_p99_ms":104.98853123513977,"validation_errors":[],"timestamp":"2026-07-16T08:12:22+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":1024,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":68.986773,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c32_i1024_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c32_i1024_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":240,"failed":0,"duration_s":386.91459054895677,"request_tps":0.6202919348672961,"input_tps":635.1789413041112,"output_tps":635.1789413041112,"total_tps":1270.3578826082223,"mean_input_tokens":1024.0,"mean_output_tokens":1024.0,"ttft_p50_ms":5370.0150235090405,"ttft_p95_ms":15133.30938139696,"ttft_p99_ms":32185.267814283718,"tpot_p50_ms":60.0038268968676,"tpot_p95_ms":91.36051824342555,"tpot_p99_ms":103.14346713503731,"e2e_p50_ms":65575.55996044539,"e2e_p95_ms":99691.6358396411,"e2e_p99_ms":112446.33609429697,"itl_p50_ms":51.48644756991416,"itl_p95_ms":73.74423729488626,"itl_p99_ms":83.97575234295799,"validation_errors":[],"timestamp":"2026-07-16T08:19:12+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":1024,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":1,"gain_pct":9.771114,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c48_i1024_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c48_i1024_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":473.66198480315506,"request_tps":0.16889681369140588,"input_tps":172.95033721999962,"output_tps":345.90067443999925,"total_tps":518.8510116599988,"mean_input_tokens":1024.0,"mean_output_tokens":2048.0,"ttft_p50_ms":1107.0592959877104,"ttft_p95_ms":1914.7031036205587,"ttft_p99_ms":1937.422300640028,"tpot_p50_ms":45.60600052494192,"tpot_p95_ms":46.180353910758,"tpot_p99_ms":46.185254772231495,"e2e_p50_ms":94571.73220254481,"e2e_p95_ms":95557.28795382893,"e2e_p99_ms":95606.2740921299,"itl_p50_ms":38.73952745925635,"itl_p95_ms":62.4737826641649,"itl_p99_ms":67.41200149524958,"validation_errors":[],"timestamp":"2026-07-16T08:27:27+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":562.1656430249568,"request_tps":0.28461362231077675,"input_tps":291.4443492462354,"output_tps":582.8886984924708,"total_tps":874.3330477387061,"mean_input_tokens":1024.0,"mean_output_tokens":2048.0,"ttft_p50_ms":1720.5767229897901,"ttft_p95_ms":3765.552905423101,"ttft_p99_ms":3778.2479317323305,"tpot_p50_ms":53.25908665974094,"tpot_p95_ms":55.20441076613287,"tpot_p99_ms":55.39557766710268,"e2e_p50_ms":109979.57844298799,"e2e_p95_ms":114323.36549273459,"e2e_p99_ms":114707.30229713488,"itl_p50_ms":46.27890046685934,"itl_p95_ms":69.01592136127874,"itl_p99_ms":77.31038637226447,"validation_errors":[],"timestamp":"2026-07-16T08:37:12+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":2048,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":68.513317,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c32_i1024_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c32_i1024_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":240,"failed":0,"duration_s":785.6881854759995,"request_tps":0.3054646925288802,"input_tps":312.7958451495733,"output_tps":625.5916902991466,"total_tps":938.3875354487201,"mean_input_tokens":1024.0,"mean_output_tokens":2048.0,"ttft_p50_ms":5525.519606540911,"ttft_p95_ms":38321.421813906585,"ttft_p99_ms":61896.90983353647,"tpot_p50_ms":60.786382354451064,"tpot_p95_ms":94.29399440951978,"tpot_p99_ms":106.96621055514711,"e2e_p50_ms":128179.90333400667,"e2e_p95_ms":211186.33220964108,"e2e_p99_ms":235519.09909247872,"itl_p50_ms":52.96039255335927,"itl_p95_ms":74.77014367468655,"itl_p99_ms":93.00403241533787,"validation_errors":[],"timestamp":"2026-07-16T08:50:41+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":2048,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":1,"gain_pct":7.326097,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c48_i1024_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c48_i1024_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":948.3546966928989,"request_tps":0.08435662340153519,"input_tps":86.38118236317203,"output_tps":345.5247294526881,"total_tps":431.9059118158602,"mean_input_tokens":1024.0,"mean_output_tokens":4096.0,"ttft_p50_ms":587.1013974538073,"ttft_p95_ms":2000.1368144294247,"ttft_p99_ms":2003.0350286723112,"tpot_p50_ms":45.94598221112161,"tpot_p95_ms":46.736607726203374,"tpot_p99_ms":46.843049427122,"e2e_p50_ms":188973.83886645548,"e2e_p95_ms":191972.2936287988,"e2e_p99_ms":192374.7143190843,"itl_p50_ms":39.03631155844778,"itl_p95_ms":63.36234454065562,"itl_p99_ms":68.28304521273822,"validation_errors":[],"timestamp":"2026-07-16T09:06:52+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":4096,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":840.8592580729164,"request_tps":0.09514077324110604,"input_tps":97.42415179889258,"output_tps":389.6966071955703,"total_tps":487.1207589944629,"mean_input_tokens":1024.0,"mean_output_tokens":4096.0,"ttft_p50_ms":972.442637081258,"ttft_p95_ms":5176.270531245973,"ttft_p99_ms":5179.259422505274,"tpot_p50_ms":40.55560706117929,"tpot_p95_ms":40.93768200460964,"tpot_p99_ms":40.989130662076434,"e2e_p50_ms":167707.99649693072,"e2e_p95_ms":171209.72607890144,"e2e_p99_ms":171213.10957846232,"itl_p50_ms":38.21910894475877,"itl_p95_ms":56.424582388717674,"itl_p99_ms":61.033719554543495,"validation_errors":[],"timestamp":"2026-07-16T09:31:28+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":4096,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i1024_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i1024_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":750.264118807856,"request_tps":0.0533145581632754,"input_tps":54.59410755919401,"output_tps":218.37643023677603,"total_tps":272.97053779597,"mean_input_tokens":1024.0,"mean_output_tokens":4096.0,"ttft_p50_ms":457.8002068446949,"ttft_p95_ms":1004.7794182202779,"ttft_p99_ms":1006.1946984101087,"tpot_p50_ms":36.49401828900812,"tpot_p95_ms":36.67125693285014,"tpot_p99_ms":36.679555167656694,"e2e_p50_ms":150253.00148909446,"e2e_p95_ms":150511.7817995837,"e2e_p99_ms":150794.05072218738,"itl_p50_ms":34.35514506418258,"itl_p95_ms":53.22418445721268,"itl_p99_ms":59.324696937110275,"validation_errors":[],"timestamp":"2026-07-16T09:44:20+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":4096,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i1024_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i1024_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":64.04329815506935,"request_tps":1.2491549046442667,"input_tps":5116.538489422916,"output_tps":159.89182779446614,"total_tps":5276.430317217383,"mean_input_tokens":4096.0,"mean_output_tokens":128.0,"ttft_p50_ms":4576.742604491301,"ttft_p95_ms":7486.383566202128,"ttft_p99_ms":8069.572048275731,"tpot_p50_ms":63.25949257419364,"tpot_p95_ms":88.51012927140879,"tpot_p99_ms":93.78976958789158,"e2e_p50_ms":12597.990370937623,"e2e_p95_ms":16119.662079052066,"e2e_p99_ms":16428.539164408576,"itl_p50_ms":38.49607554730028,"itl_p95_ms":59.04390099458394,"itl_p99_ms":859.1562826023437,"validation_errors":[],"timestamp":"2026-07-16T09:45:45+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":42.330964882159606,"request_tps":0.9449347566574843,"input_tps":3870.4527632690556,"output_tps":120.95164885215799,"total_tps":3991.4044121212137,"mean_input_tokens":4096.0,"mean_output_tokens":128.0,"ttft_p50_ms":2789.9403674528003,"ttft_p95_ms":3850.861389178317,"ttft_p99_ms":3857.0726825506426,"tpot_p50_ms":44.318099374200884,"tpot_p95_ms":52.61084187132357,"tpot_p99_ms":56.91049005684083,"e2e_p50_ms":8397.533455048688,"e2e_p95_ms":10301.826596679166,"e2e_p99_ms":10350.095471963286,"itl_p50_ms":34.42738042213023,"itl_p95_ms":54.41078649600968,"itl_p99_ms":108.43641549814492,"validation_errors":[],"timestamp":"2026-07-16T09:46:49+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":95.2746979680378,"request_tps":0.839677287949398,"input_tps":3439.318171440734,"output_tps":214.9573857150459,"total_tps":3654.27555715578,"mean_input_tokens":4096.0,"mean_output_tokens":256.0,"ttft_p50_ms":4683.939669514075,"ttft_p95_ms":8062.7326244954,"ttft_p99_ms":8673.460278843995,"tpot_p50_ms":51.88298721149491,"tpot_p95_ms":64.92892745028561,"tpot_p99_ms":75.71912046924533,"e2e_p50_ms":17954.677841393277,"e2e_p95_ms":22523.24340294581,"e2e_p99_ms":24458.481649178953,"itl_p50_ms":38.46407646778971,"itl_p95_ms":59.771430026739836,"itl_p99_ms":67.52078937599435,"validation_errors":[],"timestamp":"2026-07-16T09:48:46+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":65.8774013610091,"request_tps":0.6071884921628803,"input_tps":2487.0440638991577,"output_tps":155.44025399369735,"total_tps":2642.4843178928554,"mean_input_tokens":4096.0,"mean_output_tokens":256.0,"ttft_p50_ms":1971.202465472743,"ttft_p95_ms":3824.870833219029,"ttft_p99_ms":3841.111220801249,"tpot_p50_ms":43.596625505654394,"tpot_p95_ms":48.1240840446131,"tpot_p99_ms":49.99360711309218,"e2e_p50_ms":13162.176689947955,"e2e_p95_ms":14974.834044638555,"e2e_p99_ms":14991.584688341245,"itl_p50_ms":34.47401651646942,"itl_p95_ms":54.485111916437745,"itl_p99_ms":60.43242933228612,"validation_errors":[],"timestamp":"2026-07-16T09:50:14+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":160.08099704002962,"request_tps":0.4997470123202401,"input_tps":2046.9637624637035,"output_tps":255.87047030796293,"total_tps":2302.8342327716664,"mean_input_tokens":4096.0,"mean_output_tokens":512.0,"ttft_p50_ms":5474.024421186186,"ttft_p95_ms":8615.011906716973,"ttft_p99_ms":8951.299454767715,"tpot_p50_ms":46.494569923772445,"tpot_p95_ms":53.43938896643479,"tpot_p99_ms":76.29711307971617,"e2e_p50_ms":28335.586922592483,"e2e_p95_ms":32704.220330703534,"e2e_p99_ms":44918.695579937164,"itl_p50_ms":38.48353505600244,"itl_p95_ms":59.288766619283706,"itl_p99_ms":65.03905458841473,"validation_errors":[],"timestamp":"2026-07-16T09:53:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":112.8795186879579,"request_tps":0.3543601218798184,"input_tps":1451.4590592197362,"output_tps":181.43238240246703,"total_tps":1632.8914416222033,"mean_input_tokens":4096.0,"mean_output_tokens":512.0,"ttft_p50_ms":1941.342479432933,"ttft_p95_ms":3813.8611613423564,"ttft_p99_ms":3815.337903073523,"tpot_p50_ms":40.17744154405986,"tpot_p95_ms":42.320974605816815,"tpot_p99_ms":43.39261491248045,"e2e_p50_ms":22431.270663044415,"e2e_p95_ms":24335.910513892304,"e2e_p99_ms":24337.53361347597,"itl_p50_ms":34.43968354258686,"itl_p95_ms":54.208886146079735,"itl_p99_ms":59.69965270254761,"validation_errors":[],"timestamp":"2026-07-16T09:55:31+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":282.66458361805417,"request_tps":0.2830209535839788,"input_tps":1159.253825879977,"output_tps":289.81345646999426,"total_tps":1449.0672823499715,"mean_input_tokens":4096.0,"mean_output_tokens":1024.0,"ttft_p50_ms":4658.318318077363,"ttft_p95_ms":10095.44171601301,"ttft_p99_ms":12803.387446557112,"tpot_p50_ms":44.42285834210774,"tpot_p95_ms":49.33753012539997,"tpot_p99_ms":79.11847314414067,"e2e_p50_ms":49335.88792406954,"e2e_p95_ms":58133.938353240934,"e2e_p99_ms":87081.42475777536,"itl_p50_ms":38.47858391236514,"itl_p95_ms":57.68822985701263,"itl_p99_ms":62.88705667946487,"validation_errors":[],"timestamp":"2026-07-16T10:00:35+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":207.04861099808477,"request_tps":0.1931913467430603,"input_tps":791.311756259575,"output_tps":197.82793906489374,"total_tps":989.1396953244687,"mean_input_tokens":4096.0,"mean_output_tokens":1024.0,"ttft_p50_ms":1074.1113065741956,"ttft_p95_ms":3878.6458508693613,"ttft_p99_ms":3916.904898113571,"tpot_p50_ms":39.24930339234399,"tpot_p95_ms":40.05846651063872,"tpot_p99_ms":40.82051611248493,"e2e_p50_ms":41186.332597979344,"e2e_p95_ms":43346.47120544687,"e2e_p99_ms":43701.94057547487,"itl_p50_ms":34.467250341549516,"itl_p95_ms":54.143763531465076,"itl_p99_ms":59.957590696867555,"validation_errors":[],"timestamp":"2026-07-16T10:04:24+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":534.5376915351953,"request_tps":0.14966203743320614,"input_tps":613.0157053264123,"output_tps":306.50785266320617,"total_tps":919.5235579896186,"mean_input_tokens":4096.0,"mean_output_tokens":2048.0,"ttft_p50_ms":5104.571636882611,"ttft_p95_ms":13598.845852899827,"ttft_p99_ms":15530.32725170254,"tpot_p50_ms":42.516595378821116,"tpot_p95_ms":48.41014748285291,"tpot_p99_ms":80.66643662295293,"e2e_p50_ms":90892.7560900338,"e2e_p95_ms":106289.35107835554,"e2e_p99_ms":171094.83309885245,"itl_p50_ms":38.54017308913171,"itl_p95_ms":58.33887221524492,"itl_p99_ms":64.16293417569251,"validation_errors":[],"timestamp":"2026-07-16T10:13:42+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":394.49152314290404,"request_tps":0.10139634859913087,"input_tps":415.31944386204003,"output_tps":207.65972193102002,"total_tps":622.97916579306,"mean_input_tokens":4096.0,"mean_output_tokens":2048.0,"ttft_p50_ms":1034.7788120852783,"ttft_p95_ms":3756.3848297810177,"ttft_p99_ms":3805.5001059989445,"tpot_p50_ms":37.88820180411141,"tpot_p95_ms":38.25126371524308,"tpot_p99_ms":38.27709748982381,"e2e_p50_ms":78674.81265543029,"e2e_p95_ms":80982.15972669423,"e2e_p99_ms":80984.67170314165,"itl_p50_ms":34.52226798981428,"itl_p95_ms":53.908159665297724,"itl_p99_ms":59.665183436591185,"validation_errors":[],"timestamp":"2026-07-16T10:20:38+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":2048,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1029.577203406021,"request_tps":0.07770179811222125,"input_tps":318.26656506765823,"output_tps":318.26656506765823,"total_tps":636.5331301353165,"mean_input_tokens":4096.0,"mean_output_tokens":4096.0,"ttft_p50_ms":5483.361787861213,"ttft_p95_ms":22374.68960867263,"ttft_p99_ms":150920.9240530175,"tpot_p50_ms":41.06715219183478,"tpot_p95_ms":46.9445546033481,"tpot_p99_ms":79.17070015191274,"e2e_p50_ms":173329.3942649616,"e2e_p95_ms":302973.24356382014,"e2e_p99_ms":330154.8701896191,"itl_p50_ms":38.36022887844592,"itl_p95_ms":57.20986550441011,"itl_p99_ms":62.143948464654386,"validation_errors":[],"timestamp":"2026-07-16T10:38:11+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":4096,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i4096_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i4096_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":762.6022247918881,"request_tps":0.052451984402374224,"input_tps":214.84332811212482,"output_tps":214.84332811212482,"total_tps":429.68665622424965,"mean_input_tokens":4096.0,"mean_output_tokens":4096.0,"ttft_p50_ms":1038.2880391553044,"ttft_p95_ms":3893.24660198763,"ttft_p99_ms":3900.2761626965366,"tpot_p50_ms":36.76516193150564,"tpot_p95_ms":37.05582632169512,"tpot_p99_ms":37.09407625586006,"e2e_p50_ms":152243.55608585756,"e2e_p95_ms":154084.0833232156,"e2e_p99_ms":154084.53153002774,"itl_p50_ms":34.463146002963185,"itl_p95_ms":51.49366667028517,"itl_p99_ms":59.961400236934416,"validation_errors":[],"timestamp":"2026-07-16T10:51:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":4096,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i4096_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i4096_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":214.8227549989242,"request_tps":0.37240002810875705,"input_tps":6101.402060533876,"output_tps":47.6672035979209,"total_tps":6149.069264131796,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":33368.671427015215,"ttft_p95_ms":41899.411271174904,"ttft_p99_ms":42125.512621703085,"tpot_p50_ms":52.554487114163535,"tpot_p95_ms":84.39758658330287,"tpot_p99_ms":111.94620711651932,"e2e_p50_ms":42495.81566103734,"e2e_p95_ms":49210.346029268105,"e2e_p99_ms":52561.92248010076,"itl_p50_ms":34.52568908687681,"itl_p95_ms":52.17893534572764,"itl_p99_ms":66.4443611598108,"validation_errors":[],"timestamp":"2026-07-16T10:55:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":109.44023031182587,"request_tps":0.36549630685195744,"input_tps":5988.291491462471,"output_tps":46.78352727705055,"total_tps":6035.075018739521,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":11529.161532991566,"ttft_p95_ms":21725.308789138217,"ttft_p99_ms":21941.029288116843,"tpot_p50_ms":76.67213628768891,"tpot_p95_ms":101.16250104450535,"tpot_p99_ms":138.8094991766004,"e2e_p50_ms":16583.38130300399,"e2e_p95_ms":32255.35403857939,"e2e_p99_ms":37416.11094208899,"itl_p50_ms":34.63044541422278,"itl_p95_ms":54.926601552870125,"itl_p99_ms":1767.2955490718596,"validation_errors":[],"timestamp":"2026-07-16T10:57:31+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":39.83333700802177,"request_tps":0.12552300097260452,"input_tps":2056.5688479351525,"output_tps":16.06694412449338,"total_tps":2072.6357920596456,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":3952.2010260261595,"ttft_p95_ms":4079.302547173574,"ttft_p99_ms":4102.609633402899,"tpot_p50_ms":31.602383598567936,"tpot_p95_ms":31.640485883140425,"tpot_p99_ms":31.642765031291507,"e2e_p50_ms":7969.095474109054,"e2e_p95_ms":8074.662221223116,"e2e_p99_ms":8093.300537876784,"itl_p50_ms":29.8505169339478,"itl_p95_ms":49.705708399415016,"itl_p99_ms":51.2071222346276,"validation_errors":[],"timestamp":"2026-07-16T10:58:35+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":270.44398706802167,"request_tps":0.2958098675711304,"input_tps":4846.548870285401,"output_tps":75.72732609820939,"total_tps":4922.27619638361,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":38130.55603799876,"ttft_p95_ms":51039.480149233714,"ttft_p99_ms":51170.22580615245,"tpot_p50_ms":44.34177505135463,"tpot_p95_ms":60.19466479852174,"tpot_p99_ms":74.01323430483028,"e2e_p50_ms":50331.59309858456,"e2e_p95_ms":65593.75085806241,"e2e_p99_ms":65883.99838647107,"itl_p50_ms":34.52300652861595,"itl_p95_ms":51.78628868889062,"itl_p99_ms":60.336364856921136,"validation_errors":[],"timestamp":"2026-07-16T11:03:31+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":141.670446844073,"request_tps":0.28234540718308926,"input_tps":4625.947151287734,"output_tps":72.28042423887085,"total_tps":4698.2275755266055,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":11554.360230453312,"ttft_p95_ms":26391.11408374738,"ttft_p99_ms":26587.14834881015,"tpot_p50_ms":56.31336195065695,"tpot_p95_ms":68.25714148759073,"tpot_p99_ms":87.6372731319245,"e2e_p50_ms":21265.938312513754,"e2e_p95_ms":41577.943935687654,"e2e_p99_ms":46795.674697651986,"itl_p50_ms":34.58584542386234,"itl_p95_ms":52.59093521162865,"itl_p99_ms":61.120713129639775,"validation_errors":[],"timestamp":"2026-07-16T11:06:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":60.537773119984195,"request_tps":0.08259306119652168,"input_tps":1353.2047146438113,"output_tps":21.14382366630955,"total_tps":1374.3485383101208,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":3945.7851310726255,"ttft_p95_ms":4093.810023879632,"ttft_p99_ms":4111.218916652724,"tpot_p50_ms":31.821994341033346,"tpot_p95_ms":32.01389185737307,"tpot_p99_ms":32.03676525156433,"e2e_p50_ms":12080.163977108896,"e2e_p95_ms":12253.15336487256,"e2e_p99_ms":12279.754239274189,"itl_p50_ms":29.860365903005004,"itl_p95_ms":48.98134677205235,"itl_p99_ms":55.7887158729136,"validation_errors":[],"timestamp":"2026-07-16T11:07:42+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":256,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":390.9635538929142,"request_tps":0.20462265396204216,"input_tps":3352.5375625140987,"output_tps":104.76679882856558,"total_tps":3457.304361342664,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":52149.89124494605,"ttft_p95_ms":69626.60835890565,"ttft_p99_ms":69753.87210340472,"tpot_p50_ms":40.293584739718284,"tpot_p95_ms":48.141768489123194,"tpot_p99_ms":54.960999020335215,"e2e_p50_ms":73592.7937864326,"e2e_p95_ms":93283.15660590306,"e2e_p99_ms":93629.28111733867,"itl_p50_ms":34.522912465035915,"itl_p95_ms":51.489772961940616,"itl_p99_ms":59.926832308992736,"validation_errors":[],"timestamp":"2026-07-16T11:14:38+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":206.38675206596963,"request_tps":0.19381088950522546,"input_tps":3175.397613653614,"output_tps":99.23117542667543,"total_tps":3274.628789080289,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":11589.022769010626,"ttft_p95_ms":35687.3680736986,"ttft_p99_ms":35902.96996307094,"tpot_p50_ms":46.31837186974942,"tpot_p95_ms":52.744000344951814,"tpot_p99_ms":61.98560190418053,"e2e_p50_ms":30626.526765991002,"e2e_p95_ms":60265.1604500832,"e2e_p99_ms":65432.796396943726,"itl_p50_ms":34.61834159679711,"itl_p95_ms":51.886969921179116,"itl_p99_ms":60.28145674150437,"validation_errors":[],"timestamp":"2026-07-16T11:18:29+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":100.54988616588525,"request_tps":0.04972656052291394,"input_tps":814.719967607422,"output_tps":25.459998987731936,"total_tps":840.1799665951539,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":3932.124942075461,"ttft_p95_ms":4127.591159660369,"ttft_p99_ms":4166.129624564201,"tpot_p50_ms":31.635086914185443,"tpot_p95_ms":31.776042928674975,"tpot_p99_ms":31.788027078066378,"e2e_p50_ms":20141.837269999087,"e2e_p95_ms":20233.498825039715,"e2e_p99_ms":20244.73103923723,"itl_p50_ms":29.790908098220825,"itl_p95_ms":50.02487420570105,"itl_p99_ms":54.69925904646516,"validation_errors":[],"timestamp":"2026-07-16T11:20:34+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":512,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":632.0467019469943,"request_tps":0.1265729253132138,"input_tps":2073.7708083316948,"output_tps":129.61067552073092,"total_tps":2203.3814838524254,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":80126.8644919619,"ttft_p95_ms":106781.9698160165,"ttft_p99_ms":107180.66528041381,"tpot_p50_ms":38.308278328050854,"tpot_p95_ms":42.30265493578899,"tpot_p99_ms":45.4902937474118,"e2e_p50_ms":120186.66901800316,"e2e_p95_ms":148021.29719536752,"e2e_p99_ms":148871.62709633118,"itl_p50_ms":34.52094295062125,"itl_p95_ms":51.694627082906685,"itl_p99_ms":59.966907631605864,"validation_errors":[],"timestamp":"2026-07-16T11:31:32+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":335.39274771884084,"request_tps":0.11926316317826863,"input_tps":1954.0076655127532,"output_tps":122.12547909454707,"total_tps":2076.1331446073004,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":11495.135091943666,"ttft_p95_ms":54293.259906279854,"ttft_p99_ms":54475.75780400075,"tpot_p50_ms":41.142632146611746,"tpot_p95_ms":44.4825864581911,"tpot_p99_ms":48.84856583131568,"e2e_p50_ms":49175.88768794667,"e2e_p95_ms":97491.41289745457,"e2e_p99_ms":102364.25640797475,"itl_p50_ms":34.5357779879123,"itl_p95_ms":52.090133994352065,"itl_p99_ms":60.2705589751713,"validation_errors":[],"timestamp":"2026-07-16T11:37:32+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":181.57242772518657,"request_tps":0.02753722061571814,"input_tps":451.169822567926,"output_tps":28.198113910495376,"total_tps":479.3679364784214,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":3940.7668448984623,"ttft_p95_ms":4193.481011828408,"ttft_p99_ms":4228.460800005123,"tpot_p50_ms":31.63807294540879,"tpot_p95_ms":31.730153837458815,"tpot_p99_ms":31.734481836830287,"e2e_p50_ms":36320.72346494533,"e2e_p95_ms":36463.93566392362,"e2e_p99_ms":36480.87247610092,"itl_p50_ms":29.840212780982256,"itl_p95_ms":46.52133162599058,"itl_p99_ms":54.09107577055689,"validation_errors":[],"timestamp":"2026-07-16T11:40:58+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":1024,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1115.2481476869434,"request_tps":0.07173291447820136,"input_tps":1175.2720708108511,"output_tps":146.9090088513564,"total_tps":1322.1810796622076,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":136037.42311848328,"ttft_p95_ms":181299.1330652847,"ttft_p99_ms":181444.37291203765,"tpot_p50_ms":37.30771111527574,"tpot_p95_ms":39.328867612731,"tpot_p99_ms":40.83525040460833,"e2e_p50_ms":213226.2396500446,"e2e_p95_ms":259703.93927409317,"e2e_p99_ms":260337.14725418712,"itl_p50_ms":34.49960285797715,"itl_p95_ms":51.94193584611639,"itl_p99_ms":60.15488663455472,"validation_errors":[],"timestamp":"2026-07-16T12:00:00+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":584.8935852469876,"request_tps":0.06838850862607578,"input_tps":1120.4773253296255,"output_tps":140.0596656662032,"total_tps":1260.5369909958285,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":11495.196483563632,"ttft_p95_ms":91582.56554905092,"ttft_p99_ms":91693.05469256826,"tpot_p50_ms":38.41638142817654,"tpot_p95_ms":41.38107354754197,"tpot_p99_ms":42.587399352946996,"e2e_p50_ms":89565.8936999971,"e2e_p95_ms":172048.6941325711,"e2e_p99_ms":176627.42560487473,"itl_p50_ms":34.59935949649662,"itl_p95_ms":51.98211170500145,"itl_p99_ms":60.23820979986343,"validation_errors":[],"timestamp":"2026-07-16T12:10:09+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":2048,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":342.39226463483647,"request_tps":0.014603133646528293,"input_tps":239.25774166471956,"output_tps":29.907217708089945,"total_tps":269.1649593728095,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":3971.6174241621047,"ttft_p95_ms":4094.661511434242,"ttft_p99_ms":4115.855502327904,"tpot_p50_ms":31.558119324293,"tpot_p95_ms":31.620720006947725,"tpot_p99_ms":31.632591595734013,"e2e_p50_ms":68487.32287203893,"e2e_p95_ms":68711.56300739385,"e2e_p99_ms":68739.65807267465,"itl_p50_ms":29.78591388091445,"itl_p95_ms":45.330130099318914,"itl_p99_ms":55.03684663679451,"validation_errors":[],"timestamp":"2026-07-16T12:16:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":2048,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":765.7493072859943,"request_tps":0.1044728336530135,"input_tps":6846.731626283893,"output_tps":13.372522707585729,"total_tps":6860.104148991479,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":128567.64044612646,"ttft_p95_ms":151961.6587853641,"ttft_p99_ms":156960.82069837945,"tpot_p50_ms":161.96727417372753,"tpot_p95_ms":305.1854889790388,"tpot_p99_ms":437.706828691014,"e2e_p50_ms":146586.26923209522,"e2e_p95_ms":179067.64789508888,"e2e_p99_ms":186050.76994073341,"itl_p50_ms":36.512586986646056,"itl_p95_ms":53.31773438956598,"itl_p99_ms":61.62503688596189,"validation_errors":[],"timestamp":"2026-07-16T12:29:43+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":386.69948341907,"request_tps":0.10343949685769714,"input_tps":6779.01086606604,"output_tps":13.240255597785234,"total_tps":6792.251121663825,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":52697.72171450313,"ttft_p95_ms":77315.9501316608,"ttft_p99_ms":77702.88953800453,"tpot_p50_ms":161.86251543282611,"tpot_p95_ms":301.6361779644242,"tpot_p99_ms":390.316338291553,"e2e_p50_ms":73942.72513350006,"e2e_p95_ms":98705.0266831764,"e2e_p99_ms":108562.47204806423,"itl_p50_ms":36.458888091146946,"itl_p95_ms":54.5758139109239,"itl_p99_ms":62.127575105987496,"validation_errors":[],"timestamp":"2026-07-16T12:36:49+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":109.37776633584872,"request_tps":0.045713129528055126,"input_tps":2995.8556567506207,"output_tps":5.851280579591056,"total_tps":3001.706937330212,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":17729.769844096154,"ttft_p95_ms":17954.093989543617,"ttft_p99_ms":17964.415912255645,"tpot_p50_ms":32.0961656301891,"tpot_p95_ms":32.58936951105578,"tpot_p99_ms":32.640932479609305,"e2e_p50_ms":21835.877286968753,"e2e_p95_ms":22044.52510732226,"e2e_p99_ms":22048.520961580798,"itl_p50_ms":30.422291019931436,"itl_p95_ms":47.328294953331344,"itl_p99_ms":56.53701915405691,"validation_errors":[],"timestamp":"2026-07-16T12:39:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":832.5039105331525,"request_tps":0.09609564470245716,"input_tps":6297.724171220232,"output_tps":24.600485043829032,"total_tps":6322.324656264061,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":136335.5990794953,"ttft_p95_ms":162983.08086185716,"ttft_p99_ms":166663.45600951457,"tpot_p50_ms":100.12674616266263,"tpot_p95_ms":175.98028106304506,"tpot_p99_ms":236.80305566614055,"e2e_p50_ms":156982.1368450066,"e2e_p95_ms":189405.96394817112,"e2e_p99_ms":206104.3898640247,"itl_p50_ms":36.563907051458955,"itl_p95_ms":53.42142162844535,"itl_p99_ms":62.24962119013072,"validation_errors":[],"timestamp":"2026-07-16T12:53:50+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":419.1958255460486,"request_tps":0.09542079754228375,"input_tps":6253.497387731108,"output_tps":24.42772417082464,"total_tps":6277.925111901933,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":52760.39652095642,"ttft_p95_ms":81073.16757803783,"ttft_p99_ms":81568.98809597129,"tpot_p50_ms":99.82632672934629,"tpot_p95_ms":169.0839746892087,"tpot_p99_ms":210.72348307306856,"e2e_p50_ms":80142.28945260402,"e2e_p95_ms":107118.48287502071,"e2e_p99_ms":117810.70945826358,"itl_p50_ms":36.5109599661082,"itl_p95_ms":53.218919737264514,"itl_p99_ms":61.9426407897845,"validation_errors":[],"timestamp":"2026-07-16T13:01:28+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":129.80575013905764,"request_tps":0.038519094837044014,"input_tps":2524.3873992405165,"output_tps":9.860888278283268,"total_tps":2534.2482875187998,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":17821.7026530765,"ttft_p95_ms":17941.041450900957,"ttft_p99_ms":17956.14996048622,"tpot_p50_ms":32.05825664723913,"tpot_p95_ms":32.285422013122954,"tpot_p99_ms":32.28693708328201,"e2e_p50_ms":26052.55355197005,"e2e_p95_ms":26127.57891532965,"e2e_p99_ms":26133.34180940874,"itl_p50_ms":30.41153005324304,"itl_p95_ms":46.34950009640306,"itl_p99_ms":56.67572288773954,"validation_errors":[],"timestamp":"2026-07-16T13:04:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":256,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":973.7963511580601,"request_tps":0.08215270051573127,"input_tps":5383.959380998965,"output_tps":42.06218266405441,"total_tps":5426.021563663019,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":153779.00782134384,"ttft_p95_ms":181627.71489518927,"ttft_p99_ms":183322.4648270011,"tpot_p50_ms":69.34998192962937,"tpot_p95_ms":106.79924657778838,"tpot_p99_ms":136.58104994009412,"e2e_p50_ms":189005.7152614463,"e2e_p95_ms":234631.01633958286,"e2e_p99_ms":236243.1676932983,"itl_p50_ms":36.63114202208817,"itl_p95_ms":53.81391057744622,"itl_p99_ms":62.73157489486038,"validation_errors":[],"timestamp":"2026-07-16T13:21:10+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":487.3736931588501,"request_tps":0.08207254630578259,"input_tps":5378.706394695768,"output_tps":42.021143708560686,"total_tps":5420.727538404329,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":52526.89733938314,"ttft_p95_ms":84162.39211051725,"ttft_p99_ms":89592.60615824256,"tpot_p50_ms":77.38403851756873,"tpot_p95_ms":114.1166405163389,"tpot_p99_ms":167.8516751349313,"e2e_p50_ms":79288.78492198419,"e2e_p95_ms":137056.76206527278,"e2e_p99_ms":142416.35865559801,"itl_p50_ms":36.64115187712014,"itl_p95_ms":55.09973680600524,"itl_p99_ms":62.820788719691336,"validation_errors":[],"timestamp":"2026-07-16T13:29:57+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":170.8164885980077,"request_tps":0.029271178918604212,"input_tps":1918.3159816096456,"output_tps":14.986843606325357,"total_tps":1933.302825215971,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":17657.30693913065,"ttft_p95_ms":17947.263365006074,"ttft_p99_ms":17974.353000214323,"tpot_p50_ms":32.16144313886325,"tpot_p95_ms":32.28488007000849,"tpot_p99_ms":32.28917933327971,"e2e_p50_ms":34091.804383089766,"e2e_p95_ms":34388.406378310174,"e2e_p99_ms":34406.406796555966,"itl_p50_ms":30.36797302775085,"itl_p95_ms":46.908468334004276,"itl_p99_ms":55.469688819721355,"validation_errors":[],"timestamp":"2026-07-16T13:33:25+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":512,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1253.2045440499205,"request_tps":0.0638363468915201,"input_tps":4183.578829882661,"output_tps":65.36841921691658,"total_tps":4248.947249099578,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":202414.35351443943,"ttft_p95_ms":223751.87583953375,"ttft_p99_ms":226827.1145781991,"tpot_p50_ms":53.94717318803356,"tpot_p95_ms":71.33589557350184,"tpot_p99_ms":86.74920972760991,"e2e_p50_ms":247910.9200029634,"e2e_p95_ms":278792.46321712853,"e2e_p99_ms":292009.2727964232,"itl_p50_ms":36.60361119545996,"itl_p95_ms":54.5408145524561,"itl_p99_ms":62.74137625936419,"validation_errors":[],"timestamp":"2026-07-16T13:54:59+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i65536_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":626.4431783319451,"request_tps":0.06385255899267604,"input_tps":4184.641306144017,"output_tps":65.38502040850027,"total_tps":4250.026326552517,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":51137.51745200716,"ttft_p95_ms":96054.72948674572,"ttft_p99_ms":112415.82493769703,"tpot_p50_ms":70.99086488608475,"tpot_p95_ms":93.18777094192289,"tpot_p99_ms":94.39113497935946,"e2e_p50_ms":112995.05897297058,"e2e_p95_ms":168512.40640780888,"e2e_p99_ms":178801.59651178867,"itl_p50_ms":36.47732094395906,"itl_p95_ms":61.783290666062385,"itl_p99_ms":68.58176962938157,"validation_errors":[],"timestamp":"2026-07-16T14:06:05+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i65536_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":256.18154512299225,"request_tps":0.019517409021791596,"input_tps":1279.092917652134,"output_tps":19.985826838314594,"total_tps":1299.0787444904488,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":17658.26961491257,"ttft_p95_ms":18013.91982375644,"ttft_p99_ms":18047.10275827907,"tpot_p50_ms":32.7376385434701,"tpot_p95_ms":32.79802809666955,"tpot_p99_ms":32.798152351014515,"e2e_p50_ms":51210.01679683104,"e2e_p95_ms":51457.70765142515,"e2e_p99_ms":51481.52730548754,"itl_p50_ms":30.361955054104328,"itl_p95_ms":53.69571931660175,"itl_p99_ms":60.88819689117371,"validation_errors":[],"timestamp":"2026-07-16T14:11:00+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":1024,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i65536_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1677.9406901570037,"request_tps":0.047677489716584955,"input_tps":6249.183932132223,"output_tps":6.102718683722874,"total_tps":6255.286650815946,"mean_input_tokens":131072.0,"mean_output_tokens":128.0,"ttft_p50_ms":300625.9035965195,"ttft_p95_ms":338951.3012407697,"ttft_p99_ms":340203.0271626217,"tpot_p50_ms":41.211593476322115,"tpot_p95_ms":345.65387451581245,"tpot_p99_ms":346.94827124932925,"e2e_p50_ms":343158.3466755692,"e2e_p95_ms":344526.5441220952,"e2e_p99_ms":345839.6717084944,"itl_p50_ms":33.60768407583237,"itl_p95_ms":58.138947119005024,"itl_p99_ms":181.5207198727876,"validation_errors":[],"timestamp":"2026-07-16T14:40:04+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i131072_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i131072_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":818.1570529809687,"request_tps":0.04889036873086816,"input_tps":6408.158410292352,"output_tps":6.257967197551125,"total_tps":6414.4163774899025,"mean_input_tokens":131072.0,"mean_output_tokens":128.0,"ttft_p50_ms":129584.32166394778,"ttft_p95_ms":167181.3310468453,"ttft_p99_ms":168040.67650724668,"tpot_p50_ms":36.84465442857463,"tpot_p95_ms":343.3015784548456,"tpot_p99_ms":345.92269529082523,"e2e_p50_ms":171526.36588714086,"e2e_p95_ms":173146.40125581063,"e2e_p99_ms":173291.67424835498,"itl_p50_ms":33.540375996381044,"itl_p95_ms":58.00506006926298,"itl_p99_ms":276.8040000228211,"validation_errors":[],"timestamp":"2026-07-16T14:54:46+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c8_i131072_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c8_i131072_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":226.54693996487185,"request_tps":0.022070481290876385,"input_tps":2892.8221237577495,"output_tps":2.8250216052321773,"total_tps":2895.647145362982,"mean_input_tokens":131072.0,"mean_output_tokens":128.0,"ttft_p50_ms":40983.879873063415,"ttft_p95_ms":41427.96418489888,"ttft_p99_ms":41483.68903929368,"tpot_p50_ms":33.28039894415814,"tpot_p95_ms":33.693099089744166,"tpot_p99_ms":33.698522558318466,"e2e_p50_ms":45259.45955491625,"e2e_p95_ms":45639.66128444299,"e2e_p99_ms":45701.79939834401,"itl_p50_ms":30.720365000888705,"itl_p95_ms":55.317965033464134,"itl_p99_ms":62.53650498110801,"validation_errors":[],"timestamp":"2026-07-16T14:59:34+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c1_i131072_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c1_i131072_o128_a1.log"}
|
||||||
|
{"timestamp":"2026-07-16T16:03:27+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":2,"status":"FAILED","completed":0,"failed":80,"error_type":"benchmark rc=124","validation_errors":[],"gain_pct":null,"plateau_streak":0,"raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i131072_o256_a2.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i131072_o256_a2.log"}
|
||||||
|
{"timestamp":"2026-07-16T17:11:50+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":2,"status":"FAILED","completed":0,"failed":80,"error_type":"benchmark rc=124","validation_errors":[],"gain_pct":null,"plateau_streak":0,"raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/raw_outputs/sglang_adaptive_c16_i131072_o512_a2.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp4_dp2/logs/sglang_c16_i131072_o512_a2.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":40.91812721895985,"request_tps":1.9551236930250107,"input_tps":2002.046661657611,"output_tps":250.25583270720136,"total_tps":2252.3024943648124,"mean_input_tokens":1024.0,"mean_output_tokens":128.0,"ttft_p50_ms":2898.4853614820167,"ttft_p95_ms":5916.232369153295,"ttft_p99_ms":5917.164536933415,"tpot_p50_ms":36.42805760238671,"tpot_p95_ms":45.71566509713573,"tpot_p99_ms":52.58390468757908,"e2e_p50_ms":7513.254255056381,"e2e_p95_ms":10505.285932659172,"e2e_p99_ms":10506.672543857712,"itl_p50_ms":33.98504899814725,"itl_p95_ms":55.32407406717539,"itl_p99_ms":61.596665140241385,"validation_errors":[],"timestamp":"2026-07-16T17:22:33+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":23.175778079079464,"request_tps":1.7259398956752865,"input_tps":1767.3624531714934,"output_tps":220.92030664643667,"total_tps":1988.28275981793,"mean_input_tokens":1024.0,"mean_output_tokens":128.0,"ttft_p50_ms":586.4921135362238,"ttft_p95_ms":622.7808163734153,"ttft_p99_ms":623.125078517478,"tpot_p50_ms":31.85279993340373,"tpot_p95_ms":32.22969000018752,"tpot_p99_ms":32.44051498659762,"e2e_p50_ms":4641.481174970977,"e2e_p95_ms":4670.240912691224,"e2e_p99_ms":4670.475004990585,"itl_p50_ms":29.695762554183602,"itl_p95_ms":49.66246752301232,"itl_p99_ms":57.65656981850043,"validation_errors":[],"timestamp":"2026-07-16T17:23:16+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i1024_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i1024_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":49.60790970898233,"request_tps":1.6126460572378174,"input_tps":1651.349562611525,"output_tps":412.83739065288125,"total_tps":2064.1869532644064,"mean_input_tokens":1024.0,"mean_output_tokens":256.0,"ttft_p50_ms":935.7815920375288,"ttft_p95_ms":987.9027201328427,"ttft_p99_ms":988.8454519142397,"tpot_p50_ms":35.24840932302907,"tpot_p95_ms":35.34214854765428,"tpot_p99_ms":35.575809728372484,"e2e_p50_ms":9909.394073998556,"e2e_p95_ms":9992.725722829346,"e2e_p99_ms":9993.638517349027,"itl_p50_ms":33.14175654668361,"itl_p95_ms":51.57231901539489,"itl_p99_ms":59.3762440350838,"validation_errors":[],"timestamp":"2026-07-16T17:24:25+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":72.94000257505104,"request_tps":2.1935836900385253,"input_tps":2246.22969859945,"output_tps":561.5574246498625,"total_tps":2807.7871232493126,"mean_input_tokens":1024.0,"mean_output_tokens":256.0,"ttft_p50_ms":2986.9613859336823,"ttft_p95_ms":5638.966712448744,"ttft_p99_ms":5717.785984254442,"tpot_p50_ms":43.25931125649196,"tpot_p95_ms":52.76475823822194,"tpot_p99_ms":60.14857301456559,"e2e_p50_ms":14733.08369051665,"e2e_p95_ms":16383.586864836978,"e2e_p99_ms":16391.066697821952,"itl_p50_ms":40.04870797507465,"itl_p95_ms":61.18088683579117,"itl_p99_ms":67.89261806756254,"validation_errors":[],"timestamp":"2026-07-16T17:25:59+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":256,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":36.023877,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c32_i1024_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c32_i1024_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":94.76783368107863,"request_tps":0.8441682888861142,"input_tps":864.4283278193809,"output_tps":432.21416390969046,"total_tps":1296.6424917290713,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":947.5187105126679,"ttft_p95_ms":1040.429971262347,"ttft_p99_ms":1041.2525061913766,"tpot_p50_ms":35.295242745540456,"tpot_p95_ms":35.35704588877044,"tpot_p99_ms":35.459834945952224,"e2e_p50_ms":18922.931105131283,"e2e_p95_ms":19052.50374833122,"e2e_p99_ms":19055.464482815005,"itl_p50_ms":33.14921457786113,"itl_p95_ms":51.78932250710204,"itl_p99_ms":59.60216530133038,"validation_errors":[],"timestamp":"2026-07-16T17:27:54+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":116.47414470394142,"request_tps":1.3736954274847377,"input_tps":1406.6641177443714,"output_tps":703.3320588721857,"total_tps":2109.996176616557,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":1728.7794339936227,"ttft_p95_ms":1780.919153173454,"ttft_p99_ms":1782.5818243529648,"tpot_p50_ms":42.263241610289306,"tpot_p95_ms":42.444828835041704,"tpot_p99_ms":42.508655317581514,"e2e_p50_ms":23282.43359492626,"e2e_p95_ms":23419.968224375043,"e2e_p99_ms":23422.798421205953,"itl_p50_ms":40.06265802308917,"itl_p95_ms":58.03677138173953,"itl_p99_ms":67.34221158549191,"validation_errors":[],"timestamp":"2026-07-16T17:30:11+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":512,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":62.727675,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c32_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c32_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":240,"failed":0,"duration_s":538.8399277571589,"request_tps":0.44540129199215867,"input_tps":456.0909229999705,"output_tps":228.04546149998524,"total_tps":684.1363844999557,"mean_input_tokens":1024.0,"mean_output_tokens":512.0,"ttft_p50_ms":2837.4407634837553,"ttft_p95_ms":8041.054585680831,"ttft_p99_ms":8290.32544139307,"tpot_p50_ms":202.26081837298543,"tpot_p95_ms":209.53028491557413,"tpot_p99_ms":212.22897237452446,"e2e_p50_ms":106162.95209643431,"e2e_p95_ms":111388.48089557141,"e2e_p99_ms":111393.2417512266,"itl_p50_ms":197.14024697896093,"itl_p95_ms":226.85785598587245,"itl_p99_ms":233.4809394343756,"validation_errors":[],"timestamp":"2026-07-16T17:39:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":512,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":1,"gain_pct":-67.576416,"plateau_streak":1,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c48_i1024_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c48_i1024_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":195.86190408095717,"request_tps":0.40845104807585736,"input_tps":418.25387322967794,"output_tps":418.25387322967794,"total_tps":836.5077464593559,"mean_input_tokens":1024.0,"mean_output_tokens":1024.0,"ttft_p50_ms":2806.6968551138416,"ttft_p95_ms":2903.0821086023934,"ttft_p99_ms":2903.6138558760285,"tpot_p50_ms":35.50938803068511,"tpot_p95_ms":36.587063905092016,"tpot_p99_ms":36.846195016498285,"e2e_p50_ms":39140.523318434134,"e2e_p95_ms":39359.496829425916,"e2e_p99_ms":39362.11585983634,"itl_p50_ms":33.30066951457411,"itl_p95_ms":52.0355909015052,"itl_p99_ms":61.44081162754446,"validation_errors":[],"timestamp":"2026-07-16T17:43:08+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":235.65979528496973,"request_tps":0.6789448314954245,"input_tps":695.2395074513147,"output_tps":695.2395074513147,"total_tps":1390.4790149026294,"mean_input_tokens":1024.0,"mean_output_tokens":1024.0,"ttft_p50_ms":2981.6380555275828,"ttft_p95_ms":5691.65476138005,"ttft_p99_ms":5695.868596371729,"tpot_p50_ms":42.80786131479516,"tpot_p95_ms":45.161864508836935,"tpot_p99_ms":45.66859811666182,"e2e_p50_ms":47101.06354195159,"e2e_p95_ms":49224.63555838913,"e2e_p99_ms":49227.704358948395,"itl_p50_ms":40.25210742838681,"itl_p95_ms":59.54401966882868,"itl_p99_ms":67.25824875058606,"validation_errors":[],"timestamp":"2026-07-16T17:47:25+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":1024,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":66.224284,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c32_i1024_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c32_i1024_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":375.86731938296,"request_tps":0.21284106352031737,"input_tps":217.949249044805,"output_tps":435.89849808961,"total_tps":653.8477471344149,"mean_input_tokens":1024.0,"mean_output_tokens":2048.0,"ttft_p50_ms":1012.9853350808844,"ttft_p95_ms":1060.0732530583628,"ttft_p99_ms":1060.6082381820306,"tpot_p50_ms":36.26880324916558,"tpot_p95_ms":36.36031736202677,"tpot_p99_ms":36.50945045857321,"e2e_p50_ms":75173.97456045728,"e2e_p95_ms":75301.6543291742,"e2e_p99_ms":75354.01330835419,"itl_p50_ms":34.3891941010952,"itl_p95_ms":56.740176025778055,"itl_p99_ms":61.57854507677257,"validation_errors":[],"timestamp":"2026-07-16T17:54:03+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":455.1479200378526,"request_tps":0.35153406827981004,"input_tps":359.9708859185255,"output_tps":719.941771837051,"total_tps":1079.9126577555764,"mean_input_tokens":1024.0,"mean_output_tokens":2048.0,"ttft_p50_ms":2997.489857953042,"ttft_p95_ms":4408.118210406974,"ttft_p99_ms":4409.418380276766,"tpot_p50_ms":43.03901399925722,"tpot_p95_ms":43.72752447672071,"tpot_p99_ms":43.95574388355626,"e2e_p50_ms":91594.98853096738,"e2e_p95_ms":92511.16100725485,"e2e_p99_ms":92513.411904776,"itl_p50_ms":40.91528104618192,"itl_p95_ms":61.7140918620862,"itl_p99_ms":67.57471586111933,"validation_errors":[],"timestamp":"2026-07-16T18:01:59+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":2048,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":65.16271,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c32_i1024_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c32_i1024_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":751.3376580928452,"request_tps":0.1064767606658605,"input_tps":109.03220292184115,"output_tps":436.1288116873646,"total_tps":545.1610146092057,"mean_input_tokens":1024.0,"mean_output_tokens":4096.0,"ttft_p50_ms":1016.063955030404,"ttft_p95_ms":1555.9732719440944,"ttft_p99_ms":1557.605766323395,"tpot_p50_ms":36.36836581573491,"tpot_p95_ms":36.73897300841474,"tpot_p99_ms":36.74271235461387,"e2e_p50_ms":149854.72987149842,"e2e_p95_ms":151522.32534443028,"e2e_p99_ms":151525.3242665343,"itl_p50_ms":34.41172291059047,"itl_p95_ms":56.844778708182275,"itl_p99_ms":61.78853524383158,"validation_errors":[],"timestamp":"2026-07-16T18:14:52+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":4096,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i1024_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i1024_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":160,"failed":0,"duration_s":898.5537388259545,"request_tps":0.17806391881364284,"input_tps":182.33745286517026,"output_tps":729.349811460681,"total_tps":911.6872643258514,"mean_input_tokens":1024.0,"mean_output_tokens":4096.0,"ttft_p50_ms":3016.2806854350492,"ttft_p95_ms":3879.865962965414,"ttft_p99_ms":3881.218241781462,"tpot_p50_ms":43.30877847119516,"tpot_p95_ms":43.5328147706794,"tpot_p99_ms":43.585850752513146,"e2e_p50_ms":179637.5885854941,"e2e_p95_ms":181239.3637237721,"e2e_p99_ms":181245.79159579705,"itl_p50_ms":40.978090022690594,"itl_p95_ms":61.92717199446633,"itl_p99_ms":67.58996407035738,"validation_errors":[],"timestamp":"2026-07-16T18:30:14+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":4096,"concurrency":32,"num_prompts":160,"warmup_requests":32,"attempt":1,"gain_pct":67.23266,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c32_i1024_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c32_i1024_o4096_a1.log"}
|
||||||
|
{"timestamp":"2026-07-16T19:35:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":4096,"concurrency":48,"num_prompts":240,"warmup_requests":48,"attempt":2,"status":"FAILED","completed":0,"failed":240,"error_type":"benchmark rc=124","validation_errors":[],"gain_pct":null,"plateau_streak":0,"raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c48_i1024_o4096_a2.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c48_i1024_o4096_a2.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":81.84067250299267,"request_tps":0.9775090740740753,"input_tps":4003.8771674074123,"output_tps":125.12116148148164,"total_tps":4128.998328888893,"mean_input_tokens":4096.0,"mean_output_tokens":128.0,"ttft_p50_ms":7960.9383844072,"ttft_p95_ms":11711.344682960771,"ttft_p99_ms":11753.230824810453,"tpot_p50_ms":66.47893063738708,"tpot_p95_ms":110.99464661427078,"tpot_p99_ms":112.9407165098686,"e2e_p50_ms":16375.667929998599,"e2e_p95_ms":16515.728015080094,"e2e_p99_ms":16517.64271946624,"itl_p50_ms":34.67924799770117,"itl_p95_ms":57.22848706645891,"itl_p99_ms":62.60195288574323,"validation_errors":[],"timestamp":"2026-07-16T19:42:19+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":49.58700227085501,"request_tps":0.8066629997415711,"input_tps":3304.091646941475,"output_tps":103.2528639669211,"total_tps":3407.3445109083964,"mean_input_tokens":4096.0,"mean_output_tokens":128.0,"ttft_p50_ms":5025.09073051624,"ttft_p95_ms":5915.067831950728,"ttft_p99_ms":5934.279049527831,"tpot_p50_ms":39.28249386619542,"tpot_p95_ms":62.01645719083613,"tpot_p99_ms":62.2953052225332,"e2e_p50_ms":10006.435959017836,"e2e_p95_ms":10091.394168324769,"e2e_p99_ms":10092.632040372118,"itl_p50_ms":30.577882542274892,"itl_p95_ms":49.62789301062003,"itl_p99_ms":57.752098962664604,"validation_errors":[],"timestamp":"2026-07-16T19:43:30+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":18.347577747888863,"request_tps":0.2725155368574643,"input_tps":1116.2236389681739,"output_tps":34.88198871775543,"total_tps":1151.1056276859294,"mean_input_tokens":4096.0,"mean_output_tokens":128.0,"ttft_p50_ms":250.82844705320895,"ttft_p95_ms":256.7141311708838,"ttft_p99_ms":257.63806560076773,"tpot_p50_ms":26.909995739766227,"tpot_p95_ms":27.076458471907873,"tpot_p99_ms":27.091512166432977,"e2e_p50_ms":3668.39790600352,"e2e_p95_ms":3695.424357103184,"e2e_p99_ms":3698.260110737756,"itl_p50_ms":24.74083099514246,"itl_p95_ms":47.6031189551577,"itl_p99_ms":49.66991530265658,"validation_errors":[],"timestamp":"2026-07-16T19:44:08+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":78.74828192405403,"request_tps":1.0158951794929716,"input_tps":4161.106655203212,"output_tps":260.0691659502007,"total_tps":4421.1758211534125,"mean_input_tokens":4096.0,"mean_output_tokens":256.0,"ttft_p50_ms":3091.937036952004,"ttft_p95_ms":11642.126731504686,"ttft_p99_ms":11714.910287000239,"tpot_p50_ms":37.18031867425524,"tpot_p95_ms":68.26280832984575,"tpot_p99_ms":73.90357631190186,"e2e_p50_ms":15660.35121458117,"e2e_p95_ms":21110.056800360326,"e2e_p99_ms":21111.51674753055,"itl_p50_ms":34.62257899809629,"itl_p95_ms":57.151046535000205,"itl_p99_ms":62.4327381211333,"validation_errors":[],"timestamp":"2026-07-16T19:45:48+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":70.42232585605234,"request_tps":0.5680016885804441,"input_tps":2326.534916425499,"output_tps":145.4084322765937,"total_tps":2471.943348702093,"mean_input_tokens":4096.0,"mean_output_tokens":256.0,"ttft_p50_ms":5035.245958482847,"ttft_p95_ms":5984.106912557036,"ttft_p99_ms":6042.94591952581,"tpot_p50_ms":35.78011738455982,"tpot_p95_ms":46.93394760007733,"tpot_p99_ms":47.50900067536927,"e2e_p50_ms":14144.402876030654,"e2e_p95_ms":14412.95803807443,"e2e_p99_ms":14413.140350417234,"itl_p50_ms":30.597370583564043,"itl_p95_ms":50.928640994243324,"itl_p99_ms":56.54409846989438,"validation_errors":[],"timestamp":"2026-07-16T19:47:19+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":35.27845326298848,"request_tps":0.1417295696817192,"input_tps":580.5243174163219,"output_tps":36.28276983852012,"total_tps":616.807087254842,"mean_input_tokens":4096.0,"mean_output_tokens":256.0,"ttft_p50_ms":268.0197979789227,"ttft_p95_ms":291.9198756571859,"ttft_p99_ms":292.9051029961556,"tpot_p50_ms":26.539895318302456,"tpot_p95_ms":26.827862831928275,"tpot_p99_ms":26.862892086781997,"e2e_p50_ms":7060.824715998024,"e2e_p95_ms":7112.919608317316,"e2e_p99_ms":7118.816237747669,"itl_p50_ms":24.650061037391424,"itl_p95_ms":44.851708575151854,"itl_p99_ms":49.219665322452784,"validation_errors":[],"timestamp":"2026-07-16T19:48:14+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":256,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":126.69021714199334,"request_tps":0.63146154300404,"input_tps":2586.466480144548,"output_tps":323.3083100180685,"total_tps":2909.7747901626167,"mean_input_tokens":4096.0,"mean_output_tokens":512.0,"ttft_p50_ms":3073.0733450036496,"ttft_p95_ms":11019.555248494726,"ttft_p99_ms":11065.65095537575,"tpot_p50_ms":38.32130294193836,"tpot_p95_ms":53.03848213837399,"tpot_p99_ms":58.19536544753461,"e2e_p50_ms":25832.93817844242,"e2e_p95_ms":30626.584287162405,"e2e_p99_ms":30627.329029503744,"itl_p50_ms":34.66650005429983,"itl_p95_ms":56.83219095226377,"itl_p99_ms":62.253236598335214,"validation_errors":[],"timestamp":"2026-07-16T19:50:41+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":112.55219594296068,"request_tps":0.3553906671023215,"input_tps":1455.6801724511088,"output_tps":181.9600215563886,"total_tps":1637.6401940074975,"mean_input_tokens":4096.0,"mean_output_tokens":512.0,"ttft_p50_ms":5065.312996157445,"ttft_p95_ms":6016.085800586734,"ttft_p99_ms":6074.749361765571,"tpot_p50_ms":34.33472787484767,"tpot_p95_ms":40.08658976554269,"tpot_p99_ms":40.18997862394681,"e2e_p50_ms":22591.4841449121,"e2e_p95_ms":22795.752924506087,"e2e_p99_ms":22796.309465037193,"itl_p50_ms":30.596577562391758,"itl_p95_ms":50.246595905628055,"itl_p99_ms":56.772089656442404,"validation_errors":[],"timestamp":"2026-07-16T19:52:54+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":69.34047695202753,"request_tps":0.07210795511918958,"input_tps":295.3541841682005,"output_tps":36.91927302102506,"total_tps":332.27345718922555,"mean_input_tokens":4096.0,"mean_output_tokens":512.0,"ttft_p50_ms":248.92402300611138,"ttft_p95_ms":266.0276424139738,"ttft_p99_ms":268.3548469096422,"tpot_p50_ms":26.661601453862808,"tpot_p95_ms":26.700651652677212,"tpot_p99_ms":26.707691252833037,"e2e_p50_ms":13878.469962859526,"e2e_p95_ms":13891.861443966627,"e2e_p99_ms":13893.72499614954,"itl_p50_ms":24.624896002933383,"itl_p95_ms":45.65126097295369,"itl_p99_ms":49.571911790408194,"validation_errors":[],"timestamp":"2026-07-16T19:54:24+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":512,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":219.81785867293365,"request_tps":0.363937673139796,"input_tps":1490.6887091806045,"output_tps":372.67217729515113,"total_tps":1863.3608864757555,"mean_input_tokens":4096.0,"mean_output_tokens":1024.0,"ttft_p50_ms":3086.4613954909146,"ttft_p95_ms":11646.606297139078,"ttft_p99_ms":11760.603103360627,"tpot_p50_ms":37.052429024333115,"tpot_p95_ms":44.42799982865398,"tpot_p99_ms":45.902305760968,"e2e_p50_ms":43634.44979744963,"e2e_p95_ms":49578.70842582779,"e2e_p99_ms":49580.79158608103,"itl_p50_ms":34.58560153376311,"itl_p95_ms":56.85673162806779,"itl_p99_ms":61.24365669908002,"validation_errors":[],"timestamp":"2026-07-16T19:58:24+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":196.3215377200395,"request_tps":0.2037473853584074,"input_tps":834.5492904280367,"output_tps":208.63732260700917,"total_tps":1043.1866130350459,"mean_input_tokens":4096.0,"mean_output_tokens":1024.0,"ttft_p50_ms":5105.625024996698,"ttft_p95_ms":5958.576378412545,"ttft_p99_ms":6064.859193183947,"tpot_p50_ms":33.55810541205767,"tpot_p95_ms":36.34212366745689,"tpot_p99_ms":36.55665332609869,"e2e_p50_ms":39269.71339690499,"e2e_p95_ms":39593.57306190068,"e2e_p99_ms":39594.54825331457,"itl_p50_ms":30.5747389793396,"itl_p95_ms":50.719153042882674,"itl_p99_ms":56.99894693214446,"validation_errors":[],"timestamp":"2026-07-16T20:02:01+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":137.4567740750499,"request_tps":0.0363750716081119,"input_tps":148.99229330682635,"output_tps":37.24807332670659,"total_tps":186.24036663353294,"mean_input_tokens":4096.0,"mean_output_tokens":1024.0,"ttft_p50_ms":250.88916602544487,"ttft_p95_ms":253.6237696185708,"ttft_p99_ms":253.63623090088367,"tpot_p50_ms":26.61167698223162,"tpot_p95_ms":26.682842084191165,"tpot_p99_ms":26.683082640242063,"e2e_p50_ms":27477.38489904441,"e2e_p95_ms":27547.797128511593,"e2e_p99_ms":27548.662202609703,"itl_p50_ms":24.606115883216262,"itl_p95_ms":47.627381072379656,"itl_p99_ms":50.5407242104411,"validation_errors":[],"timestamp":"2026-07-16T20:04:39+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":1024,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":407.2602603479754,"request_tps":0.19643458443906506,"input_tps":804.5960578624105,"output_tps":402.29802893120524,"total_tps":1206.8940867936158,"mean_input_tokens":4096.0,"mean_output_tokens":2048.0,"ttft_p50_ms":3240.20006461069,"ttft_p95_ms":11741.664571419824,"ttft_p99_ms":11840.223265609238,"tpot_p50_ms":36.97641234489819,"tpot_p95_ms":40.52301196099482,"tpot_p99_ms":41.266952687252264,"e2e_p50_ms":80944.1796640167,"e2e_p95_ms":87096.1231123074,"e2e_p99_ms":87098.59982100548,"itl_p50_ms":34.466654527932405,"itl_p95_ms":56.807562068570405,"itl_p99_ms":61.39076543040574,"validation_errors":[],"timestamp":"2026-07-16T20:11:47+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":363.24991058488376,"request_tps":0.11011702641741698,"input_tps":451.03934020573996,"output_tps":225.51967010286998,"total_tps":676.5590103086099,"mean_input_tokens":4096.0,"mean_output_tokens":2048.0,"ttft_p50_ms":5055.334709468298,"ttft_p95_ms":5906.876079260837,"ttft_p99_ms":6023.611045922153,"tpot_p50_ms":33.129859284787166,"tpot_p95_ms":34.53019272802082,"tpot_p99_ms":34.57053666370664,"e2e_p50_ms":72728.55528548826,"e2e_p95_ms":73087.71414748626,"e2e_p99_ms":73088.92423222307,"itl_p50_ms":30.50815698225051,"itl_p95_ms":50.1406642026268,"itl_p99_ms":56.95488589582965,"validation_errors":[],"timestamp":"2026-07-16T20:18:11+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":2048,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":273.5867548820097,"request_tps":0.018275738539156837,"input_tps":74.8574250563864,"output_tps":37.4287125281932,"total_tps":112.28613758457962,"mean_input_tokens":4096.0,"mean_output_tokens":2048.0,"ttft_p50_ms":261.7046369705349,"ttft_p95_ms":349.8987853061408,"ttft_p99_ms":362.0478419493884,"tpot_p50_ms":26.592822536383906,"tpot_p95_ms":26.61543222684295,"tpot_p99_ms":26.61759176356131,"e2e_p50_ms":54736.32405605167,"e2e_p95_ms":54748.984178341925,"e2e_p99_ms":54749.01293158531,"itl_p50_ms":24.60544602945447,"itl_p95_ms":47.4006032804027,"itl_p99_ms":49.956067339517176,"validation_errors":[],"timestamp":"2026-07-16T20:23:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":2048,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":781.9880918110721,"request_tps":0.10230334814270799,"input_tps":419.0345139925319,"output_tps":419.0345139925319,"total_tps":838.0690279850638,"mean_input_tokens":4096.0,"mean_output_tokens":4096.0,"ttft_p50_ms":3097.252030042,"ttft_p95_ms":11633.983958070166,"ttft_p99_ms":11746.600034988951,"tpot_p50_ms":36.86781218838315,"tpot_p95_ms":38.49309932329363,"tpot_p99_ms":38.90570051973158,"e2e_p50_ms":155683.48012503702,"e2e_p95_ms":162603.52390279295,"e2e_p99_ms":162604.7951752739,"itl_p50_ms":34.45248049683869,"itl_p95_ms":56.79855167400092,"itl_p99_ms":61.650155324023224,"validation_errors":[],"timestamp":"2026-07-16T20:36:29+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":4096,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i4096_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i4096_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":697.7969978589099,"request_tps":0.057323261812152054,"input_tps":234.79608038257481,"output_tps":234.79608038257481,"total_tps":469.59216076514963,"mean_input_tokens":4096.0,"mean_output_tokens":4096.0,"ttft_p50_ms":5059.23117196653,"ttft_p95_ms":6020.107874658424,"ttft_p99_ms":6025.99322974449,"tpot_p50_ms":32.921226551871115,"tpot_p95_ms":33.65806957458343,"tpot_p99_ms":33.68422295676342,"e2e_p50_ms":139956.67370257434,"e2e_p95_ms":140090.03641876625,"e2e_p99_ms":140090.8339079097,"itl_p50_ms":30.493864906020463,"itl_p95_ms":50.366084929555655,"itl_p99_ms":57.08304369123653,"validation_errors":[],"timestamp":"2026-07-16T20:48:28+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":4096,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i4096_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i4096_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":548.1068854129408,"request_tps":0.009122308318081111,"input_tps":37.36497487086023,"output_tps":37.36497487086023,"total_tps":74.72994974172046,"mean_input_tokens":4096.0,"mean_output_tokens":4096.0,"ttft_p50_ms":270.29989496804774,"ttft_p95_ms":280.77137977816164,"ttft_p99_ms":282.4734351504594,"tpot_p50_ms":26.703159420515647,"tpot_p95_ms":26.721241031938273,"tpot_p99_ms":26.721443862750927,"e2e_p50_ms":109619.73772197962,"e2e_p95_ms":109701.13868489861,"e2e_p99_ms":109702.0101480931,"itl_p50_ms":24.71006289124489,"itl_p95_ms":45.58566838968543,"itl_p99_ms":49.87936850637197,"validation_errors":[],"timestamp":"2026-07-16T20:57:57+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":4096,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i4096_o4096_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i4096_o4096_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":256.7601454681717,"request_tps":0.31157483516037693,"input_tps":5104.842099267616,"output_tps":39.88157890052825,"total_tps":5144.723678168144,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":25187.957516987808,"ttft_p95_ms":47894.87515146611,"ttft_p99_ms":49477.59126278339,"tpot_p50_ms":198.17720644458336,"tpot_p95_ms":368.67252661503744,"tpot_p99_ms":391.46561684295915,"e2e_p50_ms":54196.20531098917,"e2e_p95_ms":54292.84199364483,"e2e_p99_ms":54294.34734713286,"itl_p50_ms":35.08518589660525,"itl_p95_ms":58.37633586488664,"itl_p99_ms":64.08695708261803,"validation_errors":[],"timestamp":"2026-07-16T21:02:35+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":141.1061867091339,"request_tps":0.283474459432832,"input_tps":4644.44554334752,"output_tps":36.2847308074025,"total_tps":4680.730274154922,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":14016.848506988026,"ttft_p95_ms":24515.615026152227,"ttft_p99_ms":24870.744992317632,"tpot_p50_ms":105.65934030661785,"tpot_p95_ms":191.19213953821398,"tpot_p99_ms":193.04246251707838,"e2e_p50_ms":28676.901251543313,"e2e_p95_ms":29117.543387238402,"e2e_p99_ms":29119.045873775613,"itl_p50_ms":30.759848072193563,"itl_p95_ms":50.73059933492914,"itl_p99_ms":57.838678041007384,"validation_errors":[],"timestamp":"2026-07-16T21:05:20+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":30.202961774077266,"request_tps":0.16554667841520834,"input_tps":2712.3167791547735,"output_tps":21.189974837146668,"total_tps":2733.5067539919205,"mean_input_tokens":16384.0,"mean_output_tokens":128.0,"ttft_p50_ms":3189.695225097239,"ttft_p95_ms":3287.1102603152394,"ttft_p99_ms":3298.7277361750603,"tpot_p50_ms":27.00854029953773,"tpot_p95_ms":27.13199242340826,"tpot_p99_ms":27.15187601517798,"e2e_p50_ms":6534.987838938832,"e2e_p95_ms":6732.873298088089,"e2e_p99_ms":6747.015990102664,"itl_p50_ms":24.80782102793455,"itl_p95_ms":45.76855925843107,"itl_p99_ms":49.268019353039556,"validation_errors":[],"timestamp":"2026-07-16T21:06:13+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i16384_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i16384_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":278.9136179748457,"request_tps":0.2868271566690406,"input_tps":4699.376134865561,"output_tps":73.42775210727439,"total_tps":4772.803886972835,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":24543.6039295746,"ttft_p95_ms":47646.79518254706,"ttft_p99_ms":49249.08797713695,"tpot_p50_ms":123.02242240768072,"tpot_p95_ms":202.41332247026955,"tpot_p99_ms":213.29141370800562,"e2e_p50_ms":58673.51271153893,"e2e_p95_ms":59028.76297895564,"e2e_p99_ms":59031.15820220439,"itl_p50_ms":35.10860598180443,"itl_p95_ms":59.34609065297991,"itl_p99_ms":63.67699921829625,"validation_errors":[],"timestamp":"2026-07-16T21:11:13+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":162.15329257398844,"request_tps":0.24668015903376442,"input_tps":4041.607725609196,"output_tps":63.15012071264369,"total_tps":4104.75784632184,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":14056.083535542712,"ttft_p95_ms":24572.28690140182,"ttft_p99_ms":24781.60693206126,"tpot_p50_ms":69.22504512739239,"tpot_p95_ms":111.92492204695465,"tpot_p99_ms":112.1908097408584,"e2e_p50_ms":32945.15154813416,"e2e_p95_ms":33157.10190224927,"e2e_p99_ms":33157.680223954376,"itl_p50_ms":30.78671346884221,"itl_p95_ms":52.047876664437354,"itl_p99_ms":57.51584505662322,"validation_errors":[],"timestamp":"2026-07-16T21:14:19+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":47.27609982597642,"request_tps":0.10576168546908539,"input_tps":1732.799454725495,"output_tps":27.07499148008586,"total_tps":1759.8744462055809,"mean_input_tokens":16384.0,"mean_output_tokens":256.0,"ttft_p50_ms":3095.8918230608106,"ttft_p95_ms":3272.758443793282,"ttft_p99_ms":3304.284312753007,"tpot_p50_ms":27.063766945445657,"tpot_p95_ms":27.16058336417465,"tpot_p99_ms":27.17001092872199,"e2e_p50_ms":9997.152394149452,"e2e_p95_ms":10198.707201657817,"e2e_p99_ms":10232.637099577114,"itl_p50_ms":24.803407955914736,"itl_p95_ms":48.14038993790746,"itl_p99_ms":50.25792038068175,"validation_errors":[],"timestamp":"2026-07-16T21:15:29+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":256,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i16384_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i16384_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":326.68043872504495,"request_tps":0.2448876348771317,"input_tps":4012.2390098269257,"output_tps":125.38246905709143,"total_tps":4137.621478884017,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":24534.058870980516,"ttft_p95_ms":47585.56273026624,"ttft_p99_ms":49259.400079401676,"tpot_p50_ms":80.12083880425615,"tpot_p95_ms":119.69057329970916,"tpot_p99_ms":125.067007240863,"e2e_p50_ms":68319.87011944875,"e2e_p95_ms":68537.59080437012,"e2e_p99_ms":68539.8418288771,"itl_p50_ms":35.064953030087054,"itl_p95_ms":58.41207688208669,"itl_p99_ms":63.36872420506552,"validation_errors":[],"timestamp":"2026-07-16T21:21:17+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":204.11131394607946,"request_tps":0.19597149823143506,"input_tps":3210.797027023832,"output_tps":100.33740709449475,"total_tps":3311.134434118327,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":14109.92513189558,"ttft_p95_ms":24601.094605890103,"ttft_p99_ms":24752.130593049806,"tpot_p50_ms":50.97825450885589,"tpot_p95_ms":72.14095964029964,"tpot_p99_ms":72.50778853802152,"e2e_p50_ms":41347.00059145689,"e2e_p95_ms":41603.95504550543,"e2e_p99_ms":41605.48491582973,"itl_p50_ms":30.709165963344276,"itl_p95_ms":51.023998111486435,"itl_p99_ms":57.31804177630693,"validation_errors":[],"timestamp":"2026-07-16T21:25:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":81.74180358601734,"request_tps":0.06116821235463043,"input_tps":1002.179991218265,"output_tps":31.31812472557078,"total_tps":1033.4981159438357,"mean_input_tokens":16384.0,"mean_output_tokens":512.0,"ttft_p50_ms":3152.1379488985986,"ttft_p95_ms":3280.934850871563,"ttft_p99_ms":3303.358238041401,"tpot_p50_ms":26.90626537553062,"tpot_p95_ms":26.921899768354372,"tpot_p99_ms":26.923089013626313,"e2e_p50_ms":16891.912296880037,"e2e_p95_ms":17034.689639275894,"e2e_p99_ms":17060.389525359496,"itl_p50_ms":24.79128399863839,"itl_p95_ms":48.06917109526691,"itl_p99_ms":50.64208318479359,"validation_errors":[],"timestamp":"2026-07-16T21:26:50+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":512,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i16384_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i16384_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":423.16662176605314,"request_tps":0.1890508274639578,"input_tps":3097.4087571694845,"output_tps":193.58804732309278,"total_tps":3290.996804492577,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":24554.481841041707,"ttft_p95_ms":47767.74098525056,"ttft_p99_ms":49284.376522910316,"tpot_p50_ms":58.8259498166778,"tpot_p95_ms":78.8021439746958,"tpot_p99_ms":81.38845075320822,"e2e_p50_ms":87506.93495490123,"e2e_p95_ms":88091.98981691152,"e2e_p99_ms":88095.21754639223,"itl_p50_ms":35.09103413671255,"itl_p95_ms":58.61164742382244,"itl_p99_ms":64.02320196386427,"validation_errors":[],"timestamp":"2026-07-16T21:34:14+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":288.24989658407867,"request_tps":0.13876847996832684,"input_tps":2273.582775801067,"output_tps":142.09892348756668,"total_tps":2415.6816992886334,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":14033.50293962285,"ttft_p95_ms":24491.35324291419,"ttft_p99_ms":24759.231015304103,"tpot_p50_ms":41.92089678206183,"tpot_p95_ms":52.53722619232089,"tpot_p99_ms":52.69310522824981,"e2e_p50_ms":58040.07501259912,"e2e_p95_ms":58448.35202746326,"e2e_p99_ms":58450.28124490753,"itl_p50_ms":30.73286300059408,"itl_p95_ms":50.72684104088693,"itl_p99_ms":57.92215333785862,"validation_errors":[],"timestamp":"2026-07-16T21:39:26+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":150.84414941398427,"request_tps":0.03314679435314225,"input_tps":543.0770786818827,"output_tps":33.942317417617666,"total_tps":577.0193960995003,"mean_input_tokens":16384.0,"mean_output_tokens":1024.0,"ttft_p50_ms":3190.0415259879082,"ttft_p95_ms":3315.6291431281716,"ttft_p99_ms":3326.89755034633,"tpot_p50_ms":26.902348835946206,"tpot_p95_ms":26.929367462743006,"tpot_p99_ms":26.932126218243663,"e2e_p50_ms":30694.867955055088,"e2e_p95_ms":30857.89103670977,"e2e_p99_ms":30877.166467448696,"itl_p50_ms":24.886570870876312,"itl_p95_ms":45.76608098577708,"itl_p99_ms":50.4940598970279,"validation_errors":[],"timestamp":"2026-07-16T21:42:20+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":1024,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i16384_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i16384_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":614.5522862081416,"request_tps":0.13017606767620898,"input_tps":2132.804692807008,"output_tps":266.600586600876,"total_tps":2399.4052794078843,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":24607.956594438292,"ttft_p95_ms":47697.99113486661,"ttft_p99_ms":49292.4090649304,"tpot_p50_ms":48.10298325354711,"tpot_p95_ms":58.110758199371965,"tpot_p99_ms":59.46979306338756,"e2e_p50_ms":125611.4937895909,"e2e_p95_ms":126450.29024835676,"e2e_p99_ms":126453.36987355025,"itl_p50_ms":35.09154496714473,"itl_p95_ms":58.45766137354075,"itl_p99_ms":64.08793116686866,"validation_errors":[],"timestamp":"2026-07-16T21:52:57+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":2048,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":457.1198655979242,"request_tps":0.08750440094673856,"input_tps":1433.6721051113645,"output_tps":179.20901313892057,"total_tps":1612.881118250285,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":14047.149811987765,"ttft_p95_ms":24552.844235277735,"ttft_p99_ms":24749.485357583035,"tpot_p50_ms":37.538202461908504,"tpot_p95_ms":42.66441069203499,"tpot_p99_ms":42.9089602763204,"e2e_p50_ms":91750.90344448108,"e2e_p95_ms":92103.39087992907,"e2e_p99_ms":92105.43445243035,"itl_p50_ms":30.728504993021488,"itl_p95_ms":51.14136257907376,"itl_p99_ms":58.35309494053946,"validation_errors":[],"timestamp":"2026-07-16T22:00:58+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":2048,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":288.6096170139499,"request_tps":0.017324440022933558,"input_tps":283.8436253357434,"output_tps":35.480453166967926,"total_tps":319.32407850271136,"mean_input_tokens":16384.0,"mean_output_tokens":2048.0,"ttft_p50_ms":3130.5205351673067,"ttft_p95_ms":3326.790696242824,"ttft_p99_ms":3348.429289693013,"tpot_p50_ms":26.910882942859878,"tpot_p95_ms":26.939971389955993,"tpot_p99_ms":26.940729667464,"e2e_p50_ms":58267.43282494135,"e2e_p95_ms":58423.72469645925,"e2e_p99_ms":58437.077996963635,"itl_p50_ms":24.81332910247147,"itl_p95_ms":48.45747402869165,"itl_p99_ms":51.16548259742558,"validation_errors":[],"timestamp":"2026-07-16T22:06:10+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":2048,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i16384_o2048_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i16384_o2048_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1164.4257685828488,"request_tps":0.06870339197093095,"input_tps":4502.545496206931,"output_tps":8.794034172279162,"total_tps":4511.33953037921,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":120625.26360049378,"ttft_p95_ms":230919.13024774985,"ttft_p99_ms":233213.97572936953,"tpot_p50_ms":847.999912252151,"tpot_p95_ms":1666.0538613161907,"tpot_p99_ms":1748.680016982211,"e2e_p50_ms":236925.28252699412,"e2e_p95_ms":241650.54931361228,"e2e_p99_ms":241654.8121647397,"itl_p50_ms":36.83545487001538,"itl_p95_ms":60.686023067682996,"itl_p99_ms":88.91421182081103,"validation_errors":[],"timestamp":"2026-07-16T22:26:09+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":587.0072881928645,"request_tps":0.06814225445674156,"input_tps":4465.770788077015,"output_tps":8.72220857046292,"total_tps":4474.492996647478,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":60264.91823955439,"ttft_p95_ms":116942.56678273669,"ttft_p99_ms":116968.76564948354,"tpot_p50_ms":419.5593314018012,"tpot_p95_ms":818.3646683727837,"tpot_p99_ms":829.4111081236225,"e2e_p50_ms":119654.99441663269,"e2e_p95_ms":121336.96261633886,"e2e_p99_ms":121339.05124852201,"itl_p50_ms":31.66217007674277,"itl_p95_ms":56.3370821531862,"itl_p99_ms":61.347583890892565,"validation_errors":[],"timestamp":"2026-07-16T22:36:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":128,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":76.5485741731245,"request_tps":0.0653180030328436,"input_tps":4280.680646760438,"output_tps":8.360704388203981,"total_tps":4289.041351148642,"mean_input_tokens":65536.0,"mean_output_tokens":128.0,"ttft_p50_ms":14596.343518933281,"ttft_p95_ms":14810.734046529979,"ttft_p99_ms":14839.107076432556,"tpot_p50_ms":27.468161299956712,"tpot_p95_ms":27.539072871046862,"tpot_p99_ms":27.552164999592144,"e2e_p50_ms":18084.800004027784,"e2e_p95_ms":18259.232654888183,"e2e_p99_ms":18286.724362093955,"itl_p50_ms":25.447878055274487,"itl_p95_ms":46.60883932374417,"itl_p99_ms":51.75249738618731,"validation_errors":[],"timestamp":"2026-07-16T22:38:22+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":128,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i65536_o128_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i65536_o128_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1179.4044030888472,"request_tps":0.06783084732470125,"input_tps":4445.362410271621,"output_tps":17.36469691512352,"total_tps":4462.727107186744,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":118520.80365340225,"ttft_p95_ms":219668.39720717393,"ttft_p99_ms":232060.81151018385,"tpot_p50_ms":456.4013590040964,"tpot_p95_ms":834.1273255841166,"tpot_p99_ms":887.1042354636113,"e2e_p50_ms":241645.95780789386,"e2e_p95_ms":242439.91744236555,"e2e_p99_ms":242443.70790595654,"itl_p50_ms":36.84396401513368,"itl_p95_ms":61.602629092521966,"itl_p99_ms":67.18409784138203,"validation_errors":[],"timestamp":"2026-07-16T22:58:38+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":608.7442546100356,"request_tps":0.06570903905388674,"input_tps":4306.307583435521,"output_tps":16.821513997795005,"total_tps":4323.129097433316,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":60408.123059547506,"ttft_p95_ms":116816.32042041747,"ttft_p99_ms":117158.49739520112,"tpot_p50_ms":225.85036875497477,"tpot_p95_ms":424.8813510864662,"tpot_p99_ms":430.2625958223681,"e2e_p50_ms":123890.77430358157,"e2e_p95_ms":125879.0796181187,"e2e_p99_ms":125880.81930729328,"itl_p50_ms":31.708508962765336,"itl_p95_ms":55.004988797009,"itl_p99_ms":60.499521191231906,"validation_errors":[],"timestamp":"2026-07-16T23:09:22+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":256,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":94.3989737580996,"request_tps":0.052966677506607864,"input_tps":3471.224177073053,"output_tps":13.559469441691613,"total_tps":3484.7836465147443,"mean_input_tokens":65536.0,"mean_output_tokens":256.0,"ttft_p50_ms":14599.749388871714,"ttft_p95_ms":14722.214041789994,"ttft_p99_ms":14730.883328346536,"tpot_p50_ms":27.881732368476545,"tpot_p95_ms":27.903486282619486,"tpot_p99_ms":27.905507876263826,"e2e_p50_ms":21712.56085904315,"e2e_p95_ms":21806.284560635686,"e2e_p99_ms":21809.799600169063,"itl_p50_ms":25.44583403505385,"itl_p95_ms":49.89882283844054,"itl_p99_ms":52.76037988718599,"validation_errors":[],"timestamp":"2026-07-16T23:11:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":256,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i65536_o256_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i65536_o256_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1229.7762659930158,"request_tps":0.0650524832949202,"input_tps":4263.27954521589,"output_tps":33.30687144699914,"total_tps":4296.586416662889,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":117602.89832751732,"ttft_p95_ms":220174.25819333873,"ttft_p99_ms":232176.57839990687,"tpot_p50_ms":250.0348996674941,"tpot_p95_ms":436.4520301115598,"tpot_p99_ms":462.66546465371283,"e2e_p50_ms":251397.16331893578,"e2e_p95_ms":252849.9990378623,"e2e_p99_ms":252851.82005017297,"itl_p50_ms":36.86228208243847,"itl_p95_ms":60.973369050771,"itl_p99_ms":67.14509757934138,"validation_errors":[],"timestamp":"2026-07-16T23:32:37+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":512,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":652.6434917969164,"request_tps":0.061289203834498415,"input_tps":4016.649262497688,"output_tps":31.38007236326319,"total_tps":4048.029334860951,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":60481.61956341937,"ttft_p95_ms":116843.18738668226,"ttft_p99_ms":117324.80349385645,"tpot_p50_ms":131.188325970543,"tpot_p95_ms":229.34224289243244,"tpot_p99_ms":231.58556326105642,"e2e_p50_ms":132822.63546145987,"e2e_p95_ms":134675.67310340237,"e2e_p99_ms":134677.4176377803,"itl_p50_ms":31.73411381430924,"itl_p95_ms":53.03707402199507,"itl_p99_ms":61.14733610767871,"validation_errors":[],"timestamp":"2026-07-16T23:44:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":512,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":130.4824561579153,"request_tps":0.03831932772593422,"input_tps":2511.295461846825,"output_tps":19.619495795678322,"total_tps":2530.9149576425034,"mean_input_tokens":65536.0,"mean_output_tokens":512.0,"ttft_p50_ms":14583.937218878418,"ttft_p95_ms":14795.338988443837,"ttft_p99_ms":14826.081829732284,"tpot_p50_ms":27.982182919691134,"tpot_p95_ms":28.00536987685221,"tpot_p99_ms":28.00938046322279,"e2e_p50_ms":28872.23194888793,"e2e_p95_ms":29090.035974793136,"e2e_p99_ms":29125.418794117868,"itl_p50_ms":25.67037590779364,"itl_p95_ms":50.41482762899249,"itl_p99_ms":53.483981639146805,"validation_errors":[],"timestamp":"2026-07-16T23:46:51+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":512,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i65536_o512_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i65536_o512_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":80,"failed":0,"duration_s":1330.2602253910154,"request_tps":0.06013860932847547,"input_tps":3941.2439009509685,"output_tps":61.58193595235888,"total_tps":4002.8258369033274,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":118513.18043493666,"ttft_p95_ms":230933.78256635042,"ttft_p99_ms":232188.58536266722,"tpot_p50_ms":137.7084055620283,"tpot_p95_ms":237.46356306838157,"tpot_p99_ms":250.69030466432596,"e2e_p50_ms":271416.1142479861,"e2e_p95_ms":272710.0276091136,"e2e_p99_ms":272711.17356425617,"itl_p50_ms":36.84138902463019,"itl_p95_ms":61.22844358906149,"itl_p99_ms":67.15500121004875,"validation_errors":[],"timestamp":"2026-07-17T00:09:38+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":1024,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i65536_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":40,"failed":0,"duration_s":739.3616181469988,"request_tps":0.054100725569510504,"input_tps":3545.5451509234404,"output_tps":55.399142983178756,"total_tps":3600.9442939066194,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":60294.08791498281,"ttft_p95_ms":116841.35833900655,"ttft_p99_ms":116967.29428508785,"tpot_p50_ms":82.37472311334132,"tpot_p95_ms":131.37778040708542,"tpot_p99_ms":132.5520263926535,"e2e_p50_ms":150254.07721009105,"e2e_p95_ms":151760.6172351283,"e2e_p99_ms":151763.06750814896,"itl_p50_ms":31.72238008119166,"itl_p95_ms":53.61441783607006,"itl_p99_ms":61.79662807378918,"validation_errors":[],"timestamp":"2026-07-17T00:22:33+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":1024,"concurrency":8,"num_prompts":40,"warmup_requests":8,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c8_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c8_i65536_o1024_a1.log"}
|
||||||
|
{"status":"COMPLETED","completed":5,"failed":0,"duration_s":201.46440248796716,"request_tps":0.024818280243323058,"input_tps":1626.49081402642,"output_tps":25.41391896916281,"total_tps":1651.9047329955827,"mean_input_tokens":65536.0,"mean_output_tokens":1024.0,"ttft_p50_ms":14628.885317826644,"ttft_p95_ms":14868.979735858738,"ttft_p99_ms":14913.150340802968,"tpot_p50_ms":27.836642022547792,"tpot_p95_ms":27.844008100505842,"tpot_p99_ms":27.84499085022142,"e2e_p50_ms":43054.00717793964,"e2e_p95_ms":43330.51605997607,"e2e_p99_ms":43373.7068128027,"itl_p50_ms":25.475302943959832,"itl_p95_ms":50.770317669957876,"itl_p99_ms":54.30494689382613,"validation_errors":[],"timestamp":"2026-07-17T00:26:29+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":1024,"concurrency":1,"num_prompts":5,"warmup_requests":1,"attempt":1,"gain_pct":null,"plateau_streak":0,"error_type":"","raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c1_i65536_o1024_a1.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c1_i65536_o1024_a1.log"}
|
||||||
|
{"timestamp":"2026-07-17T01:02:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":131072,"osl":128,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":2,"status":"FAILED","completed":0,"failed":80,"error_type":"benchmark rc=124","validation_errors":[],"gain_pct":null,"plateau_streak":0,"raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i131072_o128_a2.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i131072_o128_a2.log"}
|
||||||
|
{"timestamp":"2026-07-17T02:10:35+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":131072,"osl":256,"concurrency":16,"num_prompts":80,"warmup_requests":16,"attempt":2,"status":"FAILED","completed":0,"failed":80,"error_type":"benchmark rc=124","validation_errors":[],"gain_pct":null,"plateau_streak":0,"raw_file":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/raw_outputs/sglang_adaptive_c16_i131072_o256_a2.jsonl","detail_log":"/data1/yy/sskj/experiments/p800/dsv4_p800_sglang_tp_dp_matrix/adaptive_results/adaptive_20260716-074243/tp8_dp1/logs/sglang_c16_i131072_o256_a2.log"}
|
||||||
|
|||||||
@ -1 +1,47 @@
|
|||||||
{"timestamp":"2026-07-16T07:48:44+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":1,"search_cap":512,"max_successful_concurrency":16,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":16,"best_total_tps":2233.1625760825486,"last_total_tps":2233.1625760825486}
|
{"timestamp":"2026-07-16T07:48:44+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":1,"search_cap":512,"max_successful_concurrency":16,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":16,"best_total_tps":2233.1625760825486,"last_total_tps":2233.1625760825486}
|
||||||
|
{"timestamp":"2026-07-16T07:54:03+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":48,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":48,"best_total_tps":2994.9145225702982,"last_total_tps":2994.9145225702982}
|
||||||
|
{"timestamp":"2026-07-16T08:02:57+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":48,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":48,"best_total_tps":1894.263479895674,"last_total_tps":1894.263479895674}
|
||||||
|
{"timestamp":"2026-07-16T08:19:12+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":48,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":48,"best_total_tps":1270.3578826082223,"last_total_tps":1270.3578826082223}
|
||||||
|
{"timestamp":"2026-07-16T08:50:41+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":48,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":48,"best_total_tps":938.3875354487201,"last_total_tps":938.3875354487201}
|
||||||
|
{"timestamp":"2026-07-16T09:44:20+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":1024,"osl":4096,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":487.1207589944629,"last_total_tps":272.97053779597}
|
||||||
|
{"timestamp":"2026-07-16T09:46:49+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":5276.430317217383,"last_total_tps":3991.4044121212137}
|
||||||
|
{"timestamp":"2026-07-16T09:50:14+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":3654.27555715578,"last_total_tps":2642.4843178928554}
|
||||||
|
{"timestamp":"2026-07-16T09:55:31+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":2302.8342327716664,"last_total_tps":1632.8914416222033}
|
||||||
|
{"timestamp":"2026-07-16T10:04:24+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":1449.0672823499715,"last_total_tps":989.1396953244687}
|
||||||
|
{"timestamp":"2026-07-16T10:20:38+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":919.5235579896186,"last_total_tps":622.97916579306}
|
||||||
|
{"timestamp":"2026-07-16T10:51:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":4096,"osl":4096,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":636.5331301353165,"last_total_tps":429.68665622424965}
|
||||||
|
{"timestamp":"2026-07-16T10:58:35+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":6149.069264131796,"last_total_tps":2072.6357920596456}
|
||||||
|
{"timestamp":"2026-07-16T11:07:42+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4922.27619638361,"last_total_tps":1374.3485383101208}
|
||||||
|
{"timestamp":"2026-07-16T11:20:34+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":3457.304361342664,"last_total_tps":840.1799665951539}
|
||||||
|
{"timestamp":"2026-07-16T11:40:58+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":2203.3814838524254,"last_total_tps":479.3679364784214}
|
||||||
|
{"timestamp":"2026-07-16T12:16:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":16384,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":1322.1810796622076,"last_total_tps":269.1649593728095}
|
||||||
|
{"timestamp":"2026-07-16T12:39:17+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":6860.104148991479,"last_total_tps":3001.706937330212}
|
||||||
|
{"timestamp":"2026-07-16T13:04:16+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":6322.324656264061,"last_total_tps":2534.2482875187998}
|
||||||
|
{"timestamp":"2026-07-16T13:33:26+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":5426.021563663019,"last_total_tps":1933.302825215971}
|
||||||
|
{"timestamp":"2026-07-16T14:11:00+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":65536,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":8,"best_total_tps":4250.026326552517,"last_total_tps":1299.0787444904488}
|
||||||
|
{"timestamp":"2026-07-16T14:59:34+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":8,"best_total_tps":6414.4163774899025,"last_total_tps":2895.647145362982}
|
||||||
|
{"timestamp":"2026-07-16T16:03:27+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":256,"status":"FAILED","stop_reason":"BENCHMARK_FAILED","tested_points":1,"search_cap":512,"max_successful_concurrency":0,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":null,"best_total_tps":null,"last_total_tps":null}
|
||||||
|
{"timestamp":"2026-07-16T17:11:50+00:00","engine":"sglang","tp":4,"dp":2,"mark":"Y","isl":131072,"osl":512,"status":"FAILED","stop_reason":"BENCHMARK_FAILED","tested_points":1,"search_cap":512,"max_successful_concurrency":0,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":null,"best_total_tps":null,"last_total_tps":null}
|
||||||
|
{"timestamp":"2026-07-16T17:23:16+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":2,"search_cap":512,"max_successful_concurrency":8,"saturation_concurrency":null,"stop_probe_concurrency":8,"best_tps_concurrency":16,"best_total_tps":2252.3024943648124,"last_total_tps":1988.28275981793}
|
||||||
|
{"timestamp":"2026-07-16T17:25:59+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":2,"search_cap":512,"max_successful_concurrency":32,"saturation_concurrency":null,"stop_probe_concurrency":32,"best_tps_concurrency":32,"best_total_tps":2807.7871232493126,"last_total_tps":2807.7871232493126}
|
||||||
|
{"timestamp":"2026-07-16T17:39:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":48,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":32,"best_total_tps":2109.996176616557,"last_total_tps":684.1363844999557}
|
||||||
|
{"timestamp":"2026-07-16T17:47:25+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":2,"search_cap":512,"max_successful_concurrency":32,"saturation_concurrency":null,"stop_probe_concurrency":32,"best_tps_concurrency":32,"best_total_tps":1390.4790149026294,"last_total_tps":1390.4790149026294}
|
||||||
|
{"timestamp":"2026-07-16T18:01:59+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":2,"search_cap":512,"max_successful_concurrency":32,"saturation_concurrency":null,"stop_probe_concurrency":32,"best_tps_concurrency":32,"best_total_tps":1079.9126577555764,"last_total_tps":1079.9126577555764}
|
||||||
|
{"timestamp":"2026-07-16T19:35:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":1024,"osl":4096,"status":"FAILED","stop_reason":"BENCHMARK_FAILED","tested_points":3,"search_cap":512,"max_successful_concurrency":32,"saturation_concurrency":null,"stop_probe_concurrency":48,"best_tps_concurrency":32,"best_total_tps":911.6872643258514,"last_total_tps":911.6872643258514}
|
||||||
|
{"timestamp":"2026-07-16T19:44:08+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4128.998328888893,"last_total_tps":1151.1056276859294}
|
||||||
|
{"timestamp":"2026-07-16T19:48:14+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4421.1758211534125,"last_total_tps":616.807087254842}
|
||||||
|
{"timestamp":"2026-07-16T19:54:24+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":2909.7747901626167,"last_total_tps":332.27345718922555}
|
||||||
|
{"timestamp":"2026-07-16T20:04:39+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":1863.3608864757555,"last_total_tps":186.24036663353294}
|
||||||
|
{"timestamp":"2026-07-16T20:23:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":1206.8940867936158,"last_total_tps":112.28613758457962}
|
||||||
|
{"timestamp":"2026-07-16T20:57:57+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":4096,"osl":4096,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":838.0690279850638,"last_total_tps":74.72994974172046}
|
||||||
|
{"timestamp":"2026-07-16T21:06:13+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":5144.723678168144,"last_total_tps":2733.5067539919205}
|
||||||
|
{"timestamp":"2026-07-16T21:15:29+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4772.803886972835,"last_total_tps":1759.8744462055809}
|
||||||
|
{"timestamp":"2026-07-16T21:26:50+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4137.621478884017,"last_total_tps":1033.4981159438357}
|
||||||
|
{"timestamp":"2026-07-16T21:42:20+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":3290.996804492577,"last_total_tps":577.0193960995003}
|
||||||
|
{"timestamp":"2026-07-16T22:06:10+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":16384,"osl":2048,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_BACKOFF_FOUND","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":2399.4052794078843,"last_total_tps":319.32407850271136}
|
||||||
|
{"timestamp":"2026-07-16T22:38:22+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":128,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4511.33953037921,"last_total_tps":4289.041351148642}
|
||||||
|
{"timestamp":"2026-07-16T23:11:31+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":256,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4462.727107186744,"last_total_tps":3484.7836465147443}
|
||||||
|
{"timestamp":"2026-07-16T23:46:51+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":512,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4296.586416662889,"last_total_tps":2530.9149576425034}
|
||||||
|
{"timestamp":"2026-07-17T00:26:29+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":65536,"osl":1024,"status":"TTFT_BOUNDARY","stop_reason":"TTFT_SLO_EXCEEDED","tested_points":3,"search_cap":512,"max_successful_concurrency":1,"saturation_concurrency":null,"stop_probe_concurrency":1,"best_tps_concurrency":16,"best_total_tps":4002.8258369033274,"last_total_tps":1651.9047329955827}
|
||||||
|
{"timestamp":"2026-07-17T01:02:05+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":131072,"osl":128,"status":"FAILED","stop_reason":"BENCHMARK_FAILED","tested_points":1,"search_cap":512,"max_successful_concurrency":0,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":null,"best_total_tps":null,"last_total_tps":null}
|
||||||
|
{"timestamp":"2026-07-17T02:10:35+00:00","engine":"sglang","tp":8,"dp":1,"mark":"Y","isl":131072,"osl":256,"status":"FAILED","stop_reason":"BENCHMARK_FAILED","tested_points":1,"search_cap":512,"max_successful_concurrency":0,"saturation_concurrency":null,"stop_probe_concurrency":16,"best_tps_concurrency":null,"best_total_tps":null,"last_total_tps":null}
|
||||||
|
|||||||
@ -0,0 +1,47 @@
|
|||||||
|
{"timestamp": "2026-07-16T07:48:44+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 1, "search_cap": 512, "max_successful_concurrency": 16, "saturation_concurrency": null, "stop_probe_concurrency": 16, "best_tps_concurrency": 16, "best_total_tps": 2233.1625760825486, "last_total_tps": 2233.1625760825486}
|
||||||
|
{"timestamp": "2026-07-16T07:54:03+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 48, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 48, "best_total_tps": 2994.9145225702982, "last_total_tps": 2994.9145225702982}
|
||||||
|
{"timestamp": "2026-07-16T08:02:57+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 48, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 48, "best_total_tps": 1894.263479895674, "last_total_tps": 1894.263479895674}
|
||||||
|
{"timestamp": "2026-07-16T08:19:12+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 48, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 48, "best_total_tps": 1270.3578826082223, "last_total_tps": 1270.3578826082223}
|
||||||
|
{"timestamp": "2026-07-16T08:50:41+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 48, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 48, "best_total_tps": 938.3875354487201, "last_total_tps": 938.3875354487201}
|
||||||
|
{"timestamp": "2026-07-16T09:44:20+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 1024, "osl": 4096, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 487.1207589944629, "last_total_tps": 272.97053779597}
|
||||||
|
{"timestamp": "2026-07-16T09:46:49+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 5276.430317217383, "last_total_tps": 3991.4044121212137}
|
||||||
|
{"timestamp": "2026-07-16T09:50:14+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 3654.27555715578, "last_total_tps": 2642.4843178928554}
|
||||||
|
{"timestamp": "2026-07-16T09:55:31+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 2302.8342327716664, "last_total_tps": 1632.8914416222033}
|
||||||
|
{"timestamp": "2026-07-16T10:04:24+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 1449.0672823499715, "last_total_tps": 989.1396953244687}
|
||||||
|
{"timestamp": "2026-07-16T10:20:38+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 919.5235579896186, "last_total_tps": 622.97916579306}
|
||||||
|
{"timestamp": "2026-07-16T10:51:17+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 4096, "osl": 4096, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 636.5331301353165, "last_total_tps": 429.68665622424965}
|
||||||
|
{"timestamp": "2026-07-16T10:58:35+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 16384, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 6149.069264131796, "last_total_tps": 2072.6357920596456}
|
||||||
|
{"timestamp": "2026-07-16T11:07:42+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 16384, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4922.27619638361, "last_total_tps": 1374.3485383101208}
|
||||||
|
{"timestamp": "2026-07-16T11:20:34+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 16384, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 3457.304361342664, "last_total_tps": 840.1799665951539}
|
||||||
|
{"timestamp": "2026-07-16T11:40:58+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 16384, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 2203.3814838524254, "last_total_tps": 479.3679364784214}
|
||||||
|
{"timestamp": "2026-07-16T12:16:16+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 16384, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 1322.1810796622076, "last_total_tps": 269.1649593728095}
|
||||||
|
{"timestamp": "2026-07-16T12:39:17+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 65536, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 6860.104148991479, "last_total_tps": 3001.706937330212}
|
||||||
|
{"timestamp": "2026-07-16T13:04:16+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 65536, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 6322.324656264061, "last_total_tps": 2534.2482875187998}
|
||||||
|
{"timestamp": "2026-07-16T13:33:26+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 65536, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 5426.021563663019, "last_total_tps": 1933.302825215971}
|
||||||
|
{"timestamp": "2026-07-16T14:11:00+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 65536, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 8, "best_total_tps": 4250.026326552517, "last_total_tps": 1299.0787444904488}
|
||||||
|
{"timestamp": "2026-07-16T14:59:34+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 131072, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 8, "best_total_tps": 6414.4163774899025, "last_total_tps": 2895.647145362982}
|
||||||
|
{"timestamp": "2026-07-16T16:03:27+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 131072, "osl": 256, "status": "FAILED", "stop_reason": "BENCHMARK_FAILED", "tested_points": 1, "search_cap": 512, "max_successful_concurrency": 0, "saturation_concurrency": null, "stop_probe_concurrency": 16, "best_tps_concurrency": null, "best_total_tps": null, "last_total_tps": null}
|
||||||
|
{"timestamp": "2026-07-16T17:11:50+00:00", "engine": "sglang", "tp": 4, "dp": 2, "mark": "Y", "isl": 131072, "osl": 512, "status": "FAILED", "stop_reason": "BENCHMARK_FAILED", "tested_points": 1, "search_cap": 512, "max_successful_concurrency": 0, "saturation_concurrency": null, "stop_probe_concurrency": 16, "best_tps_concurrency": null, "best_total_tps": null, "last_total_tps": null}
|
||||||
|
{"timestamp": "2026-07-16T17:23:16+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 8, "saturation_concurrency": null, "stop_probe_concurrency": 8, "best_tps_concurrency": 16, "best_total_tps": 2252.3024943648124, "last_total_tps": 1988.28275981793}
|
||||||
|
{"timestamp": "2026-07-16T17:25:59+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 32, "saturation_concurrency": null, "stop_probe_concurrency": 32, "best_tps_concurrency": 32, "best_total_tps": 2807.7871232493126, "last_total_tps": 2807.7871232493126}
|
||||||
|
{"timestamp": "2026-07-16T17:39:31+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 48, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 32, "best_total_tps": 2109.996176616557, "last_total_tps": 684.1363844999557}
|
||||||
|
{"timestamp": "2026-07-16T17:47:25+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 32, "saturation_concurrency": null, "stop_probe_concurrency": 32, "best_tps_concurrency": 32, "best_total_tps": 1390.4790149026294, "last_total_tps": 1390.4790149026294}
|
||||||
|
{"timestamp": "2026-07-16T18:01:59+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 2, "search_cap": 512, "max_successful_concurrency": 32, "saturation_concurrency": null, "stop_probe_concurrency": 32, "best_tps_concurrency": 32, "best_total_tps": 1079.9126577555764, "last_total_tps": 1079.9126577555764}
|
||||||
|
{"timestamp": "2026-07-16T19:35:31+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 1024, "osl": 4096, "status": "FAILED", "stop_reason": "BENCHMARK_FAILED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 32, "saturation_concurrency": null, "stop_probe_concurrency": 48, "best_tps_concurrency": 32, "best_total_tps": 911.6872643258514, "last_total_tps": 911.6872643258514}
|
||||||
|
{"timestamp": "2026-07-16T19:44:08+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4128.998328888893, "last_total_tps": 1151.1056276859294}
|
||||||
|
{"timestamp": "2026-07-16T19:48:14+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4421.1758211534125, "last_total_tps": 616.807087254842}
|
||||||
|
{"timestamp": "2026-07-16T19:54:24+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 2909.7747901626167, "last_total_tps": 332.27345718922555}
|
||||||
|
{"timestamp": "2026-07-16T20:04:39+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 1863.3608864757555, "last_total_tps": 186.24036663353294}
|
||||||
|
{"timestamp": "2026-07-16T20:23:05+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 1206.8940867936158, "last_total_tps": 112.28613758457962}
|
||||||
|
{"timestamp": "2026-07-16T20:57:57+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 4096, "osl": 4096, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 838.0690279850638, "last_total_tps": 74.72994974172046}
|
||||||
|
{"timestamp": "2026-07-16T21:06:13+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 16384, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 5144.723678168144, "last_total_tps": 2733.5067539919205}
|
||||||
|
{"timestamp": "2026-07-16T21:15:29+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 16384, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4772.803886972835, "last_total_tps": 1759.8744462055809}
|
||||||
|
{"timestamp": "2026-07-16T21:26:50+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 16384, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4137.621478884017, "last_total_tps": 1033.4981159438357}
|
||||||
|
{"timestamp": "2026-07-16T21:42:20+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 16384, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 3290.996804492577, "last_total_tps": 577.0193960995003}
|
||||||
|
{"timestamp": "2026-07-16T22:06:10+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 16384, "osl": 2048, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_BACKOFF_FOUND", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 2399.4052794078843, "last_total_tps": 319.32407850271136}
|
||||||
|
{"timestamp": "2026-07-16T22:38:22+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 65536, "osl": 128, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4511.33953037921, "last_total_tps": 4289.041351148642}
|
||||||
|
{"timestamp": "2026-07-16T23:11:31+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 65536, "osl": 256, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4462.727107186744, "last_total_tps": 3484.7836465147443}
|
||||||
|
{"timestamp": "2026-07-16T23:46:51+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 65536, "osl": 512, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4296.586416662889, "last_total_tps": 2530.9149576425034}
|
||||||
|
{"timestamp": "2026-07-17T00:26:29+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 65536, "osl": 1024, "status": "TTFT_BOUNDARY", "stop_reason": "TTFT_SLO_EXCEEDED", "tested_points": 3, "search_cap": 512, "max_successful_concurrency": 1, "saturation_concurrency": null, "stop_probe_concurrency": 1, "best_tps_concurrency": 16, "best_total_tps": 4002.8258369033274, "last_total_tps": 1651.9047329955827}
|
||||||
|
{"timestamp": "2026-07-17T01:02:05+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 131072, "osl": 128, "status": "FAILED", "stop_reason": "BENCHMARK_FAILED", "tested_points": 1, "search_cap": 512, "max_successful_concurrency": 0, "saturation_concurrency": null, "stop_probe_concurrency": 16, "best_tps_concurrency": null, "best_total_tps": null, "last_total_tps": null}
|
||||||
|
{"timestamp": "2026-07-17T02:10:35+00:00", "engine": "sglang", "tp": 8, "dp": 1, "mark": "Y", "isl": 131072, "osl": 256, "status": "FAILED", "stop_reason": "BENCHMARK_FAILED", "tested_points": 1, "search_cap": 512, "max_successful_concurrency": 0, "saturation_concurrency": null, "stop_probe_concurrency": 16, "best_tps_concurrency": null, "best_total_tps": null, "last_total_tps": null}
|
||||||
@ -0,0 +1,53 @@
|
|||||||
|
# Adaptive concurrency search summary
|
||||||
|
|
||||||
|
| Engine | TP | DP | ISL | OSL | Stop | Saturation C | Best TPS C | Best Total TPS | Max successful C |
|
||||||
|
|---|---:|---:|---:|---:|---|---:|---:|---:|---:|
|
||||||
|
| sglang | 4 | 2 | 1024 | 128 | TTFT_SLO_EXCEEDED | None | 16 | 2233.1625760825486 | 16 |
|
||||||
|
| sglang | 4 | 2 | 1024 | 256 | TTFT_SLO_EXCEEDED | None | 48 | 2994.9145225702982 | 48 |
|
||||||
|
| sglang | 4 | 2 | 1024 | 512 | TTFT_SLO_EXCEEDED | None | 48 | 1894.263479895674 | 48 |
|
||||||
|
| sglang | 4 | 2 | 1024 | 1024 | TTFT_SLO_EXCEEDED | None | 48 | 1270.3578826082223 | 48 |
|
||||||
|
| sglang | 4 | 2 | 1024 | 2048 | TTFT_SLO_EXCEEDED | None | 48 | 938.3875354487201 | 48 |
|
||||||
|
| sglang | 4 | 2 | 1024 | 4096 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 487.1207589944629 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 128 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 5276.430317217383 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 256 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 3654.27555715578 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 512 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 2302.8342327716664 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 1024 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 1449.0672823499715 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 2048 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 919.5235579896186 | 8 |
|
||||||
|
| sglang | 4 | 2 | 4096 | 4096 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 636.5331301353165 | 8 |
|
||||||
|
| sglang | 4 | 2 | 16384 | 128 | TTFT_SLO_EXCEEDED | None | 16 | 6149.069264131796 | 1 |
|
||||||
|
| sglang | 4 | 2 | 16384 | 256 | TTFT_SLO_EXCEEDED | None | 16 | 4922.27619638361 | 1 |
|
||||||
|
| sglang | 4 | 2 | 16384 | 512 | TTFT_SLO_EXCEEDED | None | 16 | 3457.304361342664 | 1 |
|
||||||
|
| sglang | 4 | 2 | 16384 | 1024 | TTFT_SLO_EXCEEDED | None | 16 | 2203.3814838524254 | 1 |
|
||||||
|
| sglang | 4 | 2 | 16384 | 2048 | TTFT_SLO_EXCEEDED | None | 16 | 1322.1810796622076 | 1 |
|
||||||
|
| sglang | 4 | 2 | 65536 | 128 | TTFT_SLO_EXCEEDED | None | 16 | 6860.104148991479 | 1 |
|
||||||
|
| sglang | 4 | 2 | 65536 | 256 | TTFT_SLO_EXCEEDED | None | 16 | 6322.324656264061 | 1 |
|
||||||
|
| sglang | 4 | 2 | 65536 | 512 | TTFT_SLO_EXCEEDED | None | 16 | 5426.021563663019 | 1 |
|
||||||
|
| sglang | 4 | 2 | 65536 | 1024 | TTFT_SLO_EXCEEDED | None | 8 | 4250.026326552517 | 1 |
|
||||||
|
| sglang | 4 | 2 | 131072 | 128 | TTFT_SLO_EXCEEDED | None | 8 | 6414.4163774899025 | 1 |
|
||||||
|
| sglang | 4 | 2 | 131072 | 256 | BENCHMARK_FAILED | None | None | None | 0 |
|
||||||
|
| sglang | 4 | 2 | 131072 | 512 | BENCHMARK_FAILED | None | None | None | 0 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 128 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 2252.3024943648124 | 8 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 256 | TTFT_SLO_EXCEEDED | None | 32 | 2807.7871232493126 | 32 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 512 | TTFT_SLO_EXCEEDED | None | 32 | 2109.996176616557 | 48 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 1024 | TTFT_SLO_EXCEEDED | None | 32 | 1390.4790149026294 | 32 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 2048 | TTFT_SLO_EXCEEDED | None | 32 | 1079.9126577555764 | 32 |
|
||||||
|
| sglang | 8 | 1 | 1024 | 4096 | BENCHMARK_FAILED | None | 32 | 911.6872643258514 | 32 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 128 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 4128.998328888893 | 1 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 256 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 4421.1758211534125 | 1 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 512 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 2909.7747901626167 | 1 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 1024 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 1863.3608864757555 | 1 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 2048 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 1206.8940867936158 | 1 |
|
||||||
|
| sglang | 8 | 1 | 4096 | 4096 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 838.0690279850638 | 1 |
|
||||||
|
| sglang | 8 | 1 | 16384 | 128 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 5144.723678168144 | 1 |
|
||||||
|
| sglang | 8 | 1 | 16384 | 256 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 4772.803886972835 | 1 |
|
||||||
|
| sglang | 8 | 1 | 16384 | 512 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 4137.621478884017 | 1 |
|
||||||
|
| sglang | 8 | 1 | 16384 | 1024 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 3290.996804492577 | 1 |
|
||||||
|
| sglang | 8 | 1 | 16384 | 2048 | TTFT_SLO_BACKOFF_FOUND | None | 16 | 2399.4052794078843 | 1 |
|
||||||
|
| sglang | 8 | 1 | 65536 | 128 | TTFT_SLO_EXCEEDED | None | 16 | 4511.33953037921 | 1 |
|
||||||
|
| sglang | 8 | 1 | 65536 | 256 | TTFT_SLO_EXCEEDED | None | 16 | 4462.727107186744 | 1 |
|
||||||
|
| sglang | 8 | 1 | 65536 | 512 | TTFT_SLO_EXCEEDED | None | 16 | 4296.586416662889 | 1 |
|
||||||
|
| sglang | 8 | 1 | 65536 | 1024 | TTFT_SLO_EXCEEDED | None | 16 | 4002.8258369033274 | 1 |
|
||||||
|
| sglang | 8 | 1 | 131072 | 128 | BENCHMARK_FAILED | None | None | None | 0 |
|
||||||
|
| sglang | 8 | 1 | 131072 | 256 | BENCHMARK_FAILED | None | None | None | 0 |
|
||||||
|
|
||||||
|
`Saturation C` is the first point in the final low-gain streak. `Best TPS C` is the tested point with the highest observed Total TPS.
|
||||||
@ -12,11 +12,16 @@
|
|||||||
"start_concurrency": 16,
|
"start_concurrency": 16,
|
||||||
"max_concurrency": 512,
|
"max_concurrency": 512,
|
||||||
"multiplier": 2,
|
"multiplier": 2,
|
||||||
|
"initial_backoff_concurrencies": [
|
||||||
|
8,
|
||||||
|
1
|
||||||
|
],
|
||||||
"num_prompts_multiplier": 5,
|
"num_prompts_multiplier": 5,
|
||||||
"min_tps_gain_pct": 2.0,
|
"min_tps_gain_pct": 2.0,
|
||||||
"plateau_patience": 2,
|
"plateau_patience": 2,
|
||||||
"warmup_max_requests": 0,
|
"warmup_max_requests": 0,
|
||||||
"ttft_slo_ms": 4000.0,
|
"ttft_slo_ms": 4000.0,
|
||||||
"enable_ttft_slo_stop": 1
|
"enable_ttft_slo_stop": 1,
|
||||||
|
"ttft_group_skip_ms": 8000.0
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 239.2404650580138,
|
||||||
|
"request_tps": 0.3343915920770371,
|
||||||
|
"input_tps": 342.416990286886,
|
||||||
|
"output_tps": 342.416990286886,
|
||||||
|
"total_tps": 684.833980573772,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 635.5562334647402,
|
||||||
|
"ttft_p95_ms": 1903.8669666741043,
|
||||||
|
"ttft_p99_ms": 1906.7863759305328,
|
||||||
|
"tpot_p50_ms": 45.73389088913875,
|
||||||
|
"tpot_p95_ms": 46.81248970833659,
|
||||||
|
"tpot_p99_ms": 47.236905306400615,
|
||||||
|
"e2e_p50_ms": 47589.559288928285,
|
||||||
|
"e2e_p95_ms": 48763.24818015564,
|
||||||
|
"e2e_p99_ms": 49025.48799880082,
|
||||||
|
"itl_p50_ms": 38.22040546219796,
|
||||||
|
"itl_p95_ms": 62.13192297145724,
|
||||||
|
"itl_p99_ms": 69.20204504393043,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 473.66198480315506,
|
||||||
|
"request_tps": 0.16889681369140588,
|
||||||
|
"input_tps": 172.95033721999962,
|
||||||
|
"output_tps": 345.90067443999925,
|
||||||
|
"total_tps": 518.8510116599988,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 1107.0592959877104,
|
||||||
|
"ttft_p95_ms": 1914.7031036205587,
|
||||||
|
"ttft_p99_ms": 1937.422300640028,
|
||||||
|
"tpot_p50_ms": 45.60600052494192,
|
||||||
|
"tpot_p95_ms": 46.180353910758,
|
||||||
|
"tpot_p99_ms": 46.185254772231495,
|
||||||
|
"e2e_p50_ms": 94571.73220254481,
|
||||||
|
"e2e_p95_ms": 95557.28795382893,
|
||||||
|
"e2e_p99_ms": 95606.2740921299,
|
||||||
|
"itl_p50_ms": 38.73952745925635,
|
||||||
|
"itl_p95_ms": 62.4737826641649,
|
||||||
|
"itl_p99_ms": 67.41200149524958,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 67.59083261387423,
|
||||||
|
"request_tps": 1.1835924622650464,
|
||||||
|
"input_tps": 1211.9986813594076,
|
||||||
|
"output_tps": 302.9996703398519,
|
||||||
|
"total_tps": 1514.9983516992595,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 1088.1021856330335,
|
||||||
|
"ttft_p95_ms": 1940.051195200067,
|
||||||
|
"ttft_p99_ms": 1945.7215382088907,
|
||||||
|
"tpot_p50_ms": 48.00720819381668,
|
||||||
|
"tpot_p95_ms": 51.906457407692194,
|
||||||
|
"tpot_p99_ms": 52.662985878384795,
|
||||||
|
"e2e_p50_ms": 13323.23444657959,
|
||||||
|
"e2e_p95_ms": 14546.717717882711,
|
||||||
|
"e2e_p99_ms": 14597.288672290742,
|
||||||
|
"itl_p50_ms": 38.109477958641946,
|
||||||
|
"itl_p95_ms": 62.32210020534694,
|
||||||
|
"itl_p99_ms": 91.81069683981994,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 840.8592580729164,
|
||||||
|
"request_tps": 0.09514077324110604,
|
||||||
|
"input_tps": 97.42415179889258,
|
||||||
|
"output_tps": 389.6966071955703,
|
||||||
|
"total_tps": 487.1207589944629,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 4096.0,
|
||||||
|
"ttft_p50_ms": 972.442637081258,
|
||||||
|
"ttft_p95_ms": 5176.270531245973,
|
||||||
|
"ttft_p99_ms": 5179.259422505274,
|
||||||
|
"tpot_p50_ms": 40.55560706117929,
|
||||||
|
"tpot_p95_ms": 40.93768200460964,
|
||||||
|
"tpot_p99_ms": 40.989130662076434,
|
||||||
|
"e2e_p50_ms": 167707.99649693072,
|
||||||
|
"e2e_p95_ms": 171209.72607890144,
|
||||||
|
"e2e_p99_ms": 171213.10957846232,
|
||||||
|
"itl_p50_ms": 38.21910894475877,
|
||||||
|
"itl_p95_ms": 56.424582388717674,
|
||||||
|
"itl_p99_ms": 61.033719554543495,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 123.35641185496934,
|
||||||
|
"request_tps": 0.6485272941795385,
|
||||||
|
"input_tps": 664.0919492398474,
|
||||||
|
"output_tps": 332.0459746199237,
|
||||||
|
"total_tps": 996.137923859771,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 1815.1043795514852,
|
||||||
|
"ttft_p95_ms": 2051.647847087588,
|
||||||
|
"ttft_p99_ms": 2054.4600446219556,
|
||||||
|
"tpot_p50_ms": 44.89284182270832,
|
||||||
|
"tpot_p95_ms": 47.29539701357892,
|
||||||
|
"tpot_p99_ms": 47.49259979092314,
|
||||||
|
"e2e_p50_ms": 24692.195138079114,
|
||||||
|
"e2e_p95_ms": 25352.763180364855,
|
||||||
|
"e2e_p99_ms": 25618.341889507137,
|
||||||
|
"itl_p50_ms": 37.906087934970856,
|
||||||
|
"itl_p95_ms": 61.653098394162946,
|
||||||
|
"itl_p99_ms": 66.99016668368131,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 1677.9406901570037,
|
||||||
|
"request_tps": 0.047677489716584955,
|
||||||
|
"input_tps": 6249.183932132223,
|
||||||
|
"output_tps": 6.102718683722874,
|
||||||
|
"total_tps": 6255.286650815946,
|
||||||
|
"mean_input_tokens": 131072.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 300625.9035965195,
|
||||||
|
"ttft_p95_ms": 338951.3012407697,
|
||||||
|
"ttft_p99_ms": 340203.0271626217,
|
||||||
|
"tpot_p50_ms": 41.211593476322115,
|
||||||
|
"tpot_p95_ms": 345.65387451581245,
|
||||||
|
"tpot_p99_ms": 346.94827124932925,
|
||||||
|
"e2e_p50_ms": 343158.3466755692,
|
||||||
|
"e2e_p95_ms": 344526.5441220952,
|
||||||
|
"e2e_p99_ms": 345839.6717084944,
|
||||||
|
"itl_p50_ms": 33.60768407583237,
|
||||||
|
"itl_p95_ms": 58.138947119005024,
|
||||||
|
"itl_p99_ms": 181.5207198727876,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 632.0467019469943,
|
||||||
|
"request_tps": 0.1265729253132138,
|
||||||
|
"input_tps": 2073.7708083316948,
|
||||||
|
"output_tps": 129.61067552073092,
|
||||||
|
"total_tps": 2203.3814838524254,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 80126.8644919619,
|
||||||
|
"ttft_p95_ms": 106781.9698160165,
|
||||||
|
"ttft_p99_ms": 107180.66528041381,
|
||||||
|
"tpot_p50_ms": 38.308278328050854,
|
||||||
|
"tpot_p95_ms": 42.30265493578899,
|
||||||
|
"tpot_p99_ms": 45.4902937474118,
|
||||||
|
"e2e_p50_ms": 120186.66901800316,
|
||||||
|
"e2e_p95_ms": 148021.29719536752,
|
||||||
|
"e2e_p99_ms": 148871.62709633118,
|
||||||
|
"itl_p50_ms": 34.52094295062125,
|
||||||
|
"itl_p95_ms": 51.694627082906685,
|
||||||
|
"itl_p99_ms": 59.966907631605864,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 214.8227549989242,
|
||||||
|
"request_tps": 0.37240002810875705,
|
||||||
|
"input_tps": 6101.402060533876,
|
||||||
|
"output_tps": 47.6672035979209,
|
||||||
|
"total_tps": 6149.069264131796,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 33368.671427015215,
|
||||||
|
"ttft_p95_ms": 41899.411271174904,
|
||||||
|
"ttft_p99_ms": 42125.512621703085,
|
||||||
|
"tpot_p50_ms": 52.554487114163535,
|
||||||
|
"tpot_p95_ms": 84.39758658330287,
|
||||||
|
"tpot_p99_ms": 111.94620711651932,
|
||||||
|
"e2e_p50_ms": 42495.81566103734,
|
||||||
|
"e2e_p95_ms": 49210.346029268105,
|
||||||
|
"e2e_p99_ms": 52561.92248010076,
|
||||||
|
"itl_p50_ms": 34.52568908687681,
|
||||||
|
"itl_p95_ms": 52.17893534572764,
|
||||||
|
"itl_p99_ms": 66.4443611598108,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 1115.2481476869434,
|
||||||
|
"request_tps": 0.07173291447820136,
|
||||||
|
"input_tps": 1175.2720708108511,
|
||||||
|
"output_tps": 146.9090088513564,
|
||||||
|
"total_tps": 1322.1810796622076,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 136037.42311848328,
|
||||||
|
"ttft_p95_ms": 181299.1330652847,
|
||||||
|
"ttft_p99_ms": 181444.37291203765,
|
||||||
|
"tpot_p50_ms": 37.30771111527574,
|
||||||
|
"tpot_p95_ms": 39.328867612731,
|
||||||
|
"tpot_p99_ms": 40.83525040460833,
|
||||||
|
"e2e_p50_ms": 213226.2396500446,
|
||||||
|
"e2e_p95_ms": 259703.93927409317,
|
||||||
|
"e2e_p99_ms": 260337.14725418712,
|
||||||
|
"itl_p50_ms": 34.49960285797715,
|
||||||
|
"itl_p95_ms": 51.94193584611639,
|
||||||
|
"itl_p99_ms": 60.15488663455472,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 270.44398706802167,
|
||||||
|
"request_tps": 0.2958098675711304,
|
||||||
|
"input_tps": 4846.548870285401,
|
||||||
|
"output_tps": 75.72732609820939,
|
||||||
|
"total_tps": 4922.27619638361,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 38130.55603799876,
|
||||||
|
"ttft_p95_ms": 51039.480149233714,
|
||||||
|
"ttft_p99_ms": 51170.22580615245,
|
||||||
|
"tpot_p50_ms": 44.34177505135463,
|
||||||
|
"tpot_p95_ms": 60.19466479852174,
|
||||||
|
"tpot_p99_ms": 74.01323430483028,
|
||||||
|
"e2e_p50_ms": 50331.59309858456,
|
||||||
|
"e2e_p95_ms": 65593.75085806241,
|
||||||
|
"e2e_p99_ms": 65883.99838647107,
|
||||||
|
"itl_p50_ms": 34.52300652861595,
|
||||||
|
"itl_p95_ms": 51.78628868889062,
|
||||||
|
"itl_p99_ms": 60.336364856921136,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 390.9635538929142,
|
||||||
|
"request_tps": 0.20462265396204216,
|
||||||
|
"input_tps": 3352.5375625140987,
|
||||||
|
"output_tps": 104.76679882856558,
|
||||||
|
"total_tps": 3457.304361342664,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 52149.89124494605,
|
||||||
|
"ttft_p95_ms": 69626.60835890565,
|
||||||
|
"ttft_p99_ms": 69753.87210340472,
|
||||||
|
"tpot_p50_ms": 40.293584739718284,
|
||||||
|
"tpot_p95_ms": 48.141768489123194,
|
||||||
|
"tpot_p99_ms": 54.960999020335215,
|
||||||
|
"e2e_p50_ms": 73592.7937864326,
|
||||||
|
"e2e_p95_ms": 93283.15660590306,
|
||||||
|
"e2e_p99_ms": 93629.28111733867,
|
||||||
|
"itl_p50_ms": 34.522912465035915,
|
||||||
|
"itl_p95_ms": 51.489772961940616,
|
||||||
|
"itl_p99_ms": 59.926832308992736,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 282.66458361805417,
|
||||||
|
"request_tps": 0.2830209535839788,
|
||||||
|
"input_tps": 1159.253825879977,
|
||||||
|
"output_tps": 289.81345646999426,
|
||||||
|
"total_tps": 1449.0672823499715,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 4658.318318077363,
|
||||||
|
"ttft_p95_ms": 10095.44171601301,
|
||||||
|
"ttft_p99_ms": 12803.387446557112,
|
||||||
|
"tpot_p50_ms": 44.42285834210774,
|
||||||
|
"tpot_p95_ms": 49.33753012539997,
|
||||||
|
"tpot_p99_ms": 79.11847314414067,
|
||||||
|
"e2e_p50_ms": 49335.88792406954,
|
||||||
|
"e2e_p95_ms": 58133.938353240934,
|
||||||
|
"e2e_p99_ms": 87081.42475777536,
|
||||||
|
"itl_p50_ms": 38.47858391236514,
|
||||||
|
"itl_p95_ms": 57.68822985701263,
|
||||||
|
"itl_p99_ms": 62.88705667946487,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 64.04329815506935,
|
||||||
|
"request_tps": 1.2491549046442667,
|
||||||
|
"input_tps": 5116.538489422916,
|
||||||
|
"output_tps": 159.89182779446614,
|
||||||
|
"total_tps": 5276.430317217383,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 4576.742604491301,
|
||||||
|
"ttft_p95_ms": 7486.383566202128,
|
||||||
|
"ttft_p99_ms": 8069.572048275731,
|
||||||
|
"tpot_p50_ms": 63.25949257419364,
|
||||||
|
"tpot_p95_ms": 88.51012927140879,
|
||||||
|
"tpot_p99_ms": 93.78976958789158,
|
||||||
|
"e2e_p50_ms": 12597.990370937623,
|
||||||
|
"e2e_p95_ms": 16119.662079052066,
|
||||||
|
"e2e_p99_ms": 16428.539164408576,
|
||||||
|
"itl_p50_ms": 38.49607554730028,
|
||||||
|
"itl_p95_ms": 59.04390099458394,
|
||||||
|
"itl_p99_ms": 859.1562826023437,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 534.5376915351953,
|
||||||
|
"request_tps": 0.14966203743320614,
|
||||||
|
"input_tps": 613.0157053264123,
|
||||||
|
"output_tps": 306.50785266320617,
|
||||||
|
"total_tps": 919.5235579896186,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 5104.571636882611,
|
||||||
|
"ttft_p95_ms": 13598.845852899827,
|
||||||
|
"ttft_p99_ms": 15530.32725170254,
|
||||||
|
"tpot_p50_ms": 42.516595378821116,
|
||||||
|
"tpot_p95_ms": 48.41014748285291,
|
||||||
|
"tpot_p99_ms": 80.66643662295293,
|
||||||
|
"e2e_p50_ms": 90892.7560900338,
|
||||||
|
"e2e_p95_ms": 106289.35107835554,
|
||||||
|
"e2e_p99_ms": 171094.83309885245,
|
||||||
|
"itl_p50_ms": 38.54017308913171,
|
||||||
|
"itl_p95_ms": 58.33887221524492,
|
||||||
|
"itl_p99_ms": 64.16293417569251,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 95.2746979680378,
|
||||||
|
"request_tps": 0.839677287949398,
|
||||||
|
"input_tps": 3439.318171440734,
|
||||||
|
"output_tps": 214.9573857150459,
|
||||||
|
"total_tps": 3654.27555715578,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 4683.939669514075,
|
||||||
|
"ttft_p95_ms": 8062.7326244954,
|
||||||
|
"ttft_p99_ms": 8673.460278843995,
|
||||||
|
"tpot_p50_ms": 51.88298721149491,
|
||||||
|
"tpot_p95_ms": 64.92892745028561,
|
||||||
|
"tpot_p99_ms": 75.71912046924533,
|
||||||
|
"e2e_p50_ms": 17954.677841393277,
|
||||||
|
"e2e_p95_ms": 22523.24340294581,
|
||||||
|
"e2e_p99_ms": 24458.481649178953,
|
||||||
|
"itl_p50_ms": 38.46407646778971,
|
||||||
|
"itl_p95_ms": 59.771430026739836,
|
||||||
|
"itl_p99_ms": 67.52078937599435,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 1029.577203406021,
|
||||||
|
"request_tps": 0.07770179811222125,
|
||||||
|
"input_tps": 318.26656506765823,
|
||||||
|
"output_tps": 318.26656506765823,
|
||||||
|
"total_tps": 636.5331301353165,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 4096.0,
|
||||||
|
"ttft_p50_ms": 5483.361787861213,
|
||||||
|
"ttft_p95_ms": 22374.68960867263,
|
||||||
|
"ttft_p99_ms": 150920.9240530175,
|
||||||
|
"tpot_p50_ms": 41.06715219183478,
|
||||||
|
"tpot_p95_ms": 46.9445546033481,
|
||||||
|
"tpot_p99_ms": 79.17070015191274,
|
||||||
|
"e2e_p50_ms": 173329.3942649616,
|
||||||
|
"e2e_p95_ms": 302973.24356382014,
|
||||||
|
"e2e_p99_ms": 330154.8701896191,
|
||||||
|
"itl_p50_ms": 38.36022887844592,
|
||||||
|
"itl_p95_ms": 57.20986550441011,
|
||||||
|
"itl_p99_ms": 62.143948464654386,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 160.08099704002962,
|
||||||
|
"request_tps": 0.4997470123202401,
|
||||||
|
"input_tps": 2046.9637624637035,
|
||||||
|
"output_tps": 255.87047030796293,
|
||||||
|
"total_tps": 2302.8342327716664,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 5474.024421186186,
|
||||||
|
"ttft_p95_ms": 8615.011906716973,
|
||||||
|
"ttft_p99_ms": 8951.299454767715,
|
||||||
|
"tpot_p50_ms": 46.494569923772445,
|
||||||
|
"tpot_p95_ms": 53.43938896643479,
|
||||||
|
"tpot_p99_ms": 76.29711307971617,
|
||||||
|
"e2e_p50_ms": 28335.586922592483,
|
||||||
|
"e2e_p95_ms": 32704.220330703534,
|
||||||
|
"e2e_p99_ms": 44918.695579937164,
|
||||||
|
"itl_p50_ms": 38.48353505600244,
|
||||||
|
"itl_p95_ms": 59.288766619283706,
|
||||||
|
"itl_p99_ms": 65.03905458841473,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 1253.2045440499205,
|
||||||
|
"request_tps": 0.0638363468915201,
|
||||||
|
"input_tps": 4183.578829882661,
|
||||||
|
"output_tps": 65.36841921691658,
|
||||||
|
"total_tps": 4248.947249099578,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 202414.35351443943,
|
||||||
|
"ttft_p95_ms": 223751.87583953375,
|
||||||
|
"ttft_p99_ms": 226827.1145781991,
|
||||||
|
"tpot_p50_ms": 53.94717318803356,
|
||||||
|
"tpot_p95_ms": 71.33589557350184,
|
||||||
|
"tpot_p99_ms": 86.74920972760991,
|
||||||
|
"e2e_p50_ms": 247910.9200029634,
|
||||||
|
"e2e_p95_ms": 278792.46321712853,
|
||||||
|
"e2e_p99_ms": 292009.2727964232,
|
||||||
|
"itl_p50_ms": 36.60361119545996,
|
||||||
|
"itl_p95_ms": 54.5408145524561,
|
||||||
|
"itl_p99_ms": 62.74137625936419,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 765.7493072859943,
|
||||||
|
"request_tps": 0.1044728336530135,
|
||||||
|
"input_tps": 6846.731626283893,
|
||||||
|
"output_tps": 13.372522707585729,
|
||||||
|
"total_tps": 6860.104148991479,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 128567.64044612646,
|
||||||
|
"ttft_p95_ms": 151961.6587853641,
|
||||||
|
"ttft_p99_ms": 156960.82069837945,
|
||||||
|
"tpot_p50_ms": 161.96727417372753,
|
||||||
|
"tpot_p95_ms": 305.1854889790388,
|
||||||
|
"tpot_p99_ms": 437.706828691014,
|
||||||
|
"e2e_p50_ms": 146586.26923209522,
|
||||||
|
"e2e_p95_ms": 179067.64789508888,
|
||||||
|
"e2e_p99_ms": 186050.76994073341,
|
||||||
|
"itl_p50_ms": 36.512586986646056,
|
||||||
|
"itl_p95_ms": 53.31773438956598,
|
||||||
|
"itl_p99_ms": 61.62503688596189,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 832.5039105331525,
|
||||||
|
"request_tps": 0.09609564470245716,
|
||||||
|
"input_tps": 6297.724171220232,
|
||||||
|
"output_tps": 24.600485043829032,
|
||||||
|
"total_tps": 6322.324656264061,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 136335.5990794953,
|
||||||
|
"ttft_p95_ms": 162983.08086185716,
|
||||||
|
"ttft_p99_ms": 166663.45600951457,
|
||||||
|
"tpot_p50_ms": 100.12674616266263,
|
||||||
|
"tpot_p95_ms": 175.98028106304506,
|
||||||
|
"tpot_p99_ms": 236.80305566614055,
|
||||||
|
"e2e_p50_ms": 156982.1368450066,
|
||||||
|
"e2e_p95_ms": 189405.96394817112,
|
||||||
|
"e2e_p99_ms": 206104.3898640247,
|
||||||
|
"itl_p50_ms": 36.563907051458955,
|
||||||
|
"itl_p95_ms": 53.42142162844535,
|
||||||
|
"itl_p99_ms": 62.24962119013072,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 80,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 973.7963511580601,
|
||||||
|
"request_tps": 0.08215270051573127,
|
||||||
|
"input_tps": 5383.959380998965,
|
||||||
|
"output_tps": 42.06218266405441,
|
||||||
|
"total_tps": 5426.021563663019,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 153779.00782134384,
|
||||||
|
"ttft_p95_ms": 181627.71489518927,
|
||||||
|
"ttft_p99_ms": 183322.4648270011,
|
||||||
|
"tpot_p50_ms": 69.34998192962937,
|
||||||
|
"tpot_p95_ms": 106.79924657778838,
|
||||||
|
"tpot_p99_ms": 136.58104994009412,
|
||||||
|
"e2e_p50_ms": 189005.7152614463,
|
||||||
|
"e2e_p95_ms": 234631.01633958286,
|
||||||
|
"e2e_p99_ms": 236243.1676932983,
|
||||||
|
"itl_p50_ms": 36.63114202208817,
|
||||||
|
"itl_p95_ms": 53.81391057744622,
|
||||||
|
"itl_p99_ms": 62.73157489486038,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 226.54693996487185,
|
||||||
|
"request_tps": 0.022070481290876385,
|
||||||
|
"input_tps": 2892.8221237577495,
|
||||||
|
"output_tps": 2.8250216052321773,
|
||||||
|
"total_tps": 2895.647145362982,
|
||||||
|
"mean_input_tokens": 131072.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 40983.879873063415,
|
||||||
|
"ttft_p95_ms": 41427.96418489888,
|
||||||
|
"ttft_p99_ms": 41483.68903929368,
|
||||||
|
"tpot_p50_ms": 33.28039894415814,
|
||||||
|
"tpot_p95_ms": 33.693099089744166,
|
||||||
|
"tpot_p99_ms": 33.698522558318466,
|
||||||
|
"e2e_p50_ms": 45259.45955491625,
|
||||||
|
"e2e_p95_ms": 45639.66128444299,
|
||||||
|
"e2e_p99_ms": 45701.79939834401,
|
||||||
|
"itl_p50_ms": 30.720365000888705,
|
||||||
|
"itl_p95_ms": 55.317965033464134,
|
||||||
|
"itl_p99_ms": 62.53650498110801,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 181.57242772518657,
|
||||||
|
"request_tps": 0.02753722061571814,
|
||||||
|
"input_tps": 451.169822567926,
|
||||||
|
"output_tps": 28.198113910495376,
|
||||||
|
"total_tps": 479.3679364784214,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 3940.7668448984623,
|
||||||
|
"ttft_p95_ms": 4193.481011828408,
|
||||||
|
"ttft_p99_ms": 4228.460800005123,
|
||||||
|
"tpot_p50_ms": 31.63807294540879,
|
||||||
|
"tpot_p95_ms": 31.730153837458815,
|
||||||
|
"tpot_p99_ms": 31.734481836830287,
|
||||||
|
"e2e_p50_ms": 36320.72346494533,
|
||||||
|
"e2e_p95_ms": 36463.93566392362,
|
||||||
|
"e2e_p99_ms": 36480.87247610092,
|
||||||
|
"itl_p50_ms": 29.840212780982256,
|
||||||
|
"itl_p95_ms": 46.52133162599058,
|
||||||
|
"itl_p99_ms": 54.09107577055689,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 39.83333700802177,
|
||||||
|
"request_tps": 0.12552300097260452,
|
||||||
|
"input_tps": 2056.5688479351525,
|
||||||
|
"output_tps": 16.06694412449338,
|
||||||
|
"total_tps": 2072.6357920596456,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 3952.2010260261595,
|
||||||
|
"ttft_p95_ms": 4079.302547173574,
|
||||||
|
"ttft_p99_ms": 4102.609633402899,
|
||||||
|
"tpot_p50_ms": 31.602383598567936,
|
||||||
|
"tpot_p95_ms": 31.640485883140425,
|
||||||
|
"tpot_p99_ms": 31.642765031291507,
|
||||||
|
"e2e_p50_ms": 7969.095474109054,
|
||||||
|
"e2e_p95_ms": 8074.662221223116,
|
||||||
|
"e2e_p99_ms": 8093.300537876784,
|
||||||
|
"itl_p50_ms": 29.8505169339478,
|
||||||
|
"itl_p95_ms": 49.705708399415016,
|
||||||
|
"itl_p99_ms": 51.2071222346276,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 342.39226463483647,
|
||||||
|
"request_tps": 0.014603133646528293,
|
||||||
|
"input_tps": 239.25774166471956,
|
||||||
|
"output_tps": 29.907217708089945,
|
||||||
|
"total_tps": 269.1649593728095,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 3971.6174241621047,
|
||||||
|
"ttft_p95_ms": 4094.661511434242,
|
||||||
|
"ttft_p99_ms": 4115.855502327904,
|
||||||
|
"tpot_p50_ms": 31.558119324293,
|
||||||
|
"tpot_p95_ms": 31.620720006947725,
|
||||||
|
"tpot_p99_ms": 31.632591595734013,
|
||||||
|
"e2e_p50_ms": 68487.32287203893,
|
||||||
|
"e2e_p95_ms": 68711.56300739385,
|
||||||
|
"e2e_p99_ms": 68739.65807267465,
|
||||||
|
"itl_p50_ms": 29.78591388091445,
|
||||||
|
"itl_p95_ms": 45.330130099318914,
|
||||||
|
"itl_p99_ms": 55.03684663679451,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 60.537773119984195,
|
||||||
|
"request_tps": 0.08259306119652168,
|
||||||
|
"input_tps": 1353.2047146438113,
|
||||||
|
"output_tps": 21.14382366630955,
|
||||||
|
"total_tps": 1374.3485383101208,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 3945.7851310726255,
|
||||||
|
"ttft_p95_ms": 4093.810023879632,
|
||||||
|
"ttft_p99_ms": 4111.218916652724,
|
||||||
|
"tpot_p50_ms": 31.821994341033346,
|
||||||
|
"tpot_p95_ms": 32.01389185737307,
|
||||||
|
"tpot_p99_ms": 32.03676525156433,
|
||||||
|
"e2e_p50_ms": 12080.163977108896,
|
||||||
|
"e2e_p95_ms": 12253.15336487256,
|
||||||
|
"e2e_p99_ms": 12279.754239274189,
|
||||||
|
"itl_p50_ms": 29.860365903005004,
|
||||||
|
"itl_p95_ms": 48.98134677205235,
|
||||||
|
"itl_p99_ms": 55.7887158729136,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 100.54988616588525,
|
||||||
|
"request_tps": 0.04972656052291394,
|
||||||
|
"input_tps": 814.719967607422,
|
||||||
|
"output_tps": 25.459998987731936,
|
||||||
|
"total_tps": 840.1799665951539,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 3932.124942075461,
|
||||||
|
"ttft_p95_ms": 4127.591159660369,
|
||||||
|
"ttft_p99_ms": 4166.129624564201,
|
||||||
|
"tpot_p50_ms": 31.635086914185443,
|
||||||
|
"tpot_p95_ms": 31.776042928674975,
|
||||||
|
"tpot_p99_ms": 31.788027078066378,
|
||||||
|
"e2e_p50_ms": 20141.837269999087,
|
||||||
|
"e2e_p95_ms": 20233.498825039715,
|
||||||
|
"e2e_p99_ms": 20244.73103923723,
|
||||||
|
"itl_p50_ms": 29.790908098220825,
|
||||||
|
"itl_p95_ms": 50.02487420570105,
|
||||||
|
"itl_p99_ms": 54.69925904646516,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 256.18154512299225,
|
||||||
|
"request_tps": 0.019517409021791596,
|
||||||
|
"input_tps": 1279.092917652134,
|
||||||
|
"output_tps": 19.985826838314594,
|
||||||
|
"total_tps": 1299.0787444904488,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 17658.26961491257,
|
||||||
|
"ttft_p95_ms": 18013.91982375644,
|
||||||
|
"ttft_p99_ms": 18047.10275827907,
|
||||||
|
"tpot_p50_ms": 32.7376385434701,
|
||||||
|
"tpot_p95_ms": 32.79802809666955,
|
||||||
|
"tpot_p99_ms": 32.798152351014515,
|
||||||
|
"e2e_p50_ms": 51210.01679683104,
|
||||||
|
"e2e_p95_ms": 51457.70765142515,
|
||||||
|
"e2e_p99_ms": 51481.52730548754,
|
||||||
|
"itl_p50_ms": 30.361955054104328,
|
||||||
|
"itl_p95_ms": 53.69571931660175,
|
||||||
|
"itl_p99_ms": 60.88819689117371,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 109.37776633584872,
|
||||||
|
"request_tps": 0.045713129528055126,
|
||||||
|
"input_tps": 2995.8556567506207,
|
||||||
|
"output_tps": 5.851280579591056,
|
||||||
|
"total_tps": 3001.706937330212,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 17729.769844096154,
|
||||||
|
"ttft_p95_ms": 17954.093989543617,
|
||||||
|
"ttft_p99_ms": 17964.415912255645,
|
||||||
|
"tpot_p50_ms": 32.0961656301891,
|
||||||
|
"tpot_p95_ms": 32.58936951105578,
|
||||||
|
"tpot_p99_ms": 32.640932479609305,
|
||||||
|
"e2e_p50_ms": 21835.877286968753,
|
||||||
|
"e2e_p95_ms": 22044.52510732226,
|
||||||
|
"e2e_p99_ms": 22048.520961580798,
|
||||||
|
"itl_p50_ms": 30.422291019931436,
|
||||||
|
"itl_p95_ms": 47.328294953331344,
|
||||||
|
"itl_p99_ms": 56.53701915405691,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 129.80575013905764,
|
||||||
|
"request_tps": 0.038519094837044014,
|
||||||
|
"input_tps": 2524.3873992405165,
|
||||||
|
"output_tps": 9.860888278283268,
|
||||||
|
"total_tps": 2534.2482875187998,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 17821.7026530765,
|
||||||
|
"ttft_p95_ms": 17941.041450900957,
|
||||||
|
"ttft_p99_ms": 17956.14996048622,
|
||||||
|
"tpot_p50_ms": 32.05825664723913,
|
||||||
|
"tpot_p95_ms": 32.285422013122954,
|
||||||
|
"tpot_p99_ms": 32.28693708328201,
|
||||||
|
"e2e_p50_ms": 26052.55355197005,
|
||||||
|
"e2e_p95_ms": 26127.57891532965,
|
||||||
|
"e2e_p99_ms": 26133.34180940874,
|
||||||
|
"itl_p50_ms": 30.41153005324304,
|
||||||
|
"itl_p95_ms": 46.34950009640306,
|
||||||
|
"itl_p99_ms": 56.67572288773954,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 5,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 170.8164885980077,
|
||||||
|
"request_tps": 0.029271178918604212,
|
||||||
|
"input_tps": 1918.3159816096456,
|
||||||
|
"output_tps": 14.986843606325357,
|
||||||
|
"total_tps": 1933.302825215971,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 17657.30693913065,
|
||||||
|
"ttft_p95_ms": 17947.263365006074,
|
||||||
|
"ttft_p99_ms": 17974.353000214323,
|
||||||
|
"tpot_p50_ms": 32.16144313886325,
|
||||||
|
"tpot_p95_ms": 32.28488007000849,
|
||||||
|
"tpot_p99_ms": 32.28917933327971,
|
||||||
|
"e2e_p50_ms": 34091.804383089766,
|
||||||
|
"e2e_p95_ms": 34388.406378310174,
|
||||||
|
"e2e_p99_ms": 34406.406796555966,
|
||||||
|
"itl_p50_ms": 30.36797302775085,
|
||||||
|
"itl_p95_ms": 46.908468334004276,
|
||||||
|
"itl_p99_ms": 55.469688819721355,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 160,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 283.1469709521625,
|
||||||
|
"request_tps": 0.5650775618822773,
|
||||||
|
"input_tps": 578.639423367452,
|
||||||
|
"output_tps": 578.639423367452,
|
||||||
|
"total_tps": 1157.278846734904,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 1039.5908689824864,
|
||||||
|
"ttft_p95_ms": 3708.7073334376323,
|
||||||
|
"ttft_p99_ms": 3736.5557909896597,
|
||||||
|
"tpot_p50_ms": 53.18489079777869,
|
||||||
|
"tpot_p95_ms": 56.531733996098275,
|
||||||
|
"tpot_p99_ms": 56.90662554211692,
|
||||||
|
"e2e_p50_ms": 55562.747771968134,
|
||||||
|
"e2e_p95_ms": 58622.54847800359,
|
||||||
|
"e2e_p99_ms": 58864.768436276354,
|
||||||
|
"itl_p50_ms": 44.60177046712488,
|
||||||
|
"itl_p95_ms": 68.54137406917289,
|
||||||
|
"itl_p99_ms": 104.98853123513977,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 160,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 562.1656430249568,
|
||||||
|
"request_tps": 0.28461362231077675,
|
||||||
|
"input_tps": 291.4443492462354,
|
||||||
|
"output_tps": 582.8886984924708,
|
||||||
|
"total_tps": 874.3330477387061,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 1720.5767229897901,
|
||||||
|
"ttft_p95_ms": 3765.552905423101,
|
||||||
|
"ttft_p99_ms": 3778.2479317323305,
|
||||||
|
"tpot_p50_ms": 53.25908665974094,
|
||||||
|
"tpot_p95_ms": 55.20441076613287,
|
||||||
|
"tpot_p99_ms": 55.39557766710268,
|
||||||
|
"e2e_p50_ms": 109979.57844298799,
|
||||||
|
"e2e_p95_ms": 114323.36549273459,
|
||||||
|
"e2e_p99_ms": 114707.30229713488,
|
||||||
|
"itl_p50_ms": 46.27890046685934,
|
||||||
|
"itl_p95_ms": 69.01592136127874,
|
||||||
|
"itl_p99_ms": 77.31038637226447,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 160,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 85.51100934692658,
|
||||||
|
"request_tps": 1.8711040978462112,
|
||||||
|
"input_tps": 1916.0105961945203,
|
||||||
|
"output_tps": 479.0026490486301,
|
||||||
|
"total_tps": 2395.0132452431503,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 1161.2384765176103,
|
||||||
|
"ttft_p95_ms": 3756.2352412380274,
|
||||||
|
"ttft_p99_ms": 3780.965418519918,
|
||||||
|
"tpot_p50_ms": 61.98459189412568,
|
||||||
|
"tpot_p95_ms": 66.44078092480663,
|
||||||
|
"tpot_p99_ms": 66.8051291848259,
|
||||||
|
"e2e_p50_ms": 16901.625977130607,
|
||||||
|
"e2e_p95_ms": 18837.460162944622,
|
||||||
|
"e2e_p99_ms": 18985.462763826363,
|
||||||
|
"itl_p50_ms": 44.66060339473188,
|
||||||
|
"itl_p95_ms": 68.57059714384377,
|
||||||
|
"itl_p99_ms": 449.1709415311925,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 160,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 151.8317316111643,
|
||||||
|
"request_tps": 1.0537981639421354,
|
||||||
|
"input_tps": 1079.0893198767467,
|
||||||
|
"output_tps": 539.5446599383733,
|
||||||
|
"total_tps": 1618.63397981512,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 1056.6700529307127,
|
||||||
|
"ttft_p95_ms": 3730.0430950359428,
|
||||||
|
"ttft_p99_ms": 3803.3011273038574,
|
||||||
|
"tpot_p50_ms": 55.7243038511538,
|
||||||
|
"tpot_p95_ms": 60.350505806735306,
|
||||||
|
"tpot_p99_ms": 61.641385694806246,
|
||||||
|
"e2e_p50_ms": 29571.674760896713,
|
||||||
|
"e2e_p95_ms": 32270.35216266522,
|
||||||
|
"e2e_p99_ms": 32279.17558920337,
|
||||||
|
"itl_p50_ms": 45.009728986769915,
|
||||||
|
"itl_p95_ms": 67.9298322298564,
|
||||||
|
"itl_p99_ms": 236.92634512903186,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 240,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 386.91459054895677,
|
||||||
|
"request_tps": 0.6202919348672961,
|
||||||
|
"input_tps": 635.1789413041112,
|
||||||
|
"output_tps": 635.1789413041112,
|
||||||
|
"total_tps": 1270.3578826082223,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 5370.0150235090405,
|
||||||
|
"ttft_p95_ms": 15133.30938139696,
|
||||||
|
"ttft_p99_ms": 32185.267814283718,
|
||||||
|
"tpot_p50_ms": 60.0038268968676,
|
||||||
|
"tpot_p95_ms": 91.36051824342555,
|
||||||
|
"tpot_p99_ms": 103.14346713503731,
|
||||||
|
"e2e_p50_ms": 65575.55996044539,
|
||||||
|
"e2e_p95_ms": 99691.6358396411,
|
||||||
|
"e2e_p99_ms": 112446.33609429697,
|
||||||
|
"itl_p50_ms": 51.48644756991416,
|
||||||
|
"itl_p95_ms": 73.74423729488626,
|
||||||
|
"itl_p99_ms": 83.97575234295799,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 240,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 785.6881854759995,
|
||||||
|
"request_tps": 0.3054646925288802,
|
||||||
|
"input_tps": 312.7958451495733,
|
||||||
|
"output_tps": 625.5916902991466,
|
||||||
|
"total_tps": 938.3875354487201,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 5525.519606540911,
|
||||||
|
"ttft_p95_ms": 38321.421813906585,
|
||||||
|
"ttft_p99_ms": 61896.90983353647,
|
||||||
|
"tpot_p50_ms": 60.786382354451064,
|
||||||
|
"tpot_p95_ms": 94.29399440951978,
|
||||||
|
"tpot_p99_ms": 106.96621055514711,
|
||||||
|
"e2e_p50_ms": 128179.90333400667,
|
||||||
|
"e2e_p95_ms": 211186.33220964108,
|
||||||
|
"e2e_p99_ms": 235519.09909247872,
|
||||||
|
"itl_p50_ms": 52.96039255335927,
|
||||||
|
"itl_p95_ms": 74.77014367468655,
|
||||||
|
"itl_p99_ms": 93.00403241533787,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 240,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 102.57387904892676,
|
||||||
|
"request_tps": 2.3397769707580456,
|
||||||
|
"input_tps": 2395.9316180562387,
|
||||||
|
"output_tps": 598.9829045140597,
|
||||||
|
"total_tps": 2994.9145225702982,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 4857.287564082071,
|
||||||
|
"ttft_p95_ms": 5687.151023850311,
|
||||||
|
"ttft_p99_ms": 5947.1146802348085,
|
||||||
|
"tpot_p50_ms": 60.30845692353871,
|
||||||
|
"tpot_p95_ms": 72.36586667291418,
|
||||||
|
"tpot_p99_ms": 78.81875480012964,
|
||||||
|
"e2e_p50_ms": 20432.19355738256,
|
||||||
|
"e2e_p95_ms": 21489.17214354733,
|
||||||
|
"e2e_p99_ms": 22516.328958119266,
|
||||||
|
"itl_p50_ms": 51.60140746738762,
|
||||||
|
"itl_p95_ms": 73.69143945397809,
|
||||||
|
"itl_p99_ms": 89.00051706237718,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 240,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 194.6086190820206,
|
||||||
|
"request_tps": 1.2332444530570794,
|
||||||
|
"input_tps": 1262.8423199304493,
|
||||||
|
"output_tps": 631.4211599652247,
|
||||||
|
"total_tps": 1894.263479895674,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 5219.216491095722,
|
||||||
|
"ttft_p95_ms": 5779.679291823413,
|
||||||
|
"ttft_p99_ms": 6389.283453426324,
|
||||||
|
"tpot_p50_ms": 58.08764942142558,
|
||||||
|
"tpot_p95_ms": 62.69429164324429,
|
||||||
|
"tpot_p99_ms": 91.06430042089896,
|
||||||
|
"e2e_p50_ms": 34896.657894947566,
|
||||||
|
"e2e_p95_ms": 35910.429730673786,
|
||||||
|
"e2e_p99_ms": 49593.8836463075,
|
||||||
|
"itl_p50_ms": 51.571647985838354,
|
||||||
|
"itl_p95_ms": 73.75171015737578,
|
||||||
|
"itl_p99_ms": 77.6099856174551,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 750.264118807856,
|
||||||
|
"request_tps": 0.0533145581632754,
|
||||||
|
"input_tps": 54.59410755919401,
|
||||||
|
"output_tps": 218.37643023677603,
|
||||||
|
"total_tps": 272.97053779597,
|
||||||
|
"mean_input_tokens": 1024.0,
|
||||||
|
"mean_output_tokens": 4096.0,
|
||||||
|
"ttft_p50_ms": 457.8002068446949,
|
||||||
|
"ttft_p95_ms": 1004.7794182202779,
|
||||||
|
"ttft_p99_ms": 1006.1946984101087,
|
||||||
|
"tpot_p50_ms": 36.49401828900812,
|
||||||
|
"tpot_p95_ms": 36.67125693285014,
|
||||||
|
"tpot_p99_ms": 36.679555167656694,
|
||||||
|
"e2e_p50_ms": 150253.00148909446,
|
||||||
|
"e2e_p95_ms": 150511.7817995837,
|
||||||
|
"e2e_p99_ms": 150794.05072218738,
|
||||||
|
"itl_p50_ms": 34.35514506418258,
|
||||||
|
"itl_p95_ms": 53.22418445721268,
|
||||||
|
"itl_p99_ms": 59.324696937110275,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 818.1570529809687,
|
||||||
|
"request_tps": 0.04889036873086816,
|
||||||
|
"input_tps": 6408.158410292352,
|
||||||
|
"output_tps": 6.257967197551125,
|
||||||
|
"total_tps": 6414.4163774899025,
|
||||||
|
"mean_input_tokens": 131072.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 129584.32166394778,
|
||||||
|
"ttft_p95_ms": 167181.3310468453,
|
||||||
|
"ttft_p99_ms": 168040.67650724668,
|
||||||
|
"tpot_p50_ms": 36.84465442857463,
|
||||||
|
"tpot_p95_ms": 343.3015784548456,
|
||||||
|
"tpot_p99_ms": 345.92269529082523,
|
||||||
|
"e2e_p50_ms": 171526.36588714086,
|
||||||
|
"e2e_p95_ms": 173146.40125581063,
|
||||||
|
"e2e_p99_ms": 173291.67424835498,
|
||||||
|
"itl_p50_ms": 33.540375996381044,
|
||||||
|
"itl_p95_ms": 58.00506006926298,
|
||||||
|
"itl_p99_ms": 276.8040000228211,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 335.39274771884084,
|
||||||
|
"request_tps": 0.11926316317826863,
|
||||||
|
"input_tps": 1954.0076655127532,
|
||||||
|
"output_tps": 122.12547909454707,
|
||||||
|
"total_tps": 2076.1331446073004,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 11495.135091943666,
|
||||||
|
"ttft_p95_ms": 54293.259906279854,
|
||||||
|
"ttft_p99_ms": 54475.75780400075,
|
||||||
|
"tpot_p50_ms": 41.142632146611746,
|
||||||
|
"tpot_p95_ms": 44.4825864581911,
|
||||||
|
"tpot_p99_ms": 48.84856583131568,
|
||||||
|
"e2e_p50_ms": 49175.88768794667,
|
||||||
|
"e2e_p95_ms": 97491.41289745457,
|
||||||
|
"e2e_p99_ms": 102364.25640797475,
|
||||||
|
"itl_p50_ms": 34.5357779879123,
|
||||||
|
"itl_p95_ms": 52.090133994352065,
|
||||||
|
"itl_p99_ms": 60.2705589751713,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 109.44023031182587,
|
||||||
|
"request_tps": 0.36549630685195744,
|
||||||
|
"input_tps": 5988.291491462471,
|
||||||
|
"output_tps": 46.78352727705055,
|
||||||
|
"total_tps": 6035.075018739521,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 11529.161532991566,
|
||||||
|
"ttft_p95_ms": 21725.308789138217,
|
||||||
|
"ttft_p99_ms": 21941.029288116843,
|
||||||
|
"tpot_p50_ms": 76.67213628768891,
|
||||||
|
"tpot_p95_ms": 101.16250104450535,
|
||||||
|
"tpot_p99_ms": 138.8094991766004,
|
||||||
|
"e2e_p50_ms": 16583.38130300399,
|
||||||
|
"e2e_p95_ms": 32255.35403857939,
|
||||||
|
"e2e_p99_ms": 37416.11094208899,
|
||||||
|
"itl_p50_ms": 34.63044541422278,
|
||||||
|
"itl_p95_ms": 54.926601552870125,
|
||||||
|
"itl_p99_ms": 1767.2955490718596,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 584.8935852469876,
|
||||||
|
"request_tps": 0.06838850862607578,
|
||||||
|
"input_tps": 1120.4773253296255,
|
||||||
|
"output_tps": 140.0596656662032,
|
||||||
|
"total_tps": 1260.5369909958285,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 11495.196483563632,
|
||||||
|
"ttft_p95_ms": 91582.56554905092,
|
||||||
|
"ttft_p99_ms": 91693.05469256826,
|
||||||
|
"tpot_p50_ms": 38.41638142817654,
|
||||||
|
"tpot_p95_ms": 41.38107354754197,
|
||||||
|
"tpot_p99_ms": 42.587399352946996,
|
||||||
|
"e2e_p50_ms": 89565.8936999971,
|
||||||
|
"e2e_p95_ms": 172048.6941325711,
|
||||||
|
"e2e_p99_ms": 176627.42560487473,
|
||||||
|
"itl_p50_ms": 34.59935949649662,
|
||||||
|
"itl_p95_ms": 51.98211170500145,
|
||||||
|
"itl_p99_ms": 60.23820979986343,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 141.670446844073,
|
||||||
|
"request_tps": 0.28234540718308926,
|
||||||
|
"input_tps": 4625.947151287734,
|
||||||
|
"output_tps": 72.28042423887085,
|
||||||
|
"total_tps": 4698.2275755266055,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 11554.360230453312,
|
||||||
|
"ttft_p95_ms": 26391.11408374738,
|
||||||
|
"ttft_p99_ms": 26587.14834881015,
|
||||||
|
"tpot_p50_ms": 56.31336195065695,
|
||||||
|
"tpot_p95_ms": 68.25714148759073,
|
||||||
|
"tpot_p99_ms": 87.6372731319245,
|
||||||
|
"e2e_p50_ms": 21265.938312513754,
|
||||||
|
"e2e_p95_ms": 41577.943935687654,
|
||||||
|
"e2e_p99_ms": 46795.674697651986,
|
||||||
|
"itl_p50_ms": 34.58584542386234,
|
||||||
|
"itl_p95_ms": 52.59093521162865,
|
||||||
|
"itl_p99_ms": 61.120713129639775,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 206.38675206596963,
|
||||||
|
"request_tps": 0.19381088950522546,
|
||||||
|
"input_tps": 3175.397613653614,
|
||||||
|
"output_tps": 99.23117542667543,
|
||||||
|
"total_tps": 3274.628789080289,
|
||||||
|
"mean_input_tokens": 16384.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 11589.022769010626,
|
||||||
|
"ttft_p95_ms": 35687.3680736986,
|
||||||
|
"ttft_p99_ms": 35902.96996307094,
|
||||||
|
"tpot_p50_ms": 46.31837186974942,
|
||||||
|
"tpot_p95_ms": 52.744000344951814,
|
||||||
|
"tpot_p99_ms": 61.98560190418053,
|
||||||
|
"e2e_p50_ms": 30626.526765991002,
|
||||||
|
"e2e_p95_ms": 60265.1604500832,
|
||||||
|
"e2e_p99_ms": 65432.796396943726,
|
||||||
|
"itl_p50_ms": 34.61834159679711,
|
||||||
|
"itl_p95_ms": 51.886969921179116,
|
||||||
|
"itl_p99_ms": 60.28145674150437,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 207.04861099808477,
|
||||||
|
"request_tps": 0.1931913467430603,
|
||||||
|
"input_tps": 791.311756259575,
|
||||||
|
"output_tps": 197.82793906489374,
|
||||||
|
"total_tps": 989.1396953244687,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 1074.1113065741956,
|
||||||
|
"ttft_p95_ms": 3878.6458508693613,
|
||||||
|
"ttft_p99_ms": 3916.904898113571,
|
||||||
|
"tpot_p50_ms": 39.24930339234399,
|
||||||
|
"tpot_p95_ms": 40.05846651063872,
|
||||||
|
"tpot_p99_ms": 40.82051611248493,
|
||||||
|
"e2e_p50_ms": 41186.332597979344,
|
||||||
|
"e2e_p95_ms": 43346.47120544687,
|
||||||
|
"e2e_p99_ms": 43701.94057547487,
|
||||||
|
"itl_p50_ms": 34.467250341549516,
|
||||||
|
"itl_p95_ms": 54.143763531465076,
|
||||||
|
"itl_p99_ms": 59.957590696867555,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 42.330964882159606,
|
||||||
|
"request_tps": 0.9449347566574843,
|
||||||
|
"input_tps": 3870.4527632690556,
|
||||||
|
"output_tps": 120.95164885215799,
|
||||||
|
"total_tps": 3991.4044121212137,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 128.0,
|
||||||
|
"ttft_p50_ms": 2789.9403674528003,
|
||||||
|
"ttft_p95_ms": 3850.861389178317,
|
||||||
|
"ttft_p99_ms": 3857.0726825506426,
|
||||||
|
"tpot_p50_ms": 44.318099374200884,
|
||||||
|
"tpot_p95_ms": 52.61084187132357,
|
||||||
|
"tpot_p99_ms": 56.91049005684083,
|
||||||
|
"e2e_p50_ms": 8397.533455048688,
|
||||||
|
"e2e_p95_ms": 10301.826596679166,
|
||||||
|
"e2e_p99_ms": 10350.095471963286,
|
||||||
|
"itl_p50_ms": 34.42738042213023,
|
||||||
|
"itl_p95_ms": 54.41078649600968,
|
||||||
|
"itl_p99_ms": 108.43641549814492,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 394.49152314290404,
|
||||||
|
"request_tps": 0.10139634859913087,
|
||||||
|
"input_tps": 415.31944386204003,
|
||||||
|
"output_tps": 207.65972193102002,
|
||||||
|
"total_tps": 622.97916579306,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 2048.0,
|
||||||
|
"ttft_p50_ms": 1034.7788120852783,
|
||||||
|
"ttft_p95_ms": 3756.3848297810177,
|
||||||
|
"ttft_p99_ms": 3805.5001059989445,
|
||||||
|
"tpot_p50_ms": 37.88820180411141,
|
||||||
|
"tpot_p95_ms": 38.25126371524308,
|
||||||
|
"tpot_p99_ms": 38.27709748982381,
|
||||||
|
"e2e_p50_ms": 78674.81265543029,
|
||||||
|
"e2e_p95_ms": 80982.15972669423,
|
||||||
|
"e2e_p99_ms": 80984.67170314165,
|
||||||
|
"itl_p50_ms": 34.52226798981428,
|
||||||
|
"itl_p95_ms": 53.908159665297724,
|
||||||
|
"itl_p99_ms": 59.665183436591185,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 65.8774013610091,
|
||||||
|
"request_tps": 0.6071884921628803,
|
||||||
|
"input_tps": 2487.0440638991577,
|
||||||
|
"output_tps": 155.44025399369735,
|
||||||
|
"total_tps": 2642.4843178928554,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 256.0,
|
||||||
|
"ttft_p50_ms": 1971.202465472743,
|
||||||
|
"ttft_p95_ms": 3824.870833219029,
|
||||||
|
"ttft_p99_ms": 3841.111220801249,
|
||||||
|
"tpot_p50_ms": 43.596625505654394,
|
||||||
|
"tpot_p95_ms": 48.1240840446131,
|
||||||
|
"tpot_p99_ms": 49.99360711309218,
|
||||||
|
"e2e_p50_ms": 13162.176689947955,
|
||||||
|
"e2e_p95_ms": 14974.834044638555,
|
||||||
|
"e2e_p99_ms": 14991.584688341245,
|
||||||
|
"itl_p50_ms": 34.47401651646942,
|
||||||
|
"itl_p95_ms": 54.485111916437745,
|
||||||
|
"itl_p99_ms": 60.43242933228612,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 762.6022247918881,
|
||||||
|
"request_tps": 0.052451984402374224,
|
||||||
|
"input_tps": 214.84332811212482,
|
||||||
|
"output_tps": 214.84332811212482,
|
||||||
|
"total_tps": 429.68665622424965,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 4096.0,
|
||||||
|
"ttft_p50_ms": 1038.2880391553044,
|
||||||
|
"ttft_p95_ms": 3893.24660198763,
|
||||||
|
"ttft_p99_ms": 3900.2761626965366,
|
||||||
|
"tpot_p50_ms": 36.76516193150564,
|
||||||
|
"tpot_p95_ms": 37.05582632169512,
|
||||||
|
"tpot_p99_ms": 37.09407625586006,
|
||||||
|
"e2e_p50_ms": 152243.55608585756,
|
||||||
|
"e2e_p95_ms": 154084.0833232156,
|
||||||
|
"e2e_p99_ms": 154084.53153002774,
|
||||||
|
"itl_p50_ms": 34.463146002963185,
|
||||||
|
"itl_p95_ms": 51.49366667028517,
|
||||||
|
"itl_p99_ms": 59.961400236934416,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 112.8795186879579,
|
||||||
|
"request_tps": 0.3543601218798184,
|
||||||
|
"input_tps": 1451.4590592197362,
|
||||||
|
"output_tps": 181.43238240246703,
|
||||||
|
"total_tps": 1632.8914416222033,
|
||||||
|
"mean_input_tokens": 4096.0,
|
||||||
|
"mean_output_tokens": 512.0,
|
||||||
|
"ttft_p50_ms": 1941.342479432933,
|
||||||
|
"ttft_p95_ms": 3813.8611613423564,
|
||||||
|
"ttft_p99_ms": 3815.337903073523,
|
||||||
|
"tpot_p50_ms": 40.17744154405986,
|
||||||
|
"tpot_p95_ms": 42.320974605816815,
|
||||||
|
"tpot_p99_ms": 43.39261491248045,
|
||||||
|
"e2e_p50_ms": 22431.270663044415,
|
||||||
|
"e2e_p95_ms": 24335.910513892304,
|
||||||
|
"e2e_p99_ms": 24337.53361347597,
|
||||||
|
"itl_p50_ms": 34.43968354258686,
|
||||||
|
"itl_p95_ms": 54.208886146079735,
|
||||||
|
"itl_p99_ms": 59.69965270254761,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"status": "COMPLETED",
|
||||||
|
"completed": 40,
|
||||||
|
"failed": 0,
|
||||||
|
"duration_s": 626.4431783319451,
|
||||||
|
"request_tps": 0.06385255899267604,
|
||||||
|
"input_tps": 4184.641306144017,
|
||||||
|
"output_tps": 65.38502040850027,
|
||||||
|
"total_tps": 4250.026326552517,
|
||||||
|
"mean_input_tokens": 65536.0,
|
||||||
|
"mean_output_tokens": 1024.0,
|
||||||
|
"ttft_p50_ms": 51137.51745200716,
|
||||||
|
"ttft_p95_ms": 96054.72948674572,
|
||||||
|
"ttft_p99_ms": 112415.82493769703,
|
||||||
|
"tpot_p50_ms": 70.99086488608475,
|
||||||
|
"tpot_p95_ms": 93.18777094192289,
|
||||||
|
"tpot_p99_ms": 94.39113497935946,
|
||||||
|
"e2e_p50_ms": 112995.05897297058,
|
||||||
|
"e2e_p95_ms": 168512.40640780888,
|
||||||
|
"e2e_p99_ms": 178801.59651178867,
|
||||||
|
"itl_p50_ms": 36.47732094395906,
|
||||||
|
"itl_p95_ms": 61.783290666062385,
|
||||||
|
"itl_p99_ms": 68.58176962938157,
|
||||||
|
"validation_errors": []
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user