commit only aggregated adaptive results; untrack tp*_dp* raw outputs and metrics
- keep adaptive_points/shapes/summary jsonl+md, run_manifest.json, shapes.tsv - untrack tp*_dp*/ subdirs (raw_outputs, metrics, logs, gpu_logs) and ignore them in .gitignore - update NEW_PLATFORM_GUIDE commit convention accordingly
This commit is contained in:
yy-fighting
parent
8652a685e6
commit
b01640aa20
6
.gitignore
vendored
6
.gitignore
vendored
@ -30,8 +30,10 @@ logs/
|
|||||||
# 数据集（可重新下载）
|
# 数据集（可重新下载）
|
||||||
datasets/
|
datasets/
|
||||||
|
|
||||||
# 原始请求级输出（.jsonl）需要提交入库，用作测得数据的云端存储，
|
# 原始请求级输出与中间产物不入库：只提交 adaptive_results 下的汇总级文件
|
||||||
# 因此这里不再忽略 raw_outputs/ 目录。
|
# （adaptive_points.jsonl / adaptive_shapes.jsonl / adaptive_summary.* / run_manifest.json / shapes.tsv），
|
||||||
|
# tp*_dp*/ 子目录（raw_outputs、metrics、logs、gpu_logs、server_cmd.txt）一律忽略。
|
||||||
|
experiments/*/adaptive_results/*/tp*_dp*/
|
||||||
|
|
||||||
# 实验级原始结果目录：保留 report.md / results.json / raw_outputs，忽略日志
|
# 实验级原始结果目录：保留 report.md / results.json / raw_outputs，忽略日志
|
||||||
experiments/*/results/*/logs/
|
experiments/*/results/*/logs/
|
||||||
|
|||||||
@ -109,7 +109,7 @@ tmux new-session -d -s <chip>-<engine>-adaptive \
|
|||||||
1. **分析**：用 Python 从 `adaptive_points.jsonl` / `adaptive_shapes.jsonl` 实算（SLO：TTFT P95 < 3s 且 TPOT P95 < 50ms，严格小于），禁止目测估算。
|
1. **分析**：用 Python 从 `adaptive_points.jsonl` / `adaptive_shapes.jsonl` 实算（SLO：TTFT P95 < 3s 且 TPOT P95 < 50ms，严格小于），禁止目测估算。
|
||||||
2. **飞书报告**：按飞书 wiki「显卡性能报告 / 性能报告编写指南（README）」的骨架写到「显卡性能报告/<平台>」节点下；图表用 `envs/charts/bin/python` + matplotlib 生成。
|
2. **飞书报告**：按飞书 wiki「显卡性能报告 / 性能报告编写指南（README）」的骨架写到「显卡性能报告/<平台>」节点下；图表用 `envs/charts/bin/python` + matplotlib 生成。
|
||||||
3. **多维表格**：在「模型推理适配 Bench 迭代跟踪」的对应平台表（没有则按现有 schema 新建）按"每成功探测点一行"回填；`SLO达标状态` 按上述口径判定。
|
3. **多维表格**：在「模型推理适配 Bench 迭代跟踪」的对应平台表（没有则按现有 schema 新建）按"每成功探测点一行"回填；`SLO达标状态` 按上述口径判定。
|
||||||
4. **提交**：实验代码 + `adaptive_results/<run_id>/` 的 jsonl/summary/manifest 入库；日志与 raw_outputs 不入库。
|
4. **提交**：实验代码 + `adaptive_results/<run_id>/` 下的汇总级文件入库——`adaptive_points.jsonl`、`adaptive_shapes.jsonl`、`adaptive_summary.jsonl/md`、`run_manifest.json`、`shapes.tsv`，这些是云端共享的数据资产；`tp*_dp*/` 子目录（raw_outputs 逐请求数据、metrics、logs、gpu_logs、server_cmd.txt）与中间日志一律不入库。
|
||||||
|
|
||||||
## 8. 完成检查清单
|
## 8. 完成检查清单
|
||||||
|
|
||||||
|
|||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 560,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 271.93914603099984,
|
|
||||||
"request_tps": 2.059284248602305,
|
|
||||||
"input_tps": 2108.70707056876,
|
|
||||||
"output_tps": 4217.41414113752,
|
|
||||||
"total_tps": 6326.121211706281,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 2360.8657174918335,
|
|
||||||
"ttft_p95_ms": 4785.738604894141,
|
|
||||||
"ttft_p99_ms": 5083.200499021332,
|
|
||||||
"tpot_p50_ms": 25.078506466781775,
|
|
||||||
"tpot_p95_ms": 26.261017000906556,
|
|
||||||
"tpot_p99_ms": 26.82069243732855,
|
|
||||||
"e2e_p50_ms": 54802.16980399564,
|
|
||||||
"e2e_p95_ms": 55957.59202675035,
|
|
||||||
"e2e_p99_ms": 55983.62555622443,
|
|
||||||
"itl_p50_ms": 24.729957003728487,
|
|
||||||
"itl_p95_ms": 26.741590403253216,
|
|
||||||
"itl_p99_ms": 29.055896277423017,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 82.26222992698604,
|
|
||||||
"request_tps": 0.9724997738452514,
|
|
||||||
"input_tps": 995.8397684175375,
|
|
||||||
"output_tps": 995.8397684175375,
|
|
||||||
"total_tps": 1991.679536835075,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 544.7934150070068,
|
|
||||||
"ttft_p95_ms": 881.7404834531771,
|
|
||||||
"ttft_p99_ms": 897.7118983241962,
|
|
||||||
"tpot_p50_ms": 15.400015403226577,
|
|
||||||
"tpot_p95_ms": 15.83289659242619,
|
|
||||||
"tpot_p99_ms": 16.04854842089475,
|
|
||||||
"e2e_p50_ms": 16476.469576002273,
|
|
||||||
"e2e_p95_ms": 16701.17062768695,
|
|
||||||
"e2e_p99_ms": 16705.017296750884,
|
|
||||||
"itl_p50_ms": 15.259214495017659,
|
|
||||||
"itl_p95_ms": 16.13290750174201,
|
|
||||||
"itl_p99_ms": 17.04316050017951,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 25.27050317999965,
|
|
||||||
"request_tps": 3.165746223182292,
|
|
||||||
"input_tps": 3241.724132538667,
|
|
||||||
"output_tps": 405.2155165673334,
|
|
||||||
"total_tps": 3646.9396491060006,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 1051.9641595019493,
|
|
||||||
"ttft_p95_ms": 7164.641508652858,
|
|
||||||
"ttft_p99_ms": 7186.009613565839,
|
|
||||||
"tpot_p50_ms": 18.72120111024348,
|
|
||||||
"tpot_p95_ms": 65.18447974448227,
|
|
||||||
"tpot_p99_ms": 66.09945481066043,
|
|
||||||
"e2e_p50_ms": 3493.628591997549,
|
|
||||||
"e2e_p95_ms": 9117.589514059364,
|
|
||||||
"e2e_p99_ms": 9138.18436251444,
|
|
||||||
"itl_p50_ms": 15.28507299371995,
|
|
||||||
"itl_p95_ms": 16.032998800073983,
|
|
||||||
"itl_p99_ms": 21.76707776481644,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 160.96535958899767,
|
|
||||||
"request_tps": 0.4970013436696486,
|
|
||||||
"input_tps": 508.92937591772017,
|
|
||||||
"output_tps": 1017.8587518354403,
|
|
||||||
"total_tps": 1526.7881277531606,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 408.2802780030761,
|
|
||||||
"ttft_p95_ms": 804.0736109498539,
|
|
||||||
"ttft_p99_ms": 871.2221907320782,
|
|
||||||
"tpot_p50_ms": 15.45796976209212,
|
|
||||||
"tpot_p95_ms": 15.706438855319314,
|
|
||||||
"tpot_p99_ms": 15.828019318818155,
|
|
||||||
"e2e_p50_ms": 32192.607425007736,
|
|
||||||
"e2e_p95_ms": 32558.569978662126,
|
|
||||||
"e2e_p99_ms": 32571.827183209243,
|
|
||||||
"itl_p50_ms": 15.350678993854672,
|
|
||||||
"itl_p95_ms": 16.041272398433648,
|
|
||||||
"itl_p99_ms": 16.693045923602767,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 25.495733696006937,
|
|
||||||
"request_tps": 3.137779871482159,
|
|
||||||
"input_tps": 3213.086588397731,
|
|
||||||
"output_tps": 803.2716470994327,
|
|
||||||
"total_tps": 4016.3582354971636,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 839.0470390004339,
|
|
||||||
"ttft_p95_ms": 1269.5357547512679,
|
|
||||||
"ttft_p99_ms": 1545.07869652909,
|
|
||||||
"tpot_p50_ms": 16.542784652961465,
|
|
||||||
"tpot_p95_ms": 18.45956203372243,
|
|
||||||
"tpot_p99_ms": 19.536147781054822,
|
|
||||||
"e2e_p50_ms": 5053.470186998311,
|
|
||||||
"e2e_p95_ms": 5453.463989494776,
|
|
||||||
"e2e_p99_ms": 5468.152181307523,
|
|
||||||
"itl_p50_ms": 15.194430503470358,
|
|
||||||
"itl_p95_ms": 15.815716297220208,
|
|
||||||
"itl_p99_ms": 16.660370378667714,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 316.64033691798977,
|
|
||||||
"request_tps": 0.2526525861445129,
|
|
||||||
"input_tps": 258.7162482119812,
|
|
||||||
"output_tps": 1034.864992847925,
|
|
||||||
"total_tps": 1293.581241059906,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 575.7432910031639,
|
|
||||||
"ttft_p95_ms": 887.6251749068615,
|
|
||||||
"ttft_p99_ms": 907.8684282244649,
|
|
||||||
"tpot_p50_ms": 15.333096270205825,
|
|
||||||
"tpot_p95_ms": 15.443372411818606,
|
|
||||||
"tpot_p99_ms": 15.483845902724966,
|
|
||||||
"e2e_p50_ms": 63438.04139450367,
|
|
||||||
"e2e_p95_ms": 63812.25281149527,
|
|
||||||
"e2e_p99_ms": 63831.44774455199,
|
|
||||||
"itl_p50_ms": 15.270748001057655,
|
|
||||||
"itl_p95_ms": 15.853451890870927,
|
|
||||||
"itl_p99_ms": 16.49769142619334,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 63.555366341999616,
|
|
||||||
"request_tps": 1.2587450061967969,
|
|
||||||
"input_tps": 1288.95488634552,
|
|
||||||
"output_tps": 644.47744317276,
|
|
||||||
"total_tps": 1933.4323295182799,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 651.5386349929031,
|
|
||||||
"ttft_p95_ms": 16920.61746209365,
|
|
||||||
"ttft_p99_ms": 16942.941599875485,
|
|
||||||
"tpot_p50_ms": 15.848476987276518,
|
|
||||||
"tpot_p95_ms": 23.578188225724393,
|
|
||||||
"tpot_p99_ms": 29.306536829046802,
|
|
||||||
"e2e_p50_ms": 8805.075798496546,
|
|
||||||
"e2e_p95_ms": 24636.574015000224,
|
|
||||||
"e2e_p99_ms": 24650.812979813927,
|
|
||||||
"itl_p50_ms": 15.365997998742387,
|
|
||||||
"itl_p95_ms": 16.487579100066792,
|
|
||||||
"itl_p99_ms": 18.37696666130795,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 653.2824726870022,
|
|
||||||
"request_tps": 0.12245851273332609,
|
|
||||||
"input_tps": 16050.882180982517,
|
|
||||||
"output_tps": 15.674689629865739,
|
|
||||||
"total_tps": 16066.556870612381,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 55664.04897299799,
|
|
||||||
"ttft_p95_ms": 118594.41714475105,
|
|
||||||
"ttft_p99_ms": 139872.23895933607,
|
|
||||||
"tpot_p50_ms": 544.0846560747494,
|
|
||||||
"tpot_p95_ms": 902.7456552543442,
|
|
||||||
"tpot_p99_ms": 958.5754168834561,
|
|
||||||
"e2e_p50_ms": 129924.55754899856,
|
|
||||||
"e2e_p95_ms": 185606.83365028963,
|
|
||||||
"e2e_p99_ms": 243322.26077544419,
|
|
||||||
"itl_p50_ms": 16.123155495733954,
|
|
||||||
"itl_p95_ms": 1741.9904875576322,
|
|
||||||
"itl_p99_ms": 1853.2892793747305,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 702.479465484008,
|
|
||||||
"request_tps": 0.11388233241078448,
|
|
||||||
"input_tps": 14926.785073746343,
|
|
||||||
"output_tps": 29.153877097160827,
|
|
||||||
"total_tps": 14955.938950843505,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 57824.69000250421,
|
|
||||||
"ttft_p95_ms": 127985.28046924763,
|
|
||||||
"ttft_p99_ms": 151725.3745295175,
|
|
||||||
"tpot_p50_ms": 253.09320623725628,
|
|
||||||
"tpot_p95_ms": 525.102048641203,
|
|
||||||
"tpot_p99_ms": 619.6555398026641,
|
|
||||||
"e2e_p50_ms": 138529.73486200062,
|
|
||||||
"e2e_p95_ms": 184438.2270416012,
|
|
||||||
"e2e_p99_ms": 185996.91346610835,
|
|
||||||
"itl_p50_ms": 15.797007494256832,
|
|
||||||
"itl_p95_ms": 1590.6224329526603,
|
|
||||||
"itl_p99_ms": 1839.2413294677688,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 667.9211692050012,
|
|
||||||
"request_tps": 0.1197746136646944,
|
|
||||||
"input_tps": 15699.098162258824,
|
|
||||||
"output_tps": 61.32460219632353,
|
|
||||||
"total_tps": 15760.422764455148,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 55981.32588399312,
|
|
||||||
"ttft_p95_ms": 121882.91273160065,
|
|
||||||
"ttft_p99_ms": 140760.88047753967,
|
|
||||||
"tpot_p50_ms": 141.8939732465815,
|
|
||||||
"tpot_p95_ms": 235.749314115342,
|
|
||||||
"tpot_p99_ms": 254.5075704539367,
|
|
||||||
"e2e_p50_ms": 132681.28659549984,
|
|
||||||
"e2e_p95_ms": 175247.86795944526,
|
|
||||||
"e2e_p99_ms": 234920.12947427476,
|
|
||||||
"itl_p50_ms": 15.690204003476538,
|
|
||||||
"itl_p95_ms": 1249.2242270018323,
|
|
||||||
"itl_p99_ms": 1797.7115144065467,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 111.50969841099868,
|
|
||||||
"request_tps": 0.7174263865833329,
|
|
||||||
"input_tps": 11754.313917781326,
|
|
||||||
"output_tps": 734.6446198613329,
|
|
||||||
"total_tps": 12488.958537642658,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 4311.803800497728,
|
|
||||||
"ttft_p95_ms": 7437.313835108215,
|
|
||||||
"ttft_p99_ms": 9130.135508555395,
|
|
||||||
"tpot_p50_ms": 17.93213432404895,
|
|
||||||
"tpot_p95_ms": 22.04782108866086,
|
|
||||||
"tpot_p99_ms": 22.43329354472889,
|
|
||||||
"e2e_p50_ms": 21698.23511350114,
|
|
||||||
"e2e_p95_ms": 24644.28777230205,
|
|
||||||
"e2e_p99_ms": 26412.721827901405,
|
|
||||||
"itl_p50_ms": 15.224864997435361,
|
|
||||||
"itl_p95_ms": 15.731755192973651,
|
|
||||||
"itl_p99_ms": 16.860831878148,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 56.001648912992096,
|
|
||||||
"request_tps": 1.4285293657030234,
|
|
||||||
"input_tps": 23405.025127678335,
|
|
||||||
"output_tps": 182.851758809987,
|
|
||||||
"total_tps": 23587.87688648832,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 5949.235703003069,
|
|
||||||
"ttft_p95_ms": 9227.944630107957,
|
|
||||||
"ttft_p99_ms": 11097.429403630083,
|
|
||||||
"tpot_p50_ms": 39.72050276768026,
|
|
||||||
"tpot_p95_ms": 72.95558840476615,
|
|
||||||
"tpot_p99_ms": 80.88457267013912,
|
|
||||||
"e2e_p50_ms": 11065.538848997676,
|
|
||||||
"e2e_p95_ms": 13142.54083404594,
|
|
||||||
"e2e_p99_ms": 14229.989901815272,
|
|
||||||
"itl_p50_ms": 15.273823504685424,
|
|
||||||
"itl_p95_ms": 17.06440234847832,
|
|
||||||
"itl_p99_ms": 1069.5194568477748,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 188.87208026400185,
|
|
||||||
"request_tps": 0.4235671036617879,
|
|
||||||
"input_tps": 6939.723426394733,
|
|
||||||
"output_tps": 867.4654282993416,
|
|
||||||
"total_tps": 7807.1888546940745,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 2707.821463998698,
|
|
||||||
"ttft_p95_ms": 6159.3108133580245,
|
|
||||||
"ttft_p99_ms": 7476.327504011569,
|
|
||||||
"tpot_p50_ms": 16.789645155595252,
|
|
||||||
"tpot_p95_ms": 19.235649654987466,
|
|
||||||
"tpot_p99_ms": 19.809279338319755,
|
|
||||||
"e2e_p50_ms": 37331.52540450101,
|
|
||||||
"e2e_p95_ms": 41633.905589940696,
|
|
||||||
"e2e_p99_ms": 41644.779238444025,
|
|
||||||
"itl_p50_ms": 15.280855994205922,
|
|
||||||
"itl_p95_ms": 18.66166050749598,
|
|
||||||
"itl_p99_ms": 19.038321707921565,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 64.32757946698985,
|
|
||||||
"request_tps": 1.2436345446676191,
|
|
||||||
"input_tps": 20375.708379834272,
|
|
||||||
"output_tps": 318.3704434349105,
|
|
||||||
"total_tps": 20694.07882326918,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 4847.543320000113,
|
|
||||||
"ttft_p95_ms": 8945.170914454502,
|
|
||||||
"ttft_p99_ms": 10200.10803171811,
|
|
||||||
"tpot_p50_ms": 30.697892317645408,
|
|
||||||
"tpot_p95_ms": 45.12670790233909,
|
|
||||||
"tpot_p99_ms": 46.21028031682417,
|
|
||||||
"e2e_p50_ms": 12845.963567502622,
|
|
||||||
"e2e_p95_ms": 15142.341455946007,
|
|
||||||
"e2e_p99_ms": 15201.770440481196,
|
|
||||||
"itl_p50_ms": 15.241901011904702,
|
|
||||||
"itl_p95_ms": 16.105174794211056,
|
|
||||||
"itl_p99_ms": 806.8968500866322,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 72.41434827599733,
|
|
||||||
"request_tps": 1.1047534349835064,
|
|
||||||
"input_tps": 18100.28027876977,
|
|
||||||
"output_tps": 565.6337587115553,
|
|
||||||
"total_tps": 18665.914037481325,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 3977.608113505994,
|
|
||||||
"ttft_p95_ms": 6827.726796599017,
|
|
||||||
"ttft_p99_ms": 7064.793197970429,
|
|
||||||
"tpot_p50_ms": 20.6349563414749,
|
|
||||||
"tpot_p95_ms": 26.606408149222858,
|
|
||||||
"tpot_p99_ms": 27.689232515706706,
|
|
||||||
"e2e_p50_ms": 14308.850026493019,
|
|
||||||
"e2e_p95_ms": 14974.010983452172,
|
|
||||||
"e2e_p99_ms": 14998.68528622057,
|
|
||||||
"itl_p50_ms": 15.342150996730197,
|
|
||||||
"itl_p95_ms": 16.593942245526705,
|
|
||||||
"itl_p99_ms": 19.848780611646326,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 1511.6757434350002,
|
|
||||||
"request_tps": 0.05292140219053523,
|
|
||||||
"input_tps": 13873.028055835668,
|
|
||||||
"output_tps": 6.77393948038851,
|
|
||||||
"total_tps": 13879.801995316056,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 70474.41060249548,
|
|
||||||
"ttft_p95_ms": 206680.64425829216,
|
|
||||||
"ttft_p99_ms": 263259.9471526309,
|
|
||||||
"tpot_p50_ms": 1770.9045697204808,
|
|
||||||
"tpot_p95_ms": 1859.1631874913205,
|
|
||||||
"tpot_p99_ms": 1883.4251564144938,
|
|
||||||
"e2e_p50_ms": 293508.3537885075,
|
|
||||||
"e2e_p95_ms": 418159.4987228506,
|
|
||||||
"e2e_p99_ms": 491266.18717678235,
|
|
||||||
"itl_p50_ms": 1741.0211320020608,
|
|
||||||
"itl_p95_ms": 2297.1252548493794,
|
|
||||||
"itl_p99_ms": 2395.285054906126,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 1557.1239152909984,
|
|
||||||
"request_tps": 0.0513767717613209,
|
|
||||||
"input_tps": 13468.112456599705,
|
|
||||||
"output_tps": 13.15245357089815,
|
|
||||||
"total_tps": 13481.264910170603,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 128086.70317550423,
|
|
||||||
"ttft_p95_ms": 293147.14115650055,
|
|
||||||
"ttft_p99_ms": 327246.22319525003,
|
|
||||||
"tpot_p50_ms": 694.1213535215974,
|
|
||||||
"tpot_p95_ms": 1110.7064755825186,
|
|
||||||
"tpot_p99_ms": 1121.9122485736023,
|
|
||||||
"e2e_p50_ms": 320114.9868900029,
|
|
||||||
"e2e_p95_ms": 442507.49014469614,
|
|
||||||
"e2e_p99_ms": 540593.0392589162,
|
|
||||||
"itl_p50_ms": 16.51875649986323,
|
|
||||||
"itl_p95_ms": 2135.694057152432,
|
|
||||||
"itl_p99_ms": 2348.4220405294036,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 84.46414080199611,
|
|
||||||
"request_tps": 0.9471475023648069,
|
|
||||||
"input_tps": 3879.516169686249,
|
|
||||||
"output_tps": 969.8790424215623,
|
|
||||||
"total_tps": 4849.395212107812,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 755.6758849968901,
|
|
||||||
"ttft_p95_ms": 1184.9343558955297,
|
|
||||||
"ttft_p99_ms": 1330.5567739978123,
|
|
||||||
"tpot_p50_ms": 15.835003849462446,
|
|
||||||
"tpot_p95_ms": 16.285198607422213,
|
|
||||||
"tpot_p99_ms": 16.55195564271857,
|
|
||||||
"e2e_p50_ms": 16962.706995996996,
|
|
||||||
"e2e_p95_ms": 17168.894911993266,
|
|
||||||
"e2e_p99_ms": 17178.149301250814,
|
|
||||||
"itl_p50_ms": 15.242625500832219,
|
|
||||||
"itl_p95_ms": 16.113438254251378,
|
|
||||||
"itl_p99_ms": 16.882939507922856,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 44.308171736003715,
|
|
||||||
"request_tps": 1.8055360188782963,
|
|
||||||
"input_tps": 7395.475533325502,
|
|
||||||
"output_tps": 231.10861041642193,
|
|
||||||
"total_tps": 7626.584143741924,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 1465.1815819961485,
|
|
||||||
"ttft_p95_ms": 23821.693100642005,
|
|
||||||
"ttft_p99_ms": 24259.730825575534,
|
|
||||||
"tpot_p50_ms": 22.75063312988532,
|
|
||||||
"tpot_p95_ms": 194.92663006346027,
|
|
||||||
"tpot_p99_ms": 197.04311093216822,
|
|
||||||
"e2e_p50_ms": 4635.849632992176,
|
|
||||||
"e2e_p95_ms": 26182.2721034383,
|
|
||||||
"e2e_p99_ms": 26191.62205229819,
|
|
||||||
"itl_p50_ms": 15.226024501316715,
|
|
||||||
"itl_p95_ms": 16.216387940949062,
|
|
||||||
"itl_p99_ms": 414.4044824814774,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 161.3202662160038,
|
|
||||||
"request_tps": 0.4959079344245688,
|
|
||||||
"input_tps": 2031.2388994030339,
|
|
||||||
"output_tps": 1015.6194497015169,
|
|
||||||
"total_tps": 3046.858349104551,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 766.6918885006453,
|
|
||||||
"ttft_p95_ms": 1151.3519862077374,
|
|
||||||
"ttft_p99_ms": 1174.3669241876341,
|
|
||||||
"tpot_p50_ms": 15.32661797704204,
|
|
||||||
"tpot_p95_ms": 15.672662826941613,
|
|
||||||
"tpot_p99_ms": 15.77215861052261,
|
|
||||||
"e2e_p50_ms": 32366.54036600521,
|
|
||||||
"e2e_p95_ms": 32729.988757940737,
|
|
||||||
"e2e_p99_ms": 32742.016700837994,
|
|
||||||
"itl_p50_ms": 15.24984300340293,
|
|
||||||
"itl_p95_ms": 15.832478402444394,
|
|
||||||
"itl_p99_ms": 16.448479092650818,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 32.527519440001925,
|
|
||||||
"request_tps": 2.4594559123256423,
|
|
||||||
"input_tps": 10073.931416885831,
|
|
||||||
"output_tps": 629.6207135553644,
|
|
||||||
"total_tps": 10703.552130441196,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 1533.099703505286,
|
|
||||||
"ttft_p95_ms": 2719.3758142995644,
|
|
||||||
"ttft_p99_ms": 2959.5566013005737,
|
|
||||||
"tpot_p50_ms": 19.392000529407206,
|
|
||||||
"tpot_p95_ms": 23.99643604938105,
|
|
||||||
"tpot_p99_ms": 24.379314330662144,
|
|
||||||
"e2e_p50_ms": 6627.143721503671,
|
|
||||||
"e2e_p95_ms": 6811.619238646381,
|
|
||||||
"e2e_p99_ms": 6835.494205219293,
|
|
||||||
"itl_p50_ms": 15.377331001218408,
|
|
||||||
"itl_p95_ms": 16.272442803892773,
|
|
||||||
"itl_p99_ms": 18.94523348833901,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 316.8896808080026,
|
|
||||||
"request_tps": 0.252453787059322,
|
|
||||||
"input_tps": 1034.0507117949828,
|
|
||||||
"output_tps": 1034.0507117949828,
|
|
||||||
"total_tps": 2068.1014235899656,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 658.9246715011541,
|
|
||||||
"ttft_p95_ms": 1005.5294943493209,
|
|
||||||
"ttft_p99_ms": 1026.091546237294,
|
|
||||||
"tpot_p50_ms": 15.320106219048359,
|
|
||||||
"tpot_p95_ms": 15.378604879511569,
|
|
||||||
"tpot_p99_ms": 15.45278130631852,
|
|
||||||
"e2e_p50_ms": 63453.26198550174,
|
|
||||||
"e2e_p95_ms": 63513.736396850436,
|
|
||||||
"e2e_p99_ms": 63521.74632662762,
|
|
||||||
"itl_p50_ms": 15.256563012371771,
|
|
||||||
"itl_p95_ms": 15.777264998177998,
|
|
||||||
"itl_p99_ms": 16.359419524669644,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 48.17216287400515,
|
|
||||||
"request_tps": 1.6607101534809827,
|
|
||||||
"input_tps": 6802.268788658105,
|
|
||||||
"output_tps": 850.2835985822632,
|
|
||||||
"total_tps": 7652.552387240369,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 1226.4635695100878,
|
|
||||||
"ttft_p95_ms": 1866.224592289654,
|
|
||||||
"ttft_p99_ms": 2426.2945817218856,
|
|
||||||
"tpot_p50_ms": 16.140000863013167,
|
|
||||||
"tpot_p95_ms": 17.640392132493695,
|
|
||||||
"tpot_p99_ms": 18.638964293663246,
|
|
||||||
"e2e_p50_ms": 9504.232979503286,
|
|
||||||
"e2e_p95_ms": 10157.524074796675,
|
|
||||||
"e2e_p99_ms": 10178.896629959345,
|
|
||||||
"itl_p50_ms": 15.288406488252804,
|
|
||||||
"itl_p95_ms": 15.857895543740598,
|
|
||||||
"itl_p99_ms": 17.156079869600944,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 376.9979162859963,
|
|
||||||
"request_tps": 0.21220276437631763,
|
|
||||||
"input_tps": 13906.920366166352,
|
|
||||||
"output_tps": 217.29563072134926,
|
|
||||||
"total_tps": 14124.215996887702,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 25524.604717495095,
|
|
||||||
"ttft_p95_ms": 56727.187289898575,
|
|
||||||
"ttft_p99_ms": 65385.280855649864,
|
|
||||||
"tpot_p50_ms": 45.977114398337235,
|
|
||||||
"tpot_p95_ms": 68.72763672496647,
|
|
||||||
"tpot_p99_ms": 71.65886099602294,
|
|
||||||
"e2e_p50_ms": 72648.60455600865,
|
|
||||||
"e2e_p95_ms": 102850.95250014929,
|
|
||||||
"e2e_p99_ms": 128267.6254233617,
|
|
||||||
"itl_p50_ms": 15.504026996495668,
|
|
||||||
"itl_p95_ms": 16.275252601190004,
|
|
||||||
"itl_p99_ms": 1222.4654011012171,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 263.11688939499436,
|
|
||||||
"request_tps": 0.30404737675316235,
|
|
||||||
"input_tps": 19926.048882895248,
|
|
||||||
"output_tps": 38.91806422440478,
|
|
||||||
"total_tps": 19964.966947119654,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 21783.06187000271,
|
|
||||||
"ttft_p95_ms": 48708.6885400975,
|
|
||||||
"ttft_p99_ms": 53530.08918488397,
|
|
||||||
"tpot_p50_ms": 208.85715699601343,
|
|
||||||
"tpot_p95_ms": 353.9625207657282,
|
|
||||||
"tpot_p99_ms": 368.66045906663794,
|
|
||||||
"e2e_p50_ms": 53675.07621049299,
|
|
||||||
"e2e_p95_ms": 64500.535845247214,
|
|
||||||
"e2e_p99_ms": 78019.4687606334,
|
|
||||||
"itl_p50_ms": 15.624218496668618,
|
|
||||||
"itl_p95_ms": 1377.797994745198,
|
|
||||||
"itl_p99_ms": 1593.279839480674,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 318.6807930810028,
|
|
||||||
"request_tps": 0.2510348967898592,
|
|
||||||
"input_tps": 16451.82299602021,
|
|
||||||
"output_tps": 64.26493357820395,
|
|
||||||
"total_tps": 16516.087929598416,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 25208.387847495032,
|
|
||||||
"ttft_p95_ms": 57570.60906205442,
|
|
||||||
"ttft_p99_ms": 64559.6027293129,
|
|
||||||
"tpot_p50_ms": 139.20286317058046,
|
|
||||||
"tpot_p95_ms": 218.68997558943673,
|
|
||||||
"tpot_p99_ms": 237.28532163909728,
|
|
||||||
"e2e_p50_ms": 62259.9117714999,
|
|
||||||
"e2e_p95_ms": 93757.1761358522,
|
|
||||||
"e2e_p99_ms": 109198.36791741903,
|
|
||||||
"itl_p50_ms": 15.581318992190063,
|
|
||||||
"itl_p95_ms": 1060.9013718029018,
|
|
||||||
"itl_p99_ms": 1579.705025086296,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 337.8231221129972,
|
|
||||||
"request_tps": 0.23681031511289238,
|
|
||||||
"input_tps": 15519.600811238515,
|
|
||||||
"output_tps": 121.2468813378009,
|
|
||||||
"total_tps": 15640.847692576317,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 24695.115143498697,
|
|
||||||
"ttft_p95_ms": 57546.68756374739,
|
|
||||||
"ttft_p99_ms": 64468.70614748111,
|
|
||||||
"tpot_p50_ms": 76.04405581409218,
|
|
||||||
"tpot_p95_ms": 120.77805700313024,
|
|
||||||
"tpot_p99_ms": 135.3599259530104,
|
|
||||||
"e2e_p50_ms": 66170.59861750022,
|
|
||||||
"e2e_p95_ms": 92553.88260725957,
|
|
||||||
"e2e_p99_ms": 109859.61571042174,
|
|
||||||
"itl_p50_ms": 15.599313002894633,
|
|
||||||
"itl_p95_ms": 824.2479222499242,
|
|
||||||
"itl_p99_ms": 1410.173195308016,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 97.25075286399806,
|
|
||||||
"request_tps": 0.051413483728936615,
|
|
||||||
"input_tps": 6738.86813931918,
|
|
||||||
"output_tps": 13.161851834607774,
|
|
||||||
"total_tps": 6752.029991153788,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 16969.752281991532,
|
|
||||||
"ttft_p95_ms": 17121.84501980082,
|
|
||||||
"ttft_p99_ms": 17125.887859960785,
|
|
||||||
"tpot_p50_ms": 9.60843350975683,
|
|
||||||
"tpot_p95_ms": 9.634889066653034,
|
|
||||||
"tpot_p99_ms": 9.637108200772976,
|
|
||||||
"e2e_p50_ms": 19427.356342988787,
|
|
||||||
"e2e_p95_ms": 19571.998772607185,
|
|
||||||
"e2e_p99_ms": 19575.258900129702,
|
|
||||||
"itl_p50_ms": 10.00273649697192,
|
|
||||||
"itl_p95_ms": 10.19823560709483,
|
|
||||||
"itl_p99_ms": 11.044782063545426,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 110.12223424699914,
|
|
||||||
"request_tps": 0.04540409150058859,
|
|
||||||
"input_tps": 5951.205081165148,
|
|
||||||
"output_tps": 23.24689484830136,
|
|
||||||
"total_tps": 5974.45197601345,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 16962.474314001156,
|
|
||||||
"ttft_p95_ms": 17155.541892390465,
|
|
||||||
"ttft_p99_ms": 17171.51858087047,
|
|
||||||
"tpot_p50_ms": 9.821654945213728,
|
|
||||||
"tpot_p95_ms": 9.835613834436566,
|
|
||||||
"tpot_p99_ms": 9.837956189974383,
|
|
||||||
"e2e_p50_ms": 21989.969162998023,
|
|
||||||
"e2e_p95_ms": 22173.37846419541,
|
|
||||||
"e2e_p99_ms": 22190.17843683483,
|
|
||||||
"itl_p50_ms": 9.989651007344946,
|
|
||||||
"itl_p95_ms": 10.151538599166088,
|
|
||||||
"itl_p99_ms": 10.469720991677605,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 215.86107532100868,
|
|
||||||
"request_tps": 0.02316304591999489,
|
|
||||||
"input_tps": 6072.05350965114,
|
|
||||||
"output_tps": 2.964869877759346,
|
|
||||||
"total_tps": 6075.0183795289,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 42189.58094799018,
|
|
||||||
"ttft_p95_ms": 42206.01177979843,
|
|
||||||
"ttft_p99_ms": 42207.67736715847,
|
|
||||||
"tpot_p50_ms": 9.026367795253979,
|
|
||||||
"tpot_p95_ms": 9.051597466134083,
|
|
||||||
"tpot_p99_ms": 9.053308627064942,
|
|
||||||
"e2e_p50_ms": 43335.3122970002,
|
|
||||||
"e2e_p95_ms": 43353.944078995846,
|
|
||||||
"e2e_p99_ms": 43357.12344699539,
|
|
||||||
"itl_p50_ms": 10.197708994382992,
|
|
||||||
"itl_p95_ms": 10.398621491185622,
|
|
||||||
"itl_p99_ms": 11.979914264811663,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 222.6894240649999,
|
|
||||||
"request_tps": 0.022452795057481358,
|
|
||||||
"input_tps": 5885.865507548393,
|
|
||||||
"output_tps": 5.7479155347152275,
|
|
||||||
"total_tps": 5891.6134230831085,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 41992.54209799983,
|
|
||||||
"ttft_p95_ms": 42385.26861999417,
|
|
||||||
"ttft_p99_ms": 42405.3075527912,
|
|
||||||
"tpot_p50_ms": 9.564196976419884,
|
|
||||||
"tpot_p95_ms": 9.778543265853717,
|
|
||||||
"tpot_p99_ms": 9.797764013144736,
|
|
||||||
"e2e_p50_ms": 44431.4123269869,
|
|
||||||
"e2e_p95_ms": 44819.5239008026,
|
|
||||||
"e2e_p99_ms": 44832.609473761986,
|
|
||||||
"itl_p50_ms": 10.17798750399379,
|
|
||||||
"itl_p95_ms": 10.34693365872954,
|
|
||||||
"itl_p99_ms": 10.991512061009418,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 99.20981571500306,
|
|
||||||
"request_tps": 1.6127436468547327,
|
|
||||||
"input_tps": 1651.4494943792463,
|
|
||||||
"output_tps": 1651.4494943792463,
|
|
||||||
"total_tps": 3302.8989887584926,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1029.0807624987792,
|
|
||||||
"ttft_p95_ms": 1823.7564543480403,
|
|
||||||
"ttft_p99_ms": 1851.6841998742893,
|
|
||||||
"tpot_p50_ms": 18.9562778137816,
|
|
||||||
"tpot_p95_ms": 19.887580249860058,
|
|
||||||
"tpot_p99_ms": 19.937121249460077,
|
|
||||||
"e2e_p50_ms": 20452.989998499106,
|
|
||||||
"e2e_p95_ms": 20839.25912065315,
|
|
||||||
"e2e_p99_ms": 20861.500803281087,
|
|
||||||
"itl_p50_ms": 18.491245500626974,
|
|
||||||
"itl_p95_ms": 19.586568658996843,
|
|
||||||
"itl_p99_ms": 20.498206740885504,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 197.1833312999952,
|
|
||||||
"request_tps": 0.8114276138107009,
|
|
||||||
"input_tps": 830.9018765421578,
|
|
||||||
"output_tps": 1661.8037530843155,
|
|
||||||
"total_tps": 2492.705629626473,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 656.5913640006329,
|
|
||||||
"ttft_p95_ms": 1290.7357379466703,
|
|
||||||
"ttft_p99_ms": 1304.3246922609978,
|
|
||||||
"tpot_p50_ms": 18.853423953588848,
|
|
||||||
"tpot_p95_ms": 19.17381441135775,
|
|
||||||
"tpot_p99_ms": 19.217155080916477,
|
|
||||||
"e2e_p50_ms": 39465.7350474954,
|
|
||||||
"e2e_p95_ms": 39581.901998698595,
|
|
||||||
"e2e_p99_ms": 39587.70705576069,
|
|
||||||
"itl_p50_ms": 18.70039399364032,
|
|
||||||
"itl_p95_ms": 19.59088460134808,
|
|
||||||
"itl_p99_ms": 20.509281637496308,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 58.333280092992936,
|
|
||||||
"request_tps": 2.742859646241964,
|
|
||||||
"input_tps": 2808.6882777517712,
|
|
||||||
"output_tps": 702.1720694379428,
|
|
||||||
"total_tps": 3510.8603471897136,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 1302.453588003118,
|
|
||||||
"ttft_p95_ms": 25865.006712951435,
|
|
||||||
"ttft_p99_ms": 25871.150527912396,
|
|
||||||
"tpot_p50_ms": 22.315067135302037,
|
|
||||||
"tpot_p95_ms": 116.87548523570477,
|
|
||||||
"tpot_p99_ms": 116.9072278467824,
|
|
||||||
"e2e_p50_ms": 7041.645481498563,
|
|
||||||
"e2e_p95_ms": 30671.992690498882,
|
|
||||||
"e2e_p99_ms": 30678.4747339561,
|
|
||||||
"itl_p50_ms": 18.8346959912451,
|
|
||||||
"itl_p95_ms": 20.653201197274026,
|
|
||||||
"itl_p99_ms": 25.224988315603593,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 390.9258493729867,
|
|
||||||
"request_tps": 0.40928477934275004,
|
|
||||||
"input_tps": 419.10761404697604,
|
|
||||||
"output_tps": 1676.4304561879042,
|
|
||||||
"total_tps": 2095.5380702348803,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 1059.6694034975371,
|
|
||||||
"ttft_p95_ms": 1531.6532846380142,
|
|
||||||
"ttft_p99_ms": 1558.6441262916196,
|
|
||||||
"tpot_p50_ms": 18.821365679975997,
|
|
||||||
"tpot_p95_ms": 19.016937038081775,
|
|
||||||
"tpot_p99_ms": 19.073814857857453,
|
|
||||||
"e2e_p50_ms": 77972.91494200181,
|
|
||||||
"e2e_p95_ms": 78736.66218614817,
|
|
||||||
"e2e_p99_ms": 78754.35213367702,
|
|
||||||
"itl_p50_ms": 18.73732249805471,
|
|
||||||
"itl_p95_ms": 19.672631401772378,
|
|
||||||
"itl_p99_ms": 20.632040495693218,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 107.34333604600397,
|
|
||||||
"request_tps": 1.490544321553683,
|
|
||||||
"input_tps": 6105.269541083886,
|
|
||||||
"output_tps": 1526.3173852709715,
|
|
||||||
"total_tps": 7631.5869263548575,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1697.2974899981637,
|
|
||||||
"ttft_p95_ms": 2827.3090714494147,
|
|
||||||
"ttft_p99_ms": 3041.8824867265357,
|
|
||||||
"tpot_p50_ms": 19.041262989738744,
|
|
||||||
"tpot_p95_ms": 20.483581416088622,
|
|
||||||
"tpot_p99_ms": 20.796884204547236,
|
|
||||||
"e2e_p50_ms": 21322.126428000047,
|
|
||||||
"e2e_p95_ms": 21977.338839244476,
|
|
||||||
"e2e_p99_ms": 21997.641324478464,
|
|
||||||
"itl_p50_ms": 18.680188004509546,
|
|
||||||
"itl_p95_ms": 19.772065294091586,
|
|
||||||
"itl_p99_ms": 21.03478136268678,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 203.61964830600482,
|
|
||||||
"request_tps": 0.7857787857463927,
|
|
||||||
"input_tps": 3218.5499064172245,
|
|
||||||
"output_tps": 1609.2749532086123,
|
|
||||||
"total_tps": 4827.824859625837,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 1823.4369839992723,
|
|
||||||
"ttft_p95_ms": 2880.811745747633,
|
|
||||||
"ttft_p99_ms": 2898.1182565601193,
|
|
||||||
"tpot_p50_ms": 18.92700290888939,
|
|
||||||
"tpot_p95_ms": 19.648448953663472,
|
|
||||||
"tpot_p99_ms": 19.892009308511824,
|
|
||||||
"e2e_p50_ms": 40591.624035492714,
|
|
||||||
"e2e_p95_ms": 41278.74980429624,
|
|
||||||
"e2e_p99_ms": 41301.732580115786,
|
|
||||||
"itl_p50_ms": 18.753395997919142,
|
|
||||||
"itl_p95_ms": 19.575604447891237,
|
|
||||||
"itl_p99_ms": 20.460386879567523,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 66.26290708300075,
|
|
||||||
"request_tps": 2.414623913188481,
|
|
||||||
"input_tps": 9890.299548420018,
|
|
||||||
"output_tps": 618.1437217762511,
|
|
||||||
"total_tps": 10508.443270196269,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 3032.6789590035332,
|
|
||||||
"ttft_p95_ms": 5014.513322354456,
|
|
||||||
"ttft_p99_ms": 14411.20142712994,
|
|
||||||
"tpot_p50_ms": 25.094734256865696,
|
|
||||||
"tpot_p95_ms": 108.72940146214187,
|
|
||||||
"tpot_p99_ms": 110.79764089783636,
|
|
||||||
"e2e_p50_ms": 9434.836903004907,
|
|
||||||
"e2e_p95_ms": 29872.09643335227,
|
|
||||||
"e2e_p99_ms": 29884.739060596476,
|
|
||||||
"itl_p50_ms": 18.683340997085907,
|
|
||||||
"itl_p95_ms": 19.744293195253704,
|
|
||||||
"itl_p99_ms": 26.305865632602966,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 389.95699894400605,
|
|
||||||
"request_tps": 0.4103016497543987,
|
|
||||||
"input_tps": 1680.595557394017,
|
|
||||||
"output_tps": 1680.595557394017,
|
|
||||||
"total_tps": 3361.191114788034,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 1123.4332419990096,
|
|
||||||
"ttft_p95_ms": 1676.0268465513946,
|
|
||||||
"ttft_p99_ms": 1717.2243972464637,
|
|
||||||
"tpot_p50_ms": 18.85827924127029,
|
|
||||||
"tpot_p95_ms": 19.032755866286788,
|
|
||||||
"tpot_p99_ms": 19.08965374972383,
|
|
||||||
"e2e_p50_ms": 77915.13783499977,
|
|
||||||
"e2e_p95_ms": 78629.53818804745,
|
|
||||||
"e2e_p99_ms": 78646.88334376202,
|
|
||||||
"itl_p50_ms": 18.702409994148184,
|
|
||||||
"itl_p95_ms": 19.55858354413067,
|
|
||||||
"itl_p99_ms": 20.443303814536193,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 63.387327321004705,
|
|
||||||
"request_tps": 2.52416384728972,
|
|
||||||
"input_tps": 10338.975118498693,
|
|
||||||
"output_tps": 1292.3718898123366,
|
|
||||||
"total_tps": 11631.347008311028,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 2139.6533224979066,
|
|
||||||
"ttft_p95_ms": 3791.57651143978,
|
|
||||||
"ttft_p99_ms": 4309.74766367377,
|
|
||||||
"tpot_p50_ms": 20.01632839431788,
|
|
||||||
"tpot_p95_ms": 24.519710790119547,
|
|
||||||
"tpot_p99_ms": 25.129288290261652,
|
|
||||||
"e2e_p50_ms": 12805.874345496704,
|
|
||||||
"e2e_p95_ms": 13800.694600059069,
|
|
||||||
"e2e_p99_ms": 13819.40531841843,
|
|
||||||
"itl_p50_ms": 18.705550508457236,
|
|
||||||
"itl_p95_ms": 20.301984495017674,
|
|
||||||
"itl_p99_ms": 21.65679321915376,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 108.07686458299577,
|
|
||||||
"request_tps": 2.2206417712617497,
|
|
||||||
"input_tps": 2273.9371737720317,
|
|
||||||
"output_tps": 2273.9371737720317,
|
|
||||||
"total_tps": 4547.874347544063,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1411.8448265071493,
|
|
||||||
"ttft_p95_ms": 2348.3333344942366,
|
|
||||||
"ttft_p99_ms": 2881.5906707521935,
|
|
||||||
"tpot_p50_ms": 19.57261614369442,
|
|
||||||
"tpot_p95_ms": 20.474906701757522,
|
|
||||||
"tpot_p99_ms": 20.999123584376957,
|
|
||||||
"e2e_p50_ms": 21663.715992996003,
|
|
||||||
"e2e_p95_ms": 22356.912894507695,
|
|
||||||
"e2e_p99_ms": 22380.086564826343,
|
|
||||||
"itl_p50_ms": 18.948694996652193,
|
|
||||||
"itl_p95_ms": 20.273402400198393,
|
|
||||||
"itl_p99_ms": 21.75828296574764,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 200.68862108800386,
|
|
||||||
"request_tps": 1.195882450628617,
|
|
||||||
"input_tps": 1224.5836294437038,
|
|
||||||
"output_tps": 2449.1672588874076,
|
|
||||||
"total_tps": 3673.7508883311116,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 833.4129540016875,
|
|
||||||
"ttft_p95_ms": 1647.5724648524192,
|
|
||||||
"ttft_p99_ms": 1656.5784132984118,
|
|
||||||
"tpot_p50_ms": 19.151686465800864,
|
|
||||||
"tpot_p95_ms": 19.465789479505947,
|
|
||||||
"tpot_p99_ms": 19.511620899212012,
|
|
||||||
"e2e_p50_ms": 40142.91101550043,
|
|
||||||
"e2e_p95_ms": 40502.659226708056,
|
|
||||||
"e2e_p99_ms": 40515.6316231635,
|
|
||||||
"itl_p50_ms": 18.947302000015043,
|
|
||||||
"itl_p95_ms": 20.123915805015713,
|
|
||||||
"itl_p99_ms": 21.517608713475045,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 395.37849693700264,
|
|
||||||
"request_tps": 0.6070132843826361,
|
|
||||||
"input_tps": 621.5816032078194,
|
|
||||||
"output_tps": 2486.3264128312776,
|
|
||||||
"total_tps": 3107.9080160390968,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 1144.204484502552,
|
|
||||||
"ttft_p95_ms": 1999.596584997198,
|
|
||||||
"ttft_p99_ms": 2035.2585943651502,
|
|
||||||
"tpot_p50_ms": 18.99256894627527,
|
|
||||||
"tpot_p95_ms": 19.224950862944212,
|
|
||||||
"tpot_p99_ms": 19.230104005436754,
|
|
||||||
"e2e_p50_ms": 78907.89274750568,
|
|
||||||
"e2e_p95_ms": 79839.60575450546,
|
|
||||||
"e2e_p99_ms": 79862.91444651186,
|
|
||||||
"itl_p50_ms": 18.858423005440272,
|
|
||||||
"itl_p95_ms": 20.01599159266334,
|
|
||||||
"itl_p99_ms": 21.22574228560552,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 114.72205163900799,
|
|
||||||
"request_tps": 2.092012795893852,
|
|
||||||
"input_tps": 8568.884411981218,
|
|
||||||
"output_tps": 2142.2211029953046,
|
|
||||||
"total_tps": 10711.105514976523,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 2002.436676499201,
|
|
||||||
"ttft_p95_ms": 5319.875290898926,
|
|
||||||
"ttft_p99_ms": 6160.708812107041,
|
|
||||||
"tpot_p50_ms": 19.824865686698722,
|
|
||||||
"tpot_p95_ms": 23.0216869557243,
|
|
||||||
"tpot_p99_ms": 23.20206612341515,
|
|
||||||
"e2e_p50_ms": 21923.11628299649,
|
|
||||||
"e2e_p95_ms": 25281.340099497902,
|
|
||||||
"e2e_p99_ms": 25297.796992440562,
|
|
||||||
"itl_p50_ms": 18.851815999369137,
|
|
||||||
"itl_p95_ms": 20.00413840287365,
|
|
||||||
"itl_p99_ms": 21.528728755656623,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 209.64498886499496,
|
|
||||||
"request_tps": 1.1447924479346976,
|
|
||||||
"input_tps": 4689.069866740521,
|
|
||||||
"output_tps": 2344.5349333702607,
|
|
||||||
"total_tps": 7033.604800110782,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 1850.1262010031496,
|
|
||||||
"ttft_p95_ms": 5557.007933600835,
|
|
||||||
"ttft_p99_ms": 5728.514949967212,
|
|
||||||
"tpot_p50_ms": 19.032042742548224,
|
|
||||||
"tpot_p95_ms": 20.8632619918908,
|
|
||||||
"tpot_p99_ms": 21.040059630580483,
|
|
||||||
"e2e_p50_ms": 40867.48082550184,
|
|
||||||
"e2e_p95_ms": 44286.82900869899,
|
|
||||||
"e2e_p99_ms": 44307.41918329746,
|
|
||||||
"itl_p50_ms": 18.80523749423446,
|
|
||||||
"itl_p95_ms": 20.01578960043844,
|
|
||||||
"itl_p99_ms": 21.193549290037485,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 402.4469851330068,
|
|
||||||
"request_tps": 0.596351839785012,
|
|
||||||
"input_tps": 2442.6571357594094,
|
|
||||||
"output_tps": 2442.6571357594094,
|
|
||||||
"total_tps": 4885.314271518819,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 2047.3789034949732,
|
|
||||||
"ttft_p95_ms": 4081.0472775985545,
|
|
||||||
"ttft_p99_ms": 4647.203718657256,
|
|
||||||
"tpot_p50_ms": 19.02283846043863,
|
|
||||||
"tpot_p95_ms": 19.43320571131982,
|
|
||||||
"tpot_p99_ms": 19.899421790027564,
|
|
||||||
"e2e_p50_ms": 80315.7795865045,
|
|
||||||
"e2e_p95_ms": 81844.74676285972,
|
|
||||||
"e2e_p99_ms": 81865.64722886324,
|
|
||||||
"itl_p50_ms": 18.825628008926287,
|
|
||||||
"itl_p95_ms": 19.975638002506457,
|
|
||||||
"itl_p99_ms": 21.221037310315303,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 240,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 77.56397197001206,
|
|
||||||
"request_tps": 3.0942200857479203,
|
|
||||||
"input_tps": 12673.925471223482,
|
|
||||||
"output_tps": 1584.2406839029352,
|
|
||||||
"total_tps": 14258.166155126417,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 2946.3751199946273,
|
|
||||||
"ttft_p95_ms": 6820.920404194476,
|
|
||||||
"ttft_p99_ms": 8050.301623321715,
|
|
||||||
"tpot_p50_ms": 23.094741272006033,
|
|
||||||
"tpot_p95_ms": 31.340273439141725,
|
|
||||||
"tpot_p99_ms": 33.989053702566764,
|
|
||||||
"e2e_p50_ms": 15109.904168501089,
|
|
||||||
"e2e_p95_ms": 19630.664032053755,
|
|
||||||
"e2e_p99_ms": 19649.577008009364,
|
|
||||||
"itl_p50_ms": 19.291372009320185,
|
|
||||||
"itl_p95_ms": 22.182088205590844,
|
|
||||||
"itl_p99_ms": 24.491042752633813,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 320,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 123.36385090599651,
|
|
||||||
"request_tps": 2.593952747501702,
|
|
||||||
"input_tps": 2656.207613441743,
|
|
||||||
"output_tps": 2656.207613441743,
|
|
||||||
"total_tps": 5312.415226883486,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1434.1024970053695,
|
|
||||||
"ttft_p95_ms": 3039.2205538046255,
|
|
||||||
"ttft_p99_ms": 3390.209743604064,
|
|
||||||
"tpot_p50_ms": 22.264150537145387,
|
|
||||||
"tpot_p95_ms": 23.93735160308324,
|
|
||||||
"tpot_p99_ms": 24.057023045675336,
|
|
||||||
"e2e_p50_ms": 24811.180071505078,
|
|
||||||
"e2e_p95_ms": 25911.275261299306,
|
|
||||||
"e2e_p99_ms": 25928.425582967902,
|
|
||||||
"itl_p50_ms": 21.702708501834422,
|
|
||||||
"itl_p95_ms": 23.22826191157219,
|
|
||||||
"itl_p99_ms": 25.138318116369188,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 320,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 231.29285078399698,
|
|
||||||
"request_tps": 1.3835274152024963,
|
|
||||||
"input_tps": 1416.7320731673562,
|
|
||||||
"output_tps": 2833.4641463347125,
|
|
||||||
"total_tps": 4250.196219502069,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 1245.4625594982645,
|
|
||||||
"ttft_p95_ms": 2417.803988052765,
|
|
||||||
"ttft_p99_ms": 2683.46135836342,
|
|
||||||
"tpot_p50_ms": 21.843712060574735,
|
|
||||||
"tpot_p95_ms": 22.34440391123519,
|
|
||||||
"tpot_p99_ms": 22.46978759702825,
|
|
||||||
"e2e_p50_ms": 46248.58569000207,
|
|
||||||
"e2e_p95_ms": 46951.28065479075,
|
|
||||||
"e2e_p99_ms": 46979.07613164149,
|
|
||||||
"itl_p50_ms": 21.61537500069244,
|
|
||||||
"itl_p95_ms": 23.96329630355467,
|
|
||||||
"itl_p99_ms": 25.871719614806352,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 320,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 443.57241661399894,
|
|
||||||
"request_tps": 0.7214154623110101,
|
|
||||||
"input_tps": 738.7294334064743,
|
|
||||||
"output_tps": 2954.917733625897,
|
|
||||||
"total_tps": 3693.6471670323713,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 1471.407580494997,
|
|
||||||
"ttft_p95_ms": 2597.0414660041592,
|
|
||||||
"ttft_p99_ms": 2976.737847158656,
|
|
||||||
"tpot_p50_ms": 21.723876868741236,
|
|
||||||
"tpot_p95_ms": 22.00444482439625,
|
|
||||||
"tpot_p99_ms": 22.07639913107326,
|
|
||||||
"e2e_p50_ms": 90742.18811100582,
|
|
||||||
"e2e_p95_ms": 91541.6523386979,
|
|
||||||
"e2e_p99_ms": 91559.50254334108,
|
|
||||||
"itl_p50_ms": 21.4493349994882,
|
|
||||||
"itl_p95_ms": 22.813913105346728,
|
|
||||||
"itl_p99_ms": 24.170049769309127,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 400,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 143.43459790300403,
|
|
||||||
"request_tps": 2.7887274468499945,
|
|
||||||
"input_tps": 2855.6569055743944,
|
|
||||||
"output_tps": 2855.6569055743944,
|
|
||||||
"total_tps": 5711.313811148789,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1577.1677885059034,
|
|
||||||
"ttft_p95_ms": 5509.251923993023,
|
|
||||||
"ttft_p99_ms": 20218.74289565443,
|
|
||||||
"tpot_p50_ms": 22.94427521847951,
|
|
||||||
"tpot_p95_ms": 40.54245714501829,
|
|
||||||
"tpot_p99_ms": 40.55054783283226,
|
|
||||||
"e2e_p50_ms": 24977.686929501942,
|
|
||||||
"e2e_p95_ms": 42717.079425647535,
|
|
||||||
"e2e_p99_ms": 42743.186112658295,
|
|
||||||
"itl_p50_ms": 21.95263899920974,
|
|
||||||
"itl_p95_ms": 23.614524496224476,
|
|
||||||
"itl_p99_ms": 25.837959441560088,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 400,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 235.45233723000274,
|
|
||||||
"request_tps": 1.6988576316796469,
|
|
||||||
"input_tps": 1739.6302148399584,
|
|
||||||
"output_tps": 3479.260429679917,
|
|
||||||
"total_tps": 5218.890644519875,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 1586.3785705005284,
|
|
||||||
"ttft_p95_ms": 3133.5033947099873,
|
|
||||||
"ttft_p99_ms": 3142.967526005523,
|
|
||||||
"tpot_p50_ms": 22.12632127210705,
|
|
||||||
"tpot_p95_ms": 22.91146984701974,
|
|
||||||
"tpot_p99_ms": 22.976990950927,
|
|
||||||
"e2e_p50_ms": 47054.46670300444,
|
|
||||||
"e2e_p95_ms": 48105.06032910416,
|
|
||||||
"e2e_p99_ms": 48135.96432496226,
|
|
||||||
"itl_p50_ms": 21.907794500293676,
|
|
||||||
"itl_p95_ms": 23.54792225087294,
|
|
||||||
"itl_p99_ms": 25.59503832002519,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 400,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 463.4503973050014,
|
|
||||||
"request_tps": 0.8630912872791345,
|
|
||||||
"input_tps": 883.8054781738338,
|
|
||||||
"output_tps": 3535.221912695335,
|
|
||||||
"total_tps": 4419.027390869169,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 2439.6097005082993,
|
|
||||||
"ttft_p95_ms": 3686.9959791976726,
|
|
||||||
"ttft_p99_ms": 3709.631283667256,
|
|
||||||
"tpot_p50_ms": 22.095354364345052,
|
|
||||||
"tpot_p95_ms": 22.518257637104984,
|
|
||||||
"tpot_p99_ms": 22.625587380977006,
|
|
||||||
"e2e_p50_ms": 92209.49949500209,
|
|
||||||
"e2e_p95_ms": 93844.80823813865,
|
|
||||||
"e2e_p99_ms": 93874.67774137796,
|
|
||||||
"itl_p50_ms": 21.87070349464193,
|
|
||||||
"itl_p95_ms": 23.618548751983326,
|
|
||||||
"itl_p99_ms": 25.773182249395177,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 40,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 341.0657499819936,
|
|
||||||
"request_tps": 0.11727943952775025,
|
|
||||||
"input_tps": 15372.05069778128,
|
|
||||||
"output_tps": 30.023536519104063,
|
|
||||||
"total_tps": 15402.074234300384,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 38767.44465551019,
|
|
||||||
"ttft_p95_ms": 67753.44985890988,
|
|
||||||
"ttft_p99_ms": 68613.70976785096,
|
|
||||||
"tpot_p50_ms": 99.49740432157162,
|
|
||||||
"tpot_p95_ms": 179.5039385709941,
|
|
||||||
"tpot_p99_ms": 227.61194875988735,
|
|
||||||
"e2e_p50_ms": 70759.12754750607,
|
|
||||||
"e2e_p95_ms": 71843.55354870277,
|
|
||||||
"e2e_p99_ms": 71887.88047623617,
|
|
||||||
"itl_p50_ms": 12.980383995454758,
|
|
||||||
"itl_p95_ms": 1125.97120024584,
|
|
||||||
"itl_p99_ms": 1602.8499752024072,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 40,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 368.12840555600997,
|
|
||||||
"request_tps": 0.10865773843120097,
|
|
||||||
"input_tps": 14241.987091654373,
|
|
||||||
"output_tps": 55.632762076774895,
|
|
||||||
"total_tps": 14297.619853731148,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 36897.75435599586,
|
|
||||||
"ttft_p95_ms": 68026.47409820274,
|
|
||||||
"ttft_p99_ms": 74667.961793141,
|
|
||||||
"tpot_p50_ms": 63.40141743053057,
|
|
||||||
"tpot_p95_ms": 116.81624726055898,
|
|
||||||
"tpot_p99_ms": 143.86918745584376,
|
|
||||||
"e2e_p50_ms": 74333.1425399956,
|
|
||||||
"e2e_p95_ms": 83933.02225654986,
|
|
||||||
"e2e_p99_ms": 104491.18695786949,
|
|
||||||
"itl_p50_ms": 12.98504300211789,
|
|
||||||
"itl_p95_ms": 18.927634652209232,
|
|
||||||
"itl_p99_ms": 1447.1019217558205,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 40,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 1143.3382095730049,
|
|
||||||
"request_tps": 0.03498527353068918,
|
|
||||||
"input_tps": 9171.179544428984,
|
|
||||||
"output_tps": 4.478115011928215,
|
|
||||||
"total_tps": 9175.65765944091,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 60434.31567850348,
|
|
||||||
"ttft_p95_ms": 171402.15240599928,
|
|
||||||
"ttft_p99_ms": 221100.73346630885,
|
|
||||||
"tpot_p50_ms": 1432.7522763937384,
|
|
||||||
"tpot_p95_ms": 1566.4298780566398,
|
|
||||||
"tpot_p99_ms": 1577.578224665822,
|
|
||||||
"e2e_p50_ms": 243043.1625314959,
|
|
||||||
"e2e_p95_ms": 344139.9401556495,
|
|
||||||
"e2e_p99_ms": 419677.4824362584,
|
|
||||||
"itl_p50_ms": 1394.7447990067303,
|
|
||||||
"itl_p95_ms": 1954.1919693932869,
|
|
||||||
"itl_p99_ms": 2118.3612401573937,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 40,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 818.7336148600007,
|
|
||||||
"request_tps": 0.04885593955591991,
|
|
||||||
"input_tps": 12807.291418947068,
|
|
||||||
"output_tps": 12.507120526315497,
|
|
||||||
"total_tps": 12819.798539473382,
|
|
||||||
"mean_input_tokens": 262144.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 105032.74306450476,
|
|
||||||
"ttft_p95_ms": 162785.75602490891,
|
|
||||||
"ttft_p99_ms": 165083.89961342298,
|
|
||||||
"tpot_p50_ms": 214.14210347058685,
|
|
||||||
"tpot_p95_ms": 422.65357399257476,
|
|
||||||
"tpot_p99_ms": 440.83721461195177,
|
|
||||||
"e2e_p50_ms": 164185.14075000712,
|
|
||||||
"e2e_p95_ms": 168861.28058029353,
|
|
||||||
"e2e_p99_ms": 169246.668313704,
|
|
||||||
"itl_p50_ms": 13.376075992709957,
|
|
||||||
"itl_p95_ms": 1720.7928034986253,
|
|
||||||
"itl_p99_ms": 2045.3944211971263,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 480,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 261.48785026300175,
|
|
||||||
"request_tps": 1.8356493409434549,
|
|
||||||
"input_tps": 1879.7049251260978,
|
|
||||||
"output_tps": 3759.4098502521956,
|
|
||||||
"total_tps": 5639.114775378293,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 2183.7680815006024,
|
|
||||||
"ttft_p95_ms": 3805.7011090502783,
|
|
||||||
"ttft_p99_ms": 4558.676817088854,
|
|
||||||
"tpot_p50_ms": 24.876903371278917,
|
|
||||||
"tpot_p95_ms": 25.891116515634604,
|
|
||||||
"tpot_p99_ms": 26.181815688893472,
|
|
||||||
"e2e_p50_ms": 53079.94003850763,
|
|
||||||
"e2e_p95_ms": 54830.756290954014,
|
|
||||||
"e2e_p99_ms": 54843.78045572128,
|
|
||||||
"itl_p50_ms": 24.47853601188399,
|
|
||||||
"itl_p95_ms": 26.07234639581293,
|
|
||||||
"itl_p99_ms": 27.980867308797315,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 480,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 524.4931549339963,
|
|
||||||
"request_tps": 0.9151692362132059,
|
|
||||||
"input_tps": 937.1332978823228,
|
|
||||||
"output_tps": 3748.5331915292913,
|
|
||||||
"total_tps": 4685.666489411614,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 2751.5591720002703,
|
|
||||||
"ttft_p95_ms": 4227.864060657878,
|
|
||||||
"ttft_p99_ms": 4536.041375702771,
|
|
||||||
"tpot_p50_ms": 24.893937553724363,
|
|
||||||
"tpot_p95_ms": 25.43370572488336,
|
|
||||||
"tpot_p99_ms": 25.74416057706036,
|
|
||||||
"e2e_p50_ms": 104575.3224170112,
|
|
||||||
"e2e_p95_ms": 106446.31684675842,
|
|
||||||
"e2e_p99_ms": 106478.32935052022,
|
|
||||||
"itl_p50_ms": 24.56568699562922,
|
|
||||||
"itl_p95_ms": 27.559315803227946,
|
|
||||||
"itl_p99_ms": 30.286136486683972,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1 +0,0 @@
|
|||||||
vllm serve /data1/hf_models/DeepSeek-V4-Flash --trust-remote-code --kv-cache-dtype fp8 --block-size 256 --tensor-parallel-size 2 --gpu-memory-utilization 0.9 --max-model-len 1048576 --max-num-seqs 128 --no-enable-flashinfer-autotune --host 0.0.0.0 --port 30030 --data-parallel-size 4
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 560,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 143.62631484199665,
|
|
||||||
"request_tps": 3.899006951588615,
|
|
||||||
"input_tps": 3992.5831184267417,
|
|
||||||
"output_tps": 3992.5831184267417,
|
|
||||||
"total_tps": 7985.1662368534835,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1571.4164454911952,
|
|
||||||
"ttft_p95_ms": 4123.8772861033785,
|
|
||||||
"ttft_p99_ms": 4936.747990879667,
|
|
||||||
"tpot_p50_ms": 25.948950879762915,
|
|
||||||
"tpot_p95_ms": 28.919239563878318,
|
|
||||||
"tpot_p99_ms": 29.696389907960754,
|
|
||||||
"e2e_p50_ms": 28112.99813849473,
|
|
||||||
"e2e_p95_ms": 31451.82638790211,
|
|
||||||
"e2e_p99_ms": 31470.12428371294,
|
|
||||||
"itl_p50_ms": 25.19894449505955,
|
|
||||||
"itl_p95_ms": 28.669426793931052,
|
|
||||||
"itl_p99_ms": 31.52743758109864,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 70.41322445700644,
|
|
||||||
"request_tps": 1.1361502134992716,
|
|
||||||
"input_tps": 1163.417818623254,
|
|
||||||
"output_tps": 1163.417818623254,
|
|
||||||
"total_tps": 2326.835637246508,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 421.1962485060212,
|
|
||||||
"ttft_p95_ms": 677.2944091026147,
|
|
||||||
"ttft_p99_ms": 691.2584483872342,
|
|
||||||
"tpot_p50_ms": 12.164721241445186,
|
|
||||||
"tpot_p95_ms": 15.51125149233277,
|
|
||||||
"tpot_p99_ms": 15.721278570422475,
|
|
||||||
"e2e_p50_ms": 12909.268272502231,
|
|
||||||
"e2e_p95_ms": 16315.028272505877,
|
|
||||||
"e2e_p99_ms": 16321.569995415775,
|
|
||||||
"itl_p50_ms": 12.13237400224898,
|
|
||||||
"itl_p95_ms": 15.468433196656404,
|
|
||||||
"itl_p99_ms": 16.074650280061178,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 52.56799242499983,
|
|
||||||
"request_tps": 1.5218386000594972,
|
|
||||||
"input_tps": 1558.3627264609252,
|
|
||||||
"output_tps": 194.79534080761564,
|
|
||||||
"total_tps": 1753.158067268541,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 937.427844000922,
|
|
||||||
"ttft_p95_ms": 1428.1466051499592,
|
|
||||||
"ttft_p99_ms": 18393.34938969926,
|
|
||||||
"tpot_p50_ms": 19.7043993346339,
|
|
||||||
"tpot_p95_ms": 181.49350103310894,
|
|
||||||
"tpot_p99_ms": 182.55337019226835,
|
|
||||||
"e2e_p50_ms": 3306.429382006172,
|
|
||||||
"e2e_p95_ms": 23952.145700105757,
|
|
||||||
"e2e_p99_ms": 23955.10612057842,
|
|
||||||
"itl_p50_ms": 14.576606998161878,
|
|
||||||
"itl_p95_ms": 15.284183846233645,
|
|
||||||
"itl_p99_ms": 20.36303154163767,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 137.77100664599857,
|
|
||||||
"request_tps": 0.5806736986800084,
|
|
||||||
"input_tps": 594.6098674483286,
|
|
||||||
"output_tps": 1189.2197348966572,
|
|
||||||
"total_tps": 1783.8296023449857,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 509.03518099948997,
|
|
||||||
"ttft_p95_ms": 817.3343080510676,
|
|
||||||
"ttft_p99_ms": 821.0908090611338,
|
|
||||||
"tpot_p50_ms": 12.053855090129739,
|
|
||||||
"tpot_p95_ms": 15.30965305794184,
|
|
||||||
"tpot_p99_ms": 15.332859359112767,
|
|
||||||
"e2e_p50_ms": 25319.949440003256,
|
|
||||||
"e2e_p95_ms": 31725.24674650267,
|
|
||||||
"e2e_p99_ms": 31736.754303508642,
|
|
||||||
"itl_p50_ms": 12.262283504242077,
|
|
||||||
"itl_p95_ms": 15.428829462325668,
|
|
||||||
"itl_p99_ms": 15.964222954062278,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 48.607205292995786,
|
|
||||||
"request_tps": 1.6458465266162476,
|
|
||||||
"input_tps": 1685.3468432550376,
|
|
||||||
"output_tps": 421.3367108137594,
|
|
||||||
"total_tps": 2106.683554068797,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 782.0743444972322,
|
|
||||||
"ttft_p95_ms": 12196.891359947767,
|
|
||||||
"ttft_p99_ms": 12266.554353885149,
|
|
||||||
"tpot_p50_ms": 15.961954474515354,
|
|
||||||
"tpot_p95_ms": 58.48537955195449,
|
|
||||||
"tpot_p99_ms": 60.24822486883116,
|
|
||||||
"e2e_p50_ms": 8633.398842990573,
|
|
||||||
"e2e_p95_ms": 16283.070811999643,
|
|
||||||
"e2e_p99_ms": 16288.124511987555,
|
|
||||||
"itl_p50_ms": 14.68896050209878,
|
|
||||||
"itl_p95_ms": 15.627986006438732,
|
|
||||||
"itl_p99_ms": 16.497610333171902,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 271.16081980899617,
|
|
||||||
"request_tps": 0.29502787333491415,
|
|
||||||
"input_tps": 302.1085422949521,
|
|
||||||
"output_tps": 1208.4341691798084,
|
|
||||||
"total_tps": 1510.5427114747604,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 627.1758095026598,
|
|
||||||
"ttft_p95_ms": 960.0995247899846,
|
|
||||||
"ttft_p99_ms": 971.8106476443063,
|
|
||||||
"tpot_p50_ms": 11.913466125030766,
|
|
||||||
"tpot_p95_ms": 15.119212534556272,
|
|
||||||
"tpot_p99_ms": 15.134965707044527,
|
|
||||||
"e2e_p50_ms": 49374.50365599943,
|
|
||||||
"e2e_p95_ms": 62329.371708749386,
|
|
||||||
"e2e_p99_ms": 62340.975158869114,
|
|
||||||
"itl_p50_ms": 12.052630001562648,
|
|
||||||
"itl_p95_ms": 15.1747610012535,
|
|
||||||
"itl_p99_ms": 15.672232958022514,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 37.51114835499902,
|
|
||||||
"request_tps": 2.132699304294655,
|
|
||||||
"input_tps": 2183.8840875977266,
|
|
||||||
"output_tps": 1091.9420437988633,
|
|
||||||
"total_tps": 3275.82613139659,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 565.4102255066391,
|
|
||||||
"ttft_p95_ms": 804.2641735068173,
|
|
||||||
"ttft_p99_ms": 840.2388689920183,
|
|
||||||
"tpot_p50_ms": 12.695262536184165,
|
|
||||||
"tpot_p95_ms": 16.153524499127695,
|
|
||||||
"tpot_p99_ms": 16.279702295489752,
|
|
||||||
"e2e_p50_ms": 6936.251829996763,
|
|
||||||
"e2e_p95_ms": 8593.082947545918,
|
|
||||||
"e2e_p99_ms": 8608.418859792146,
|
|
||||||
"itl_p50_ms": 12.14090500434395,
|
|
||||||
"itl_p95_ms": 15.356732595682843,
|
|
||||||
"itl_p99_ms": 15.861063279880907,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 782.5810133770137,
|
|
||||||
"request_tps": 0.10222583813371851,
|
|
||||||
"input_tps": 13398.945055862752,
|
|
||||||
"output_tps": 13.084907281115969,
|
|
||||||
"total_tps": 13412.029963143868,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 58004.37130450155,
|
|
||||||
"ttft_p95_ms": 110342.86055744666,
|
|
||||||
"ttft_p99_ms": 145009.39971607734,
|
|
||||||
"tpot_p50_ms": 726.4293738111247,
|
|
||||||
"tpot_p95_ms": 1148.8756013172838,
|
|
||||||
"tpot_p99_ms": 1185.2487841417542,
|
|
||||||
"e2e_p50_ms": 145932.8185465056,
|
|
||||||
"e2e_p95_ms": 251712.73757864692,
|
|
||||||
"e2e_p99_ms": 293697.5071116265,
|
|
||||||
"itl_p50_ms": 1063.0883960038773,
|
|
||||||
"itl_p95_ms": 1341.7487418533708,
|
|
||||||
"itl_p99_ms": 1398.4825665256358,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 81.79191887499474,
|
|
||||||
"request_tps": 0.9780917369387875,
|
|
||||||
"input_tps": 16025.055018005094,
|
|
||||||
"output_tps": 1001.5659386253184,
|
|
||||||
"total_tps": 17026.620956630413,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 999.8386475053849,
|
|
||||||
"ttft_p95_ms": 3296.2886180946934,
|
|
||||||
"ttft_p99_ms": 4053.4733163932106,
|
|
||||||
"tpot_p50_ms": 15.505134818183233,
|
|
||||||
"tpot_p95_ms": 18.54402336778967,
|
|
||||||
"tpot_p99_ms": 18.632150808406323,
|
|
||||||
"e2e_p50_ms": 16877.44059450779,
|
|
||||||
"e2e_p95_ms": 19530.405244355643,
|
|
||||||
"e2e_p99_ms": 19544.808635528025,
|
|
||||||
"itl_p50_ms": 15.109817002667114,
|
|
||||||
"itl_p95_ms": 16.077759441395756,
|
|
||||||
"itl_p99_ms": 16.771573076403005,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 61.62385746999644,
|
|
||||||
"request_tps": 1.2981985108438008,
|
|
||||||
"input_tps": 21269.684401664832,
|
|
||||||
"output_tps": 166.1694093880065,
|
|
||||||
"total_tps": 21435.85381105284,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 6200.594059002469,
|
|
||||||
"ttft_p95_ms": 9982.714110245432,
|
|
||||||
"ttft_p99_ms": 12083.678980552644,
|
|
||||||
"tpot_p50_ms": 46.56020286221444,
|
|
||||||
"tpot_p95_ms": 83.96291618064566,
|
|
||||||
"tpot_p99_ms": 94.82308160958443,
|
|
||||||
"e2e_p50_ms": 11807.360419501492,
|
|
||||||
"e2e_p95_ms": 14518.52319239406,
|
|
||||||
"e2e_p99_ms": 15406.82603497858,
|
|
||||||
"itl_p50_ms": 14.998138998635113,
|
|
||||||
"itl_p95_ms": 19.290680799167607,
|
|
||||||
"itl_p99_ms": 913.8215772807598,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 136.13383251799678,
|
|
||||||
"request_tps": 0.5876570028205448,
|
|
||||||
"input_tps": 9628.172334211806,
|
|
||||||
"output_tps": 1203.5215417764757,
|
|
||||||
"total_tps": 10831.693875988281,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 952.042529002938,
|
|
||||||
"ttft_p95_ms": 1976.2296090004384,
|
|
||||||
"ttft_p99_ms": 2348.7864647468086,
|
|
||||||
"tpot_p50_ms": 12.17877617391463,
|
|
||||||
"tpot_p95_ms": 15.498852642595702,
|
|
||||||
"tpot_p99_ms": 15.918200653907457,
|
|
||||||
"e2e_p50_ms": 25627.524990501115,
|
|
||||||
"e2e_p95_ms": 32975.65399566083,
|
|
||||||
"e2e_p99_ms": 32987.294595711865,
|
|
||||||
"itl_p50_ms": 11.984888005827088,
|
|
||||||
"itl_p95_ms": 15.32018614525441,
|
|
||||||
"itl_p99_ms": 15.952927653561346,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 48.023830674996134,
|
|
||||||
"request_tps": 1.665839623277958,
|
|
||||||
"input_tps": 27293.116387786064,
|
|
||||||
"output_tps": 426.45494355915724,
|
|
||||||
"total_tps": 27719.57133134522,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 4218.072452502383,
|
|
||||||
"ttft_p95_ms": 7202.70946295495,
|
|
||||||
"ttft_p99_ms": 9814.834247549588,
|
|
||||||
"tpot_p50_ms": 20.867704264716902,
|
|
||||||
"tpot_p95_ms": 43.5013626262719,
|
|
||||||
"tpot_p99_ms": 48.00088939475241,
|
|
||||||
"e2e_p50_ms": 10424.10009400919,
|
|
||||||
"e2e_p95_ms": 13663.989264789414,
|
|
||||||
"e2e_p99_ms": 16127.52558444379,
|
|
||||||
"itl_p50_ms": 14.975312995375134,
|
|
||||||
"itl_p95_ms": 15.900831998442298,
|
|
||||||
"itl_p99_ms": 664.3842326034808,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 56.001584703000844,
|
|
||||||
"request_tps": 1.4285310036184244,
|
|
||||||
"input_tps": 23405.051963284266,
|
|
||||||
"output_tps": 731.4078738526333,
|
|
||||||
"total_tps": 24136.4598371369,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 2150.0688629894285,
|
|
||||||
"ttft_p95_ms": 4893.693922802049,
|
|
||||||
"ttft_p99_ms": 5737.629971327547,
|
|
||||||
"tpot_p50_ms": 16.641745551845485,
|
|
||||||
"tpot_p95_ms": 22.60100227660779,
|
|
||||||
"tpot_p99_ms": 25.976594107668582,
|
|
||||||
"e2e_p50_ms": 10941.78693700087,
|
|
||||||
"e2e_p95_ms": 14099.559878905711,
|
|
||||||
"e2e_p99_ms": 14118.464289340336,
|
|
||||||
"itl_p50_ms": 15.120271986234002,
|
|
||||||
"itl_p95_ms": 16.094310994958505,
|
|
||||||
"itl_p99_ms": 19.090893008979037,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 74.54846192999685,
|
|
||||||
"request_tps": 1.0731274385663692,
|
|
||||||
"input_tps": 4395.529988367848,
|
|
||||||
"output_tps": 1098.882497091962,
|
|
||||||
"total_tps": 5494.41248545981,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 664.4156365000526,
|
|
||||||
"ttft_p95_ms": 974.0240385996003,
|
|
||||||
"ttft_p99_ms": 982.1483760114643,
|
|
||||||
"tpot_p50_ms": 14.938762804988556,
|
|
||||||
"tpot_p95_ms": 15.543498240374383,
|
|
||||||
"tpot_p99_ms": 15.80483324938253,
|
|
||||||
"e2e_p50_ms": 16048.860944996704,
|
|
||||||
"e2e_p95_ms": 16527.93401440722,
|
|
||||||
"e2e_p99_ms": 16534.28529751647,
|
|
||||||
"itl_p50_ms": 14.674316000309773,
|
|
||||||
"itl_p95_ms": 15.759427801822312,
|
|
||||||
"itl_p99_ms": 16.294228279730305,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 63.706278053999995,
|
|
||||||
"request_tps": 1.2557632064486453,
|
|
||||||
"input_tps": 5143.606093613651,
|
|
||||||
"output_tps": 160.7376904254266,
|
|
||||||
"total_tps": 5304.343784039078,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 2722.8040709960624,
|
|
||||||
"ttft_p95_ms": 26840.74661414561,
|
|
||||||
"ttft_p99_ms": 27275.76444137885,
|
|
||||||
"tpot_p50_ms": 20.65726973231667,
|
|
||||||
"tpot_p95_ms": 151.34337404136696,
|
|
||||||
"tpot_p99_ms": 207.36058258637357,
|
|
||||||
"e2e_p50_ms": 5354.688991996227,
|
|
||||||
"e2e_p95_ms": 29170.501739544852,
|
|
||||||
"e2e_p99_ms": 29177.230598040624,
|
|
||||||
"itl_p50_ms": 14.814941496297251,
|
|
||||||
"itl_p95_ms": 15.711286505393218,
|
|
||||||
"itl_p99_ms": 182.99884561245784,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 138.75295940799697,
|
|
||||||
"request_tps": 0.5765642789986448,
|
|
||||||
"input_tps": 2361.607286778449,
|
|
||||||
"output_tps": 1180.8036433892246,
|
|
||||||
"total_tps": 3542.4109301676735,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 632.4307624963694,
|
|
||||||
"ttft_p95_ms": 959.6396220949828,
|
|
||||||
"ttft_p99_ms": 969.5783248801308,
|
|
||||||
"tpot_p50_ms": 12.087159364439152,
|
|
||||||
"tpot_p95_ms": 15.38081124682696,
|
|
||||||
"tpot_p99_ms": 15.53437136937437,
|
|
||||||
"e2e_p50_ms": 25168.70826199738,
|
|
||||||
"e2e_p95_ms": 32143.357299292988,
|
|
||||||
"e2e_p99_ms": 32148.356982687113,
|
|
||||||
"itl_p50_ms": 12.129640497732908,
|
|
||||||
"itl_p95_ms": 15.666113744373433,
|
|
||||||
"itl_p99_ms": 16.459487642714517,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 25.900737181000295,
|
|
||||||
"request_tps": 3.088715175979033,
|
|
||||||
"input_tps": 12651.377360810118,
|
|
||||||
"output_tps": 790.7110850506324,
|
|
||||||
"total_tps": 13442.08844586075,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 1070.6434219973744,
|
|
||||||
"ttft_p95_ms": 2601.758133142721,
|
|
||||||
"ttft_p99_ms": 2731.048192037851,
|
|
||||||
"tpot_p50_ms": 14.66723954899684,
|
|
||||||
"tpot_p95_ms": 20.23464126315529,
|
|
||||||
"tpot_p99_ms": 21.929446337492916,
|
|
||||||
"e2e_p50_ms": 5262.183128004835,
|
|
||||||
"e2e_p95_ms": 6524.885636250838,
|
|
||||||
"e2e_p99_ms": 6530.703649207862,
|
|
||||||
"itl_p50_ms": 12.11705449532019,
|
|
||||||
"itl_p95_ms": 15.253629755170547,
|
|
||||||
"itl_p99_ms": 16.63169343562916,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 274.1387693459983,
|
|
||||||
"request_tps": 0.2918230069787383,
|
|
||||||
"input_tps": 1195.307036584912,
|
|
||||||
"output_tps": 1195.307036584912,
|
|
||||||
"total_tps": 2390.614073169824,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 1043.4461850018124,
|
|
||||||
"ttft_p95_ms": 1874.445231048594,
|
|
||||||
"ttft_p99_ms": 1897.9588461619276,
|
|
||||||
"tpot_p50_ms": 11.862804140170747,
|
|
||||||
"tpot_p95_ms": 15.333563006644244,
|
|
||||||
"tpot_p99_ms": 15.394116480524199,
|
|
||||||
"e2e_p50_ms": 49251.109377997636,
|
|
||||||
"e2e_p95_ms": 63309.41746450044,
|
|
||||||
"e2e_p99_ms": 63314.34459721451,
|
|
||||||
"itl_p50_ms": 11.962887001573108,
|
|
||||||
"itl_p95_ms": 15.223909802443814,
|
|
||||||
"itl_p99_ms": 15.640087766805664,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 37.81364577100612,
|
|
||||||
"request_tps": 2.1156383725723837,
|
|
||||||
"input_tps": 8665.654774056484,
|
|
||||||
"output_tps": 1083.2068467570605,
|
|
||||||
"total_tps": 9748.861620813545,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 626.4680629974464,
|
|
||||||
"ttft_p95_ms": 1079.5922750941827,
|
|
||||||
"ttft_p99_ms": 1085.4407174367225,
|
|
||||||
"tpot_p50_ms": 12.883891402167945,
|
|
||||||
"tpot_p95_ms": 15.779685464180407,
|
|
||||||
"tpot_p99_ms": 16.333569430081702,
|
|
||||||
"e2e_p50_ms": 7039.921916504682,
|
|
||||||
"e2e_p95_ms": 8655.18560165001,
|
|
||||||
"e2e_p99_ms": 8659.915602506953,
|
|
||||||
"itl_p50_ms": 12.184623003122397,
|
|
||||||
"itl_p95_ms": 15.375507500721142,
|
|
||||||
"itl_p99_ms": 16.372877460671585,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 390.9239437669894,
|
|
||||||
"request_tps": 0.2046433872254294,
|
|
||||||
"input_tps": 13411.509025205742,
|
|
||||||
"output_tps": 209.5548285188397,
|
|
||||||
"total_tps": 13621.063853724581,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 34109.15530550119,
|
|
||||||
"ttft_p95_ms": 49910.73756714177,
|
|
||||||
"ttft_p99_ms": 58476.69062248939,
|
|
||||||
"tpot_p50_ms": 42.09774688367832,
|
|
||||||
"tpot_p95_ms": 69.21228409907829,
|
|
||||||
"tpot_p99_ms": 69.88106847383611,
|
|
||||||
"e2e_p50_ms": 79284.3169455009,
|
|
||||||
"e2e_p95_ms": 90811.3658934919,
|
|
||||||
"e2e_p99_ms": 100775.45445886119,
|
|
||||||
"itl_p50_ms": 12.344260001555085,
|
|
||||||
"itl_p95_ms": 13.471613003639504,
|
|
||||||
"itl_p99_ms": 1075.7671469193883,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 332.67608038699836,
|
|
||||||
"request_tps": 0.24047415704470515,
|
|
||||||
"input_tps": 15759.714356081797,
|
|
||||||
"output_tps": 30.78069210172226,
|
|
||||||
"total_tps": 15790.495048183519,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 31602.030366491817,
|
|
||||||
"ttft_p95_ms": 50072.25141515082,
|
|
||||||
"ttft_p99_ms": 58527.80881931147,
|
|
||||||
"tpot_p50_ms": 269.0473995314715,
|
|
||||||
"tpot_p95_ms": 453.42705731063467,
|
|
||||||
"tpot_p99_ms": 478.3763391067141,
|
|
||||||
"e2e_p50_ms": 67748.1715354952,
|
|
||||||
"e2e_p95_ms": 83297.61391548891,
|
|
||||||
"e2e_p99_ms": 92202.8402509741,
|
|
||||||
"itl_p50_ms": 12.720453007204924,
|
|
||||||
"itl_p95_ms": 1088.2944416043756,
|
|
||||||
"itl_p99_ms": 1121.3325642509153,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 340.1405706650112,
|
|
||||||
"request_tps": 0.23519687711345766,
|
|
||||||
"input_tps": 15413.862538507561,
|
|
||||||
"output_tps": 60.21040054104516,
|
|
||||||
"total_tps": 15474.072939048607,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 33961.24007800245,
|
|
||||||
"ttft_p95_ms": 49937.02013984801,
|
|
||||||
"ttft_p99_ms": 62166.142975594805,
|
|
||||||
"tpot_p50_ms": 139.35726555489464,
|
|
||||||
"tpot_p95_ms": 236.3478919172656,
|
|
||||||
"tpot_p99_ms": 244.71531864295898,
|
|
||||||
"e2e_p50_ms": 69747.46793750091,
|
|
||||||
"e2e_p95_ms": 78453.58004349907,
|
|
||||||
"e2e_p99_ms": 96399.40396136702,
|
|
||||||
"itl_p50_ms": 12.44720199611038,
|
|
||||||
"itl_p95_ms": 1048.4594318972086,
|
|
||||||
"itl_p99_ms": 1099.1333858296275,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 80,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 365.2319012020016,
|
|
||||||
"request_tps": 0.21903891674499099,
|
|
||||||
"input_tps": 14354.93444779973,
|
|
||||||
"output_tps": 112.14792537343538,
|
|
||||||
"total_tps": 14467.082373173163,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 33553.05798899644,
|
|
||||||
"ttft_p95_ms": 49813.80844564191,
|
|
||||||
"ttft_p99_ms": 59237.044557964655,
|
|
||||||
"tpot_p50_ms": 79.3469260440206,
|
|
||||||
"tpot_p95_ms": 126.30341421105851,
|
|
||||||
"tpot_p99_ms": 137.98316977467067,
|
|
||||||
"e2e_p50_ms": 73173.0037334928,
|
|
||||||
"e2e_p95_ms": 90507.30776954733,
|
|
||||||
"e2e_p99_ms": 102572.02681419696,
|
|
||||||
"itl_p50_ms": 12.42508299765177,
|
|
||||||
"itl_p95_ms": 974.080784598482,
|
|
||||||
"itl_p99_ms": 1092.6132275909183,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 82.18055039300816,
|
|
||||||
"request_tps": 0.06084164654639981,
|
|
||||||
"input_tps": 7974.636296129715,
|
|
||||||
"output_tps": 7.787730757939175,
|
|
||||||
"total_tps": 7982.424026887654,
|
|
||||||
"mean_input_tokens": 131072.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 15437.700795999262,
|
|
||||||
"ttft_p95_ms": 15571.00856200559,
|
|
||||||
"ttft_p99_ms": 15593.639437204693,
|
|
||||||
"tpot_p50_ms": 7.764350944867827,
|
|
||||||
"tpot_p95_ms": 7.770626307061807,
|
|
||||||
"tpot_p99_ms": 7.771325803097836,
|
|
||||||
"e2e_p50_ms": 16424.126157013234,
|
|
||||||
"e2e_p95_ms": 16554.536877613282,
|
|
||||||
"e2e_p99_ms": 16576.65890193428,
|
|
||||||
"itl_p50_ms": 8.529996004654095,
|
|
||||||
"itl_p95_ms": 8.682068198686466,
|
|
||||||
"itl_p99_ms": 9.645803358289447,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 76.87229951999325,
|
|
||||||
"request_tps": 0.06504293524743045,
|
|
||||||
"input_tps": 4262.653804375602,
|
|
||||||
"output_tps": 66.60396569336878,
|
|
||||||
"total_tps": 4329.257770068971,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 6793.178814987186,
|
|
||||||
"ttft_p95_ms": 6830.732189002447,
|
|
||||||
"ttft_p99_ms": 6836.927511403337,
|
|
||||||
"tpot_p50_ms": 8.389067413488618,
|
|
||||||
"tpot_p95_ms": 8.404996331775118,
|
|
||||||
"tpot_p99_ms": 8.405754934785953,
|
|
||||||
"e2e_p50_ms": 15372.661996996612,
|
|
||||||
"e2e_p95_ms": 15412.706287001492,
|
|
||||||
"e2e_p99_ms": 15418.935102202231,
|
|
||||||
"itl_p50_ms": 8.452879003016278,
|
|
||||||
"itl_p95_ms": 8.561783994082361,
|
|
||||||
"itl_p99_ms": 8.750601002248002,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 38.88842260101228,
|
|
||||||
"request_tps": 0.12857297019472957,
|
|
||||||
"input_tps": 8426.158174681797,
|
|
||||||
"output_tps": 16.457340184925386,
|
|
||||||
"total_tps": 8442.615514866722,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 6736.3390589889605,
|
|
||||||
"ttft_p95_ms": 6827.516746404581,
|
|
||||||
"ttft_p99_ms": 6834.052311684936,
|
|
||||||
"tpot_p50_ms": 7.981791047220794,
|
|
||||||
"tpot_p95_ms": 8.017796259758422,
|
|
||||||
"tpot_p99_ms": 8.0200000455801,
|
|
||||||
"e2e_p50_ms": 7747.252895991551,
|
|
||||||
"e2e_p95_ms": 7842.1887119999155,
|
|
||||||
"e2e_p99_ms": 7847.936675201636,
|
|
||||||
"itl_p50_ms": 8.454770999378525,
|
|
||||||
"itl_p95_ms": 8.66287379467394,
|
|
||||||
"itl_p99_ms": 10.578372052987115,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 44.46872818299744,
|
|
||||||
"request_tps": 0.11243856535370274,
|
|
||||||
"input_tps": 7368.773819020263,
|
|
||||||
"output_tps": 28.7842727305479,
|
|
||||||
"total_tps": 7397.5580917508105,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 6767.2624370025005,
|
|
||||||
"ttft_p95_ms": 6892.302849600674,
|
|
||||||
"ttft_p99_ms": 6911.403660320793,
|
|
||||||
"tpot_p50_ms": 8.220937709819397,
|
|
||||||
"tpot_p95_ms": 8.2882254510004,
|
|
||||||
"tpot_p99_ms": 8.295880123954632,
|
|
||||||
"e2e_p50_ms": 8871.000218990957,
|
|
||||||
"e2e_p95_ms": 9004.320606408874,
|
|
||||||
"e2e_p99_ms": 9026.557145289844,
|
|
||||||
"itl_p50_ms": 8.483409998007119,
|
|
||||||
"itl_p95_ms": 8.675873003085144,
|
|
||||||
"itl_p99_ms": 9.017845205380567,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 5,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 55.36560429999372,
|
|
||||||
"request_tps": 0.09030877677967596,
|
|
||||||
"input_tps": 5918.4759950328435,
|
|
||||||
"output_tps": 46.23809371119409,
|
|
||||||
"total_tps": 5964.714088744037,
|
|
||||||
"mean_input_tokens": 65536.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 6785.467125999276,
|
|
||||||
"ttft_p95_ms": 6867.230469404603,
|
|
||||||
"ttft_p99_ms": 6880.844480285305,
|
|
||||||
"tpot_p50_ms": 8.349586817987925,
|
|
||||||
"tpot_p95_ms": 8.493367793736068,
|
|
||||||
"tpot_p99_ms": 8.520835669118448,
|
|
||||||
"e2e_p50_ms": 11042.787049998879,
|
|
||||||
"e2e_p95_ms": 11206.68308060267,
|
|
||||||
"e2e_p99_ms": 11234.859840924619,
|
|
||||||
"itl_p50_ms": 8.474283000396099,
|
|
||||||
"itl_p95_ms": 8.707773692003684,
|
|
||||||
"itl_p99_ms": 9.049925783474462,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 85.3312213080062,
|
|
||||||
"request_tps": 1.875046407955115,
|
|
||||||
"input_tps": 1920.0475217460378,
|
|
||||||
"output_tps": 1920.0475217460378,
|
|
||||||
"total_tps": 3840.0950434920755,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 806.487787005608,
|
|
||||||
"ttft_p95_ms": 900.6144404585939,
|
|
||||||
"ttft_p99_ms": 904.4747244215978,
|
|
||||||
"tpot_p50_ms": 15.241720033234294,
|
|
||||||
"tpot_p95_ms": 17.703688128979007,
|
|
||||||
"tpot_p99_ms": 17.972658989592805,
|
|
||||||
"e2e_p50_ms": 16181.125543495,
|
|
||||||
"e2e_p95_ms": 18731.30079265975,
|
|
||||||
"e2e_p99_ms": 18746.509167428594,
|
|
||||||
"itl_p50_ms": 15.200200999970548,
|
|
||||||
"itl_p95_ms": 17.96690479677636,
|
|
||||||
"itl_p99_ms": 18.968004560447298,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 95.59839020299842,
|
|
||||||
"request_tps": 1.6736683500658116,
|
|
||||||
"input_tps": 1713.836390467391,
|
|
||||||
"output_tps": 214.22954880842389,
|
|
||||||
"total_tps": 1928.0659392758148,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 128.0,
|
|
||||||
"ttft_p50_ms": 4119.889060501009,
|
|
||||||
"ttft_p95_ms": 26025.381604798167,
|
|
||||||
"ttft_p99_ms": 26478.019099681696,
|
|
||||||
"tpot_p50_ms": 21.413118271642976,
|
|
||||||
"tpot_p95_ms": 376.97693987998275,
|
|
||||||
"tpot_p99_ms": 387.07957727625,
|
|
||||||
"e2e_p50_ms": 7732.499119003478,
|
|
||||||
"e2e_p95_ms": 51129.2892126934,
|
|
||||||
"e2e_p99_ms": 51135.8539534836,
|
|
||||||
"itl_p50_ms": 16.923131996009033,
|
|
||||||
"itl_p95_ms": 18.388393107306907,
|
|
||||||
"itl_p99_ms": 244.194259904325,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 162.71412774299097,
|
|
||||||
"request_tps": 0.9833196552712499,
|
|
||||||
"input_tps": 1006.9193269977599,
|
|
||||||
"output_tps": 2013.8386539955197,
|
|
||||||
"total_tps": 3020.75798099328,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 871.0656835028203,
|
|
||||||
"ttft_p95_ms": 1184.4237971468829,
|
|
||||||
"ttft_p99_ms": 1192.6747960213106,
|
|
||||||
"tpot_p50_ms": 15.119994677330888,
|
|
||||||
"tpot_p95_ms": 17.311258725843935,
|
|
||||||
"tpot_p99_ms": 17.470737866078203,
|
|
||||||
"e2e_p50_ms": 31956.353274501453,
|
|
||||||
"e2e_p95_ms": 36126.69280984701,
|
|
||||||
"e2e_p99_ms": 36139.65432776953,
|
|
||||||
"itl_p50_ms": 14.982249005697668,
|
|
||||||
"itl_p95_ms": 17.40834979864303,
|
|
||||||
"itl_p99_ms": 18.12914603331592,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 318.9912624319986,
|
|
||||||
"request_tps": 0.50158113667489,
|
|
||||||
"input_tps": 513.6190839550874,
|
|
||||||
"output_tps": 2054.4763358203495,
|
|
||||||
"total_tps": 2568.0954197754368,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 4096.0,
|
|
||||||
"ttft_p50_ms": 867.0998999950825,
|
|
||||||
"ttft_p95_ms": 1568.6099555969122,
|
|
||||||
"ttft_p99_ms": 1761.4919271026156,
|
|
||||||
"tpot_p50_ms": 14.940362645543985,
|
|
||||||
"tpot_p95_ms": 17.25645533934084,
|
|
||||||
"tpot_p99_ms": 17.34749941946349,
|
|
||||||
"e2e_p50_ms": 62241.984514999785,
|
|
||||||
"e2e_p95_ms": 71418.52431923835,
|
|
||||||
"e2e_p99_ms": 71433.78462005581,
|
|
||||||
"itl_p50_ms": 14.883934498357121,
|
|
||||||
"itl_p95_ms": 17.468459051451646,
|
|
||||||
"itl_p99_ms": 18.707020131550955,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 44.825771888004965,
|
|
||||||
"request_tps": 3.569375233509694,
|
|
||||||
"input_tps": 3655.040239113927,
|
|
||||||
"output_tps": 1827.5201195569634,
|
|
||||||
"total_tps": 5482.56035867089,
|
|
||||||
"mean_input_tokens": 1024.0,
|
|
||||||
"mean_output_tokens": 512.0,
|
|
||||||
"ttft_p50_ms": 912.5870395073434,
|
|
||||||
"ttft_p95_ms": 1430.6775802040647,
|
|
||||||
"ttft_p99_ms": 1454.0497396518183,
|
|
||||||
"tpot_p50_ms": 15.280834627210176,
|
|
||||||
"tpot_p95_ms": 18.225302902624872,
|
|
||||||
"tpot_p99_ms": 18.700063700290844,
|
|
||||||
"e2e_p50_ms": 8596.120701506152,
|
|
||||||
"e2e_p95_ms": 10176.577183793415,
|
|
||||||
"e2e_p99_ms": 10183.192389139585,
|
|
||||||
"itl_p50_ms": 14.963374997023493,
|
|
||||||
"itl_p95_ms": 17.358059901744127,
|
|
||||||
"itl_p99_ms": 18.431273359310598,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 165.0628843260056,
|
|
||||||
"request_tps": 0.969327542368603,
|
|
||||||
"input_tps": 15881.462454167191,
|
|
||||||
"output_tps": 992.5914033854494,
|
|
||||||
"total_tps": 16874.05385755264,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 4267.802063499403,
|
|
||||||
"ttft_p95_ms": 19862.928376002674,
|
|
||||||
"ttft_p99_ms": 21657.044113324228,
|
|
||||||
"tpot_p50_ms": 25.376833950635163,
|
|
||||||
"tpot_p95_ms": 37.8741763598776,
|
|
||||||
"tpot_p99_ms": 40.72076095410008,
|
|
||||||
"e2e_p50_ms": 35063.07619450672,
|
|
||||||
"e2e_p95_ms": 51894.37677649694,
|
|
||||||
"e2e_p99_ms": 53688.66714284828,
|
|
||||||
"itl_p50_ms": 17.057024502719287,
|
|
||||||
"itl_p95_ms": 18.18839350307826,
|
|
||||||
"itl_p99_ms": 674.1142390163444,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 203.0028582619998,
|
|
||||||
"request_tps": 0.7881662424353681,
|
|
||||||
"input_tps": 12913.315716061072,
|
|
||||||
"output_tps": 1614.164464507634,
|
|
||||||
"total_tps": 14527.480180568706,
|
|
||||||
"mean_input_tokens": 16384.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 2550.8961580053438,
|
|
||||||
"ttft_p95_ms": 15365.738819241233,
|
|
||||||
"ttft_p99_ms": 18318.440213190624,
|
|
||||||
"tpot_p50_ms": 16.89672226160163,
|
|
||||||
"tpot_p95_ms": 21.461009534389174,
|
|
||||||
"tpot_p99_ms": 22.790276562500463,
|
|
||||||
"e2e_p50_ms": 40052.36150848941,
|
|
||||||
"e2e_p95_ms": 48792.51678490764,
|
|
||||||
"e2e_p99_ms": 48817.223941341945,
|
|
||||||
"itl_p50_ms": 14.973600002122112,
|
|
||||||
"itl_p95_ms": 17.530049994093133,
|
|
||||||
"itl_p99_ms": 18.100957148271835,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 86.79001112899277,
|
|
||||||
"request_tps": 1.843530124246648,
|
|
||||||
"input_tps": 7551.09938891427,
|
|
||||||
"output_tps": 1887.7748472285675,
|
|
||||||
"total_tps": 9438.874236142838,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 1024.0,
|
|
||||||
"ttft_p50_ms": 1054.5200394990388,
|
|
||||||
"ttft_p95_ms": 1617.5321672504651,
|
|
||||||
"ttft_p99_ms": 1622.538609701296,
|
|
||||||
"tpot_p50_ms": 15.406173739000165,
|
|
||||||
"tpot_p95_ms": 17.669181664998092,
|
|
||||||
"tpot_p99_ms": 18.132834118727168,
|
|
||||||
"e2e_p50_ms": 16632.105807497283,
|
|
||||||
"e2e_p95_ms": 18881.63431045614,
|
|
||||||
"e2e_p99_ms": 18892.737503780663,
|
|
||||||
"itl_p50_ms": 15.119137002329808,
|
|
||||||
"itl_p95_ms": 17.811838999477914,
|
|
||||||
"itl_p99_ms": 18.673117576690856,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 171.35218937399623,
|
|
||||||
"request_tps": 0.9337493765590661,
|
|
||||||
"input_tps": 3824.6374463859347,
|
|
||||||
"output_tps": 1912.3187231929674,
|
|
||||||
"total_tps": 5736.956169578902,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 2048.0,
|
|
||||||
"ttft_p50_ms": 897.8936719940975,
|
|
||||||
"ttft_p95_ms": 1402.46574630728,
|
|
||||||
"ttft_p99_ms": 1414.9006537273815,
|
|
||||||
"tpot_p50_ms": 17.125081185391775,
|
|
||||||
"tpot_p95_ms": 17.535651223030666,
|
|
||||||
"tpot_p99_ms": 17.64257730764133,
|
|
||||||
"e2e_p50_ms": 35692.707454494666,
|
|
||||||
"e2e_p95_ms": 36474.77745984506,
|
|
||||||
"e2e_p99_ms": 36482.97110349682,
|
|
||||||
"itl_p50_ms": 16.84335399477277,
|
|
||||||
"itl_p95_ms": 17.664605999016203,
|
|
||||||
"itl_p99_ms": 18.40053380583413,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
{
|
|
||||||
"status": "COMPLETED",
|
|
||||||
"completed": 160,
|
|
||||||
"failed": 0,
|
|
||||||
"duration_s": 39.068993989989394,
|
|
||||||
"request_tps": 4.0953191689808195,
|
|
||||||
"input_tps": 16774.427316145437,
|
|
||||||
"output_tps": 1048.4017072590898,
|
|
||||||
"total_tps": 17822.829023404527,
|
|
||||||
"mean_input_tokens": 4096.0,
|
|
||||||
"mean_output_tokens": 256.0,
|
|
||||||
"ttft_p50_ms": 2343.7372504995437,
|
|
||||||
"ttft_p95_ms": 5162.197779646522,
|
|
||||||
"ttft_p99_ms": 5491.549329505941,
|
|
||||||
"tpot_p50_ms": 19.213523033335253,
|
|
||||||
"tpot_p95_ms": 32.66720932803069,
|
|
||||||
"tpot_p99_ms": 34.66762047272469,
|
|
||||||
"e2e_p50_ms": 8660.10677999293,
|
|
||||||
"e2e_p95_ms": 9275.378481158987,
|
|
||||||
"e2e_p99_ms": 9288.702330155647,
|
|
||||||
"itl_p50_ms": 14.975201003835537,
|
|
||||||
"itl_p95_ms": 17.6850339921657,
|
|
||||||
"itl_p99_ms": 28.16523999499614,
|
|
||||||
"validation_errors": []
|
|
||||||
}
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user