shishi
|
98cdb67b66
|
fix(910c/glm52): 修复sglang0.5.2解析兼容性+TP=16设备挂载+health超时
sglang 0.5.2 bench_serving 输出与解析器不兼容,致 add16 c=32 崩溃:
- 缺 total_throughput -> total_tps 恒0,无法检测吞吐增益
- 缺 p95_*(仅p99) -> TTFT SLO 失效
- gain previous<=0 返回 inf -> json.loads(inf) 崩溃
修复:
1. adaptive_concurrency.py: 缺失时从 ttfts/itls 数组补算 p95/p50;
total_tps 回退 input+output throughput; gain 返回 Infinity
2. parse_backend.py: 同上补算逻辑; 补 from __future__ import annotations
(py3.9 下 dict|None 语法无法 import)
3. start_vllm_docker.sh: --device davinci0~15 支持 TP=16;
health 超时可配(默认480x5s=40min,TP=16编译16 graph约60min);
补驱动挂载+/mnt; 修容器名双后缀
4. run_adaptive_concurrency_add16.sh: --tokenizer 替代 --model;
TORCH_DEVICE_BACKEND_AUTOLOAD=0; CONTAINER_PYTHON 路径; 导出 ENGINE_TP/DP
5. config.env: 固定 CONTAINER_NAME/DOCKER_IMAGE/GPU_MEM_UTIL
6. TP8_vs_TP16_report.md: TP=8 vs TP=16 手动测速对比报告
验证: TP=8 add16 c=16->c=32 不再崩溃; TP=16 编译完成变 healthy 推理正常
|
2026-07-28 16:43:05 +08:00 |
|
shishi
|
46e79d63e7
|
feat(platform): add Ascend 910C NPU platform support
- platforms/ascend_910c.env: 8-card 910C config (16 dies, 64GB HBM/die),
Ascend Docker Runtime, ASCEND_VISIBLE_DEVICES device selection
- scripts/common/platform.sh: auto-detect 910C via npu-smi + Huawei PCI IDs
- scripts/common/npu_smi_sampler.py: standalone npu-smi -> nvidia-smi CSV
sampler so parse_backend.py needs no changes
- experiments/910c/glm52_910c_vllm_tp_dp_matrix/: GLM-5.2 (w4a8c8) experiment,
model present on host, ready for smoke after image load
- experiments/910c/dsv4_910c_vllm_tp_dp_matrix/: DSV4-Flash experiment
(placeholder MODEL_PATH, weights not yet downloaded)
- envs/ASCEND_910C_ENV_SETUP.md: full onboarding guide (permissions, image
load, Ascend Docker Runtime, NPU monitor, known pitfalls)
- Both experiments: TP2/DP4 + TP4/DP2 + TP8/DP1, matrix.json capped at 128K
context per 64GB HBM/die
|
2026-07-27 22:00:05 +08:00 |
|