Sync the glm52 add16 fixes (98cdb67) into the DSV4 experiment so the
adaptive concurrency search can actually run end-to-end.
sglang client (the main blocker):
- Built local/vllm-ascend:0.23-a3-dsv4-sglang image: sglang 0.5.2 (not
0.5.16 -- 0.5.16 deprecates bench_serving and the glm52 parser fix
targets the 0.5.2 output format) + minimal deps (ipython/traitlets/
stack_data/executing/asttokens/pure_eval/prompt_toolkit/wcwidth) via
--no-deps, so the vllm env is untouched.
- Verified: python -m sglang.bench_serving --help works in the image.
- config.env DOCKER_IMAGE -> local/vllm-ascend:0.23-a3-dsv4-sglang.
config.env (sync glm52 98cdb67):
- CONTAINER_NAME: drop ${...:-} override -> fixed value (avoids the
double-suffix bug where CONTAINER_NAME already carries _tpX_dpY).
- CONTAINER_PYTHON: /usr/local/bin/python (does not exist) ->
/usr/local/python3.12.13/bin/python3 (matches glm52 fix).
run_adaptive_concurrency_add16.sh (sync glm52 98cdb67):
- --model $SERVED_MODEL_NAME -> --tokenizer $MODEL_PATH (bench_serving
0.5.2 wants the tokenizer path).
- docker exec env: add TORCH_DEVICE_BACKEND_AUTOLOAD=0 so the client
does not try to autoload torch_npu.
- export ENGINE_TP/ENGINE_DP in engine_start_server + export line;
container_name uses ${ENGINE_TP:-${tp}} (the bench runs in a subshell
where tp/dp are not in scope).
start_vllm_docker.sh (sync glm52 98cdb67):
- Health timeout configurable via HEALTH_MAX_RETRIES /
HEALTH_RETRY_INTERVAL_S (default 480x5s=40min; TP=16 compiles 16
graphs ~60min, old hardcoded 360x10s was too rigid).
- Container name: drop the double-suffix (CONTAINER_NAME no longer
re-overridden before appending _tpX_dpY).
- Mount /mnt (bench client reads dataset from there).
d6e00d61dc