qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(dsv4): disable MTP speculative decoding for fair H20 comparison

Browse Source 
MTP (multi-token prediction) speculative decoding was enabled, giving the
910C an unfair decode throughput advantage over the H20 baseline (which
has no MTP). Disable it so the benchmark measures pure model throughput.

- config.env: add DSV4_ENABLE_MTP=0 (default off)
- start_vllm_docker.sh: only add --speculative-config when DSV4_ENABLE_MTP=1
This commit is contained in:
shishi 2026-07-29 16:50:13 +08:00
parent 455a78161b
commit 4914ff4041
2 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
experiments/910c/dsv4_910c_vllm_tp_dp_matrix/config.env
View File

@ -130,6 +130,8 @@ DSV4_COMPILATION_CONFIG="${DSV4_COMPILATION_CONFIG:-{\"cudagraph_mode\": \"FULL_
DSV4_ADDITIONAL_CONFIG="${DSV4_ADDITIONAL_CONFIG:-{\"ascend_compilation_config\":{\"enable_npugraph_ex\":true,\"enable_static_kernel\":false},\"enable_cpu_binding\": true,\"multistream_overlap_shared_expert\":true}}"
DSV4_ENABLE_EXPERT_PARALLEL="${DSV4_ENABLE_EXPERT_PARALLEL:-1}"
DSV4_ENABLE_ASYNC_SCHEDULING="${DSV4_ENABLE_ASYNC_SCHEDULING:-1}"
# MTP speculative decoding (disabled for fair comparison with H20 which has no MTP).
DSV4_ENABLE_MTP="${DSV4_ENABLE_MTP:-0}"
# Model-loader extra config (multithread load, 128 threads) to speed up 280GiB load.
DSV4_MODEL_LOADER_EXTRA_CONFIG="${DSV4_MODEL_LOADER_EXTRA_CONFIG:-{\"enable_multithread_load\": \"true\", \"num_threads\": 128}}"

4
experiments/910c/dsv4_910c_vllm_tp_dp_matrix/start_vllm_docker.sh
View File

@ -95,7 +95,9 @@ fi
# JSON-bearing args. Quoted as single strings so vllm parses each as one argv.
SERVER_ARGS+=(--model-loader-extra-config "$DSV4_MODEL_LOADER_EXTRA_CONFIG")
SERVER_ARGS+=(--speculative-config "$DSV4_SPEC_CONFIG")
if [[ "${DSV4_ENABLE_MTP:-0}" == "1" ]]; then
SERVER_ARGS+=(--speculative-config "$DSV4_SPEC_CONFIG")
fi
SERVER_ARGS+=(--compilation-config "$DSV4_COMPILATION_CONFIG")
SERVER_ARGS+=(--additional-config "$DSV4_ADDITIONAL_CONFIG")