ceb1170969
- Created adaptive_summary.md to summarize adaptive concurrency search results, including metrics such as TP, DP, and TPS. - Added run_manifest.json to document experiment configuration details, including hardware specifications and search parameters. - Introduced shapes.tsv to define input and output lengths for various configurations in the adaptive experiment.
DSV4 DSpark H20 vLLM TP×DP Matrix Benchmark
NVIDIA H20 + vLLM (Docker) + DeepSeek-V4-Flash-DSpark TP×DP matrix benchmark.
Adapted from experiments/h20/dsv4_h20_vllm_tp_dp_matrix, with the model swapped
to /data1/hf_models/DeepSeek-V4-Flash-DSpark and DSpark speculative decoding
enabled (aligned with experiments/h200/dsv4_h200_vllm_dspark_vs_default):
--spec-method dspark --spec-model <MODEL_PATH> --spec-tokens 5--tokenizer-mode deepseek_v4 --reasoning-parser deepseek_v4--no-disable-hybrid-kv-cache-manager --disable-uvicorn-access-log
All other parameters (KV cache fp8, block size 256, max-model-len 1M, max-num-seqs 128, ISL/OSL matrix, concurrency rules) are identical to the non-DSpark H20 vLLM matrix experiment.
Requirements
The serving environment must include a vLLM build with dspark speculative
decoding support (the stock vllm/vllm-openai:latest image may not). Either:
- point
DOCKER_IMAGEto an image with dspark support, or - set
USE_DOCKER=0andVENV_VLLMto a native env with dspark support.
Quick Start
# Dry run: print server args and scenario plan only
DRY_RUN=1 bash experiments/h20/dsv4_dspark_h20_vllm_tp_dp_matrix/run_bench.sh
# Full TP×DP matrix run
bash experiments/h20/dsv4_dspark_h20_vllm_tp_dp_matrix/run_bench.sh
# Adaptive concurrency search (+16 per step, aligned with
# dsv4_h20_vllm_tp_dp_matrix/run_adaptive_concurrency_add16.sh)
bash experiments/h20/dsv4_dspark_h20_vllm_tp_dp_matrix/run_adaptive_concurrency_add16.sh
Results land in experiments/h20/dsv4_dspark_h20_vllm_tp_dp_matrix/results/<RUN_ID>/.
Configuration
Edit config.env or override via environment variables:
MODEL_PATH=/data1/hf_models/DeepSeek-V4-Flash-DSpark \
VLLM_PORT=30032 \
SPEC_TOKENS=5 \
bash experiments/h20/dsv4_dspark_h20_vllm_tp_dp_matrix/run_bench.sh
Cold-start note: the DSpark draft model compiles ~140 TileLang kernels plus
CUDA graphs on first launch, which exceeds vLLM's default 600s engine-ready
timeout. config.env therefore sets VLLM_ENGINE_READY_TIMEOUT_S=3600 and
HEALTH_CHECK_RETRIES=720 (60min), and start_vllm_docker.sh persists the
TileLang/DeepGEMM JIT caches under runtime/cache/ so later TP×DP configs and
reruns start much faster.
Files
Same layout as dsv4_h20_vllm_tp_dp_matrix; see that experiment for details.
Platform
This experiment targets platforms/nvidia_h20.env.