qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Disable chunked prefill and speculative decoding in P800 long-context matrix

Browse Source
This commit is contained in:
Quantong Qiu 2026-07-09 04:47:59 +00:00
parent 90eecc4ede
commit 59f480b1bc
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
experiments/dsv4_p800_long_context_matrix/start_sglang.sh
View File

@ -92,7 +92,8 @@ env_args=(
# Launch args. Keep the proven P800 INT8 command from dsv4_p800_max_context_length, # Launch args. Keep the proven P800 INT8 command from dsv4_p800_max_context_length,
# but make --context-length and --max-running-requests dynamic per group. # but make --context-length and --max-running-requests dynamic per group.
launch_args="--host 0.0.0.0 --port ${PORT} --model-path /models --attention-backend nsa --nsa-prefill klxdsa --nsa-decode klxdsa --trust-remote-code --disable-custom-all-reduce --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.8 --max-prefill-tokens ${MAX_CONTEXT_LEN} --max-running-requests ${MAX_RUNNING} --tensor-parallel-size 8 --ep-size 8 --disable-shared-experts-fusion --quantization w8a8_int8 --kv-cache-dtype float16 --disable-piecewise-cuda-graph --cuda-graph-max-bs 32 --watchdog-timeout 3000000 --tool-call-parser deepseekv4 --reasoning-parser deepseek-v4 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --constrained-json-disable-any-whitespace --enable-metrics --enable-request-time-stats-logging --context-length ${MAX_CONTEXT_LEN}" # Note: disable chunked prefill and speculative decoding for the long-context matrix.
launch_args="--host 0.0.0.0 --port ${PORT} --model-path /models --attention-backend nsa --nsa-prefill klxdsa --nsa-decode klxdsa --trust-remote-code --disable-custom-all-reduce --page-size 64 --mem-fraction-static 0.8 --max-prefill-tokens ${MAX_CONTEXT_LEN} --max-running-requests ${MAX_RUNNING} --tensor-parallel-size 8 --ep-size 8 --disable-shared-experts-fusion --quantization w8a8_int8 --kv-cache-dtype float16 --disable-piecewise-cuda-graph --cuda-graph-max-bs 32 --watchdog-timeout 3000000 --tool-call-parser deepseekv4 --reasoning-parser deepseek-v4 --constrained-json-disable-any-whitespace --enable-metrics --enable-request-time-stats-logging --context-length ${MAX_CONTEXT_LEN}"
# Base64-encode the bootstrap command to avoid host-shell quoting issues. # Base64-encode the bootstrap command to avoid host-shell quoting issues.
server_cmd=$(cat <<EOF server_cmd=$(cat <<EOF