0fdd3749c7
- New experiments/p800/dsv4_p800_sglang_tp_dp_matrix: TP8/DP1, TP4/DP2, TP2/DP4 matrix with smoke results; TP2/DP4 documents the weight-loading OOM root cause (274 GiB INT8 weights sharded only across TP group). - Launch args drop --ep-size/--chunked-prefill-size/--max-prefill-tokens/ --max-running-requests; experts fall back to TP sharding. - Move dsv4_p800_256k_4k_probe under experiments/p800/. - scripts/common: jq-free parsing fixes in adaptive_bench_lib.sh and parse_backend.py. - .gitignore: cover raw_outputs under nested platform experiment layout.
20 lines
689 B
Bash
Executable File
20 lines
689 B
Bash
Executable File
#!/usr/bin/env bash
|
|
set -Eeuo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
EXPERIMENT_NAME="$(basename "$SCRIPT_DIR")"
|
|
|
|
# shellcheck source=/dev/null
|
|
source "${SCRIPT_DIR}/../../../scripts/common/server_docker.sh"
|
|
# shellcheck source=/dev/null
|
|
source "${SCRIPT_DIR}/config.env"
|
|
|
|
log_init "${RESULT_ROOT:-/tmp/${EXPERIMENT_NAME}}/logs/start_server.log"
|
|
log "starting server for ${EXPERIMENT_NAME}"
|
|
log "model: ${MODEL_PATH}"
|
|
log "port: ${PORT}"
|
|
log "server mode: ${SERVER_MODE}"
|
|
log "extra launch args: ${SGLANG_EXTRA_LAUNCH_ARGS:-<none>}"
|
|
|
|
docker_server_start "${MODEL_PATH}" "${PORT}" "${LOG_DIR}/server.outer.log" "${SERVER_MODE}" "${SGLANG_EXTRA_LAUNCH_ARGS:-}"
|