MiniMax-H3 / SGLang automated profile

The suite is scoped to one 8-GPU RTX 6000D host and never kills unrelated GPU processes. It aborts if a GPU process or required port already exists. Every phase has a DONE marker; reuse the same RUN_ID to resume after interruption.

Smoke validation:

MODE=smoke RUN_ID=h3-profile-smoke bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

Lightweight single-sample stage profile (recommended first pass):

MODE=light LIGHT_SCENARIO=RVA LIGHT_RESIDENT_INSTANCES=4 LIGHT_SAMPLES=5 \
  RUN_ID=h3-light-rva-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

To distribute five total samples across all four TP2 services (2/1/1/1) and start the four clients concurrently, add LIGHT_ACTIVE_INSTANCES=4.

This produces five measured outputs by default (LIGHT_SAMPLES=5). It starts four TP2 model instances but sends the requests only to instance 0. It enables synchronized SGLang stage timing and system telemetry, but does not enable Torch/Nsight by default. For a second, targeted kernel pass:

MODE=light LIGHT_SCENARIO=RVA LIGHT_RESIDENT_INSTANCES=4 LIGHT_TORCH_PROFILE=1 \
  RUN_ID=h3-light-rva-torch-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

Use LIGHT_SCENARIO=F3 for the first+last-keyframe FL2VA path. Use LIGHT_RESIDENT_INSTANCES=1 to remove the three idle resident models and obtain the clean TP2 baseline.

When GPUs 0-3 are occupied but GPUs 4-7 are free, use two resident TP2 instances on the high GPU half (requests still go only to the first instance):

MODE=light LIGHT_SCENARIO=RVA LIGHT_SAMPLES=5 \
  LIGHT_RESIDENT_INSTANCES=2 LIGHT_GPU_OFFSET=4 \
  RUN_ID=h3-light-rva-5-gpu4-7-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

Full run in tmux:

tmux new -s h3-profile
MODE=full REPEATS=1 RUN_ID=h3-profile-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

Fast two-batch input matrix (one sample per scenario):

MODE=quick_matrix RUN_ID=h3-quick-input-matrix-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

The FL2VA batch maps F0/F1/F2/F3 to replicas 0/1/2/3. The Ref2VA batch maps R1/R5/RV5/RVA_EMBEDDED to replicas 0/1/2/3. All four requests in a batch start concurrently. R5 uses five distinct frames from REFERENCE_IMAGES_DIR when present.

Targeted evidence pass after the quick matrix:

MODE=targeted_profile RUN_ID=h3-targeted-profile-$(date +%Y%m%d-%H%M%S) \
  bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

This captures one all-stage Torch trace and one bounded Nsight Systems trace for F3 and RVA_EMBEDDED. It is deliberately separate from clean stage timing because profilers perturb latency.

Result layout:

metadata/: exact hardware, topology, software and copied scripts
phases/*/client_*/: request JSONL and request-level perf dumps
phases/*/server_*/: server logs, outputs, Torch traces and Nsight reports
phases/*/system/: GPU/DCGM, CPU, disk and VM telemetry
summary.csv, summary.json: latency/QPS/serving-gap aggregation
bottleneck_comparisons.json: P2/P3/P4/P5 versus P1 contention ratios
profile_inventory.json: trace inventory and missing-artifact detection

The e2e_* phases disable synchronized stage timing and provide clean latency controls. The remaining P1-P6 phases enable synchronized stage timing for attribution. Nsight starts after a 70-second delay to omit most model-loading CUDA activity and has a bounded 30-minute capture window; override NSYS_DELAY_SECONDS/NSYS_DURATION_SECONDS when needed.

ENABLE_NCU=0 is the safe default because an unfiltered Nsight Compute pass can take many hours. After Torch/Nsight identifies the top kernel, rerun with the same RUN_ID:

ENABLE_NCU=1 NCU_KERNEL_REGEX='your_demangled_kernel_regex' \
  RUN_ID=<existing-run-id> bash /data/wxy/h3_profile/run_h3_sglang_full_profile.sh

P1-P6 isolate a single TP2 baseline, four resident models with one active, four active FL2VA, four active Ref2VA, a 2+2 production mix, and TP8. The scenario matrix includes T2VA, first/last/both keyframes, 1/5/6/9 images, 1/5/10-second videos, image+audio, video+audio, embedded video/audio, and a mixed reference workload.

4.3 KiB Raw Blame History

MiniMax-H3 / SGLang automated profile

4.3 KiB

Raw Blame History