[2026-07-30 22:09:49] Starting worker node=10.101.0.13 rank=1 container=dsv4pro_pro6000d_2node_sglang_tp16_quick_map_worker
[2026-07-30 22:09:55] Starting head node=10.101.0.11 rank=0 container=dsv4pro_pro6000d_2node_sglang_tp16_quick_map_head
[2026-07-30 22:09:56] Waiting for SGLang health at 10.101.0.11:30002
[2026-07-30 22:10:55] Still starting: checks=12/600
[2026-07-30 22:12:00] Still starting: checks=24/600
[2026-07-30 22:13:05] Still starting: checks=36/600
[2026-07-30 22:14:09] Still starting: checks=48/600
[2026-07-30 22:15:14] Still starting: checks=60/600
[2026-07-30 22:15:20] SGLang is healthy after 61 checks
[2026-07-30 22:15:20] Verified worker NCCL transport: NET/IB with mlx5_0 and mlx5_3
[2026-07-30 22:15:21] Verified head NCCL transport: NET/IB with mlx5_0 and mlx5_3
[2026-07-30 22:15:21] Validating scenario file: /data/hzy/sskj/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/quick_map_scenarios.tsv
short_prefill_latency_1k_c1: stage=prefill_latency isl=1024 osl=1 c=1 prompts=1 warmup=1
mid_prefill_latency_32k_c1: stage=prefill_latency isl=32768 osl=1 c=1 prompts=1 warmup=0
long_prefill_latency_128k_c1: stage=prefill_latency isl=131072 osl=1 c=1 prompts=1 warmup=0
mid_prefill_throughput_32k_c16: stage=prefill_throughput isl=32768 osl=1 c=16 prompts=16 warmup=0
decode_latency_1k_to_1k_c1: stage=decode_latency isl=1024 osl=1024 c=1 prompts=1 warmup=1
decode_throughput_1k_to_1k_c16: stage=decode_throughput isl=1024 osl=1024 c=16 prompts=16 warmup=1
decode_throughput_1k_to_1k_c32: stage=decode_throughput isl=1024 osl=1024 c=32 prompts=32 warmup=1
decode_throughput_1k_to_1k_c64: stage=decode_throughput isl=1024 osl=1024 c=64 prompts=64 warmup=1
balanced_32k_to_1k_c8: stage=balanced isl=32768 osl=1024 c=8 prompts=8 warmup=0
[2026-07-30 22:15:21] START case=short_prefill_latency_1k_c1 rep=1 isl=1024 osl=1 c=1
[2026-07-30 22:15:55] DONE  case=short_prefill_latency_1k_c1 rep=1 elapsed=34s
[2026-07-30 22:16:00] START case=mid_prefill_latency_32k_c1 rep=1 isl=32768 osl=1 c=1
[2026-07-30 22:16:46] DONE  case=mid_prefill_latency_32k_c1 rep=1 elapsed=46s
[2026-07-30 22:16:51] START case=long_prefill_latency_128k_c1 rep=1 isl=131072 osl=1 c=1
[2026-07-30 22:18:13] DONE  case=long_prefill_latency_128k_c1 rep=1 elapsed=81s
[2026-07-30 22:18:18] START case=mid_prefill_throughput_32k_c16 rep=1 isl=32768 osl=1 c=16
[2026-07-30 22:21:39] DONE  case=mid_prefill_throughput_32k_c16 rep=1 elapsed=201s
[2026-07-30 22:21:44] START case=decode_latency_1k_to_1k_c1 rep=1 isl=1024 osl=1024 c=1
[2026-07-30 22:22:51] DONE  case=decode_latency_1k_to_1k_c1 rep=1 elapsed=66s
[2026-07-30 22:22:56] START case=decode_throughput_1k_to_1k_c16 rep=1 isl=1024 osl=1024 c=16
[2026-07-30 22:24:25] DONE  case=decode_throughput_1k_to_1k_c16 rep=1 elapsed=89s
[2026-07-30 22:24:30] START case=decode_throughput_1k_to_1k_c32 rep=1 isl=1024 osl=1024 c=32
[2026-07-30 22:26:15] DONE  case=decode_throughput_1k_to_1k_c32 rep=1 elapsed=105s
[2026-07-30 22:26:20] START case=decode_throughput_1k_to_1k_c64 rep=1 isl=1024 osl=1024 c=64
[2026-07-30 22:28:36] DONE  case=decode_throughput_1k_to_1k_c64 rep=1 elapsed=136s
[2026-07-30 22:28:41] START case=balanced_32k_to_1k_c8 rep=1 isl=32768 osl=1024 c=8
[2026-07-30 22:31:23] DONE  case=balanced_32k_to_1k_c8 rep=1 elapsed=161s
[2026-07-30 22:31:28] Fixed quick map complete: completed=9/9 failed=0
[2026-07-30 22:31:28] START case=decode_control_1k_to_1k_c32 rep=1 isl=1024 osl=1024 c=32
[2026-07-30 22:34:26] DONE  case=decode_control_1k_to_1k_c32 rep=1 elapsed=178s
[2026-07-30 22:34:26] START mixed background rep=1; injection delay=10s
[2026-07-30 22:34:26] START case=decode_with_128k_prefill_1k_to_1k_c32 rep=1 isl=1024 osl=1024 c=32
[2026-07-30 22:35:06] START case=long_prefill_injection_128k_to_1_c1 rep=1 isl=131072 osl=1 c=1
[2026-07-30 22:36:24] DONE  case=long_prefill_injection_128k_to_1_c1 rep=1 elapsed=78s
[2026-07-30 22:38:09] DONE  case=decode_with_128k_prefill_1k_to_1k_c32 rep=1 elapsed=223s
[2026-07-30 22:38:14] Mixed-interference A/B complete: failed_repetitions=0
[2026-07-30 22:38:20] Stopped head container=dsv4pro_pro6000d_2node_sglang_tp16_quick_map_head on 10.101.0.11
[2026-07-30 22:38:22] Stopped worker container=dsv4pro_pro6000d_2node_sglang_tp16_quick_map_worker on 10.101.0.13
summarized 12 case runs into /data/hzy/sskj/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/results/dsv4pro-phase1-full-20260730-220916
[2026-07-30 22:38:22] Quick performance map complete: status=COMPLETED
[2026-07-30 22:38:22] Results: /data/hzy/sskj/experiments/pro6000/dsv4pro_pro6000d_2node_sglang_tp16_quick_map/results/dsv4pro-phase1-full-20260730-220916