qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(p800): enable operator-level timing in profiling experiment

Browse Source 
Wire the timing module into the dsv4_p800_sglang_profile
experiment's server startup so benchmarks produce per-operator
Prefill/Decode breakdowns alongside the existing aggregate metrics.

Changes to start_sglang_docker.sh:
- Mount p800_timing.py into the container site-packages (read-only)
  via PATCH_ROOT so the path resolves correctly on any host.
- Set SGLANG_TIMING_ENABLED=1 in the container env to activate
  the monkey-patches loaded by sitecustomize.
- Prepend import p800_timing to the launch_server python -c
  entrypoint so the polling thread starts before model loading.

Combined with the previous two commits, a single run of
run_profile.sh now emits /tmp/p800_timing_results_{pid}.json
per TP worker with layer/attention/moe/hc/rmsnorm timing split
by prefill and decode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
yy 2026-07-21 02:02:50 +00:00
parent bf03201168
commit 41a3ff0a6b
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
experiments/p800/dsv4_p800_sglang_profile/start_sglang_docker.sh
View File

@ -81,6 +81,7 @@ env_args=(
-e BKCL_RDMA_VERBS=1 -e BKCL_RDMA_VERBS=1
-e XSGL_INT8_LM_HEAD=1 -e XSGL_INT8_LM_HEAD=1
-e SGLANG_P800_ALL_GATHER_FALLBACK=0 -e SGLANG_P800_ALL_GATHER_FALLBACK=0
-e SGLANG_TIMING_ENABLED=1
-e XPU_ENABLE_PROFILER_TRACING=1 -e XPU_ENABLE_PROFILER_TRACING=1
) )
@ -99,7 +100,7 @@ set -Eeuo pipefail
cd /workspace cd /workspace
find /root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true find /root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
/root/miniconda/envs/python310_torch25_cuda/bin/pip install --upgrade safetensors -q 2>/dev/null /root/miniconda/envs/python310_torch25_cuda/bin/pip install --upgrade safetensors -q 2>/dev/null
/root/miniconda/envs/python310_torch25_cuda/bin/python -c "import torch; torch.float8_e8m0fnu = torch.uint8; import runpy, sys; sys.argv[0] = 'sglang.launch_server'; runpy.run_module('sglang.launch_server', run_name='__main__')" LAUNCH_ARGS_PLACEHOLDER /root/miniconda/envs/python310_torch25_cuda/bin/python -c "import p800_timing; import torch; torch.float8_e8m0fnu = torch.uint8; import runpy, sys; sys.argv[0] = 'sglang.launch_server'; runpy.run_module('sglang.launch_server', run_name='__main__')" LAUNCH_ARGS_PLACEHOLDER
ENTRYEOF ENTRYEOF
# Insert the launch args # Insert the launch args
@ -116,6 +117,7 @@ patch_mounts=(
-v "${PATCH_ROOT}/config_backup_small_w8a8_int8.json:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/configs/config_backup_small.json:ro" -v "${PATCH_ROOT}/config_backup_small_w8a8_int8.json:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/configs/config_backup_small.json:ro"
-v "${PATCH_ROOT}/parallel_state.py:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py:ro" -v "${PATCH_ROOT}/parallel_state.py:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/distributed/parallel_state.py:ro"
-v "${PATCH_ROOT}/vocab_parallel_embedding.py:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/layers/vocab_parallel_embedding.py:ro" -v "${PATCH_ROOT}/vocab_parallel_embedding.py:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/sglang/srt/layers/vocab_parallel_embedding.py:ro"
-v "${PATCH_ROOT}/p800_timing.py:/root/miniconda/envs/python310_torch25_cuda/lib/python3.10/site-packages/p800_timing.py:ro"
-v "${ENTRYPOINT_FILE}:/workspace/entrypoint.sh:ro" -v "${ENTRYPOINT_FILE}:/workspace/entrypoint.sh:ro"
) )
BENCH_SERVING_PATCH="${SCRIPT_DIR}/patches/bench_serving.py" BENCH_SERVING_PATCH="${SCRIPT_DIR}/patches/bench_serving.py"