qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(p800): auto-load timing module in every TP worker process

Browse Source 
sitecustomize.py is imported automatically by Python at startup
in every process, including the SGLang scheduler/TP worker
processes spawned for tensor parallelism. Adding the p800_timing
import here ensures the monkey-patches are applied to all 8 TP
workers, not just the launch_server parent process (which never
runs the model forward and would record empty timing data).

The timing module self-disables unless SGLANG_TIMING_ENABLED=1,
so this import is a no-op when timing is not in use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
yy 2026-07-21 02:02:06 +00:00
parent f81c647d16
commit bf03201168
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
platforms/patches/kunlun_p800/sitecustomize_xpu.py
View File

@ -5,3 +5,6 @@ import torch
# them on P800 via its own FP8 kernels. # them on P800 via its own FP8 kernels.
if not hasattr(torch, "float8_e8m0fnu"): if not hasattr(torch, "float8_e8m0fnu"):
torch.float8_e8m0fnu = torch.uint8 torch.float8_e8m0fnu = torch.uint8
# P800 Timing module
import p800_timing