bf03201168
sitecustomize.py is imported automatically by Python at startup in every process, including the SGLang scheduler/TP worker processes spawned for tensor parallelism. Adding the p800_timing import here ensures the monkey-patches are applied to all 8 TP workers, not just the launch_server parent process (which never runs the model forward and would record empty timing data). The timing module self-disables unless SGLANG_TIMING_ENABLED=1, so this import is a no-op when timing is not in use. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
11 lines
385 B
Python
11 lines
385 B
Python
import torch
|
|
# DeepSeek-V4-Flash safetensors contain F8_E8M0FN dtype metadata, but
|
|
# PyTorch 2.5.1 does not define this dtype. Alias it to uint8 so that
|
|
# safetensors deserialization can load the scales; sglang reinterprets
|
|
# them on P800 via its own FP8 kernels.
|
|
if not hasattr(torch, "float8_e8m0fnu"):
|
|
torch.float8_e8m0fnu = torch.uint8
|
|
|
|
# P800 Timing module
|
|
import p800_timing
|