qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
sskj/platforms
History
yy f81c647d16 feat(p800): add manual timing module to bypass broken PyTorch Profiler 
PyTorch Profiler crashes on P800 XPU due to a CUPTI bug in
cuptiActivityDisable() that SIGKILLs the server process. This
module provides an alternative by monkey-patching SGLang's
DeepSeek-V4 decoder layer and its sub-operators to record
per-op wall-clock timing.

What it does:
- Patches DeepseekV4DecoderLayer.forward and sub-operators
  (hc_pre/hc_post, RMSNorm, MQALayer attention, QKV projection,
  DeepseekV2MoE and its gate/shared/routed experts) with timed
  wrappers that call torch.cuda.synchronize() before and after.
- Distinguishes prefill vs decode via forward_batch flags
  (is_prefill_only / is_extend_in_batch / extend_num_tokens)
  and propagates the phase to sub-operators via threading.local.
- Writes per-PID JSON summaries (count/avg/p50/p95/p99) every
  5000 records plus on atexit/SIGTERM, so data survives when
  the server is stopped.

Usage: set SGLANG_TIMING_ENABLED=1 in the container environment.
Output: /tmp/p800_timing_results_{pid}.json

Known limitations:
- torch.cuda.synchronize() adds overhead, inflating small-op times.
- moe_routed_ms only covers forward_normal (65% of calls); the
  dual_stream path taken during CUDA graph capture is not patched.
- All-reduce patch fails (Communicator import path mismatch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 02:00:57 +00:00
..
patches/kunlun_p800
feat(p800): add manual timing module to bypass broken PyTorch Profiler
2026-07-21 02:00:57 +00:00
kunlun_p800.env
refactor(paths): derive repository files from root
2026-07-20 17:33:51 +08:00
nvidia_h20.env
refactor(paths): derive repository files from root
2026-07-20 17:33:51 +08:00
nvidia_h200.env
docs(h200): H200 quick-start guide, porting workflow, and example experiment
2026-07-08 05:55:11 +00:00
nvidia_rtx6000d.env
Add dsv4_pro6000_sglang_tp_dp_matrix experiment for RTX 6000D
2026-07-13 02:18:34 +00:00
README.md
rewrite README, add new platform onboarding guide, fix broken scripts/common paths
2026-07-17 06:18:05 +00:00

README.md

Platform Configurations

Each .env file in this directory describes one accelerator platform. They are meant to be sourced by benchmark scripts through scripts/common/platform.sh, not executed directly.

Usage

# Default platform for the current machine
bash experiments/dsv4_p800_sglang/run_bench.sh

# Explicitly select a platform
PLATFORM=kunlun_p800 bash experiments/dsv4_p800_sglang/run_bench.sh

Current platforms

File Chip/Accelerator Engine Notes
kunlun_p800.env Kunlun P800 XPU sglang-xpu Docker-based SGLang serving image
nvidia_h200.env NVIDIA H200 vllm-dspark Native host virtual environments
nvidia_h20.env NVIDIA H20 vllm / sglang Docker-based（vllm-openai / sglang 官方镜像）
nvidia_rtx6000d.env NVIDIA RTX 6000D vllm / sglang Docker-based，SM120 部署见 envs/SM120_DSV4_DEPLOYMENT_GUIDE.md

What belongs here

  • Chip/accelerator identity (CHIP, ACCELERATOR, HARDWARE, ENGINE).
  • Device selection environment variables.
  • Platform-wide paths that rarely change (model root, default port).
  • Container image / interpreter paths for Docker-based platforms.
  • Native interpreter / venv paths for host-based platforms.

What does NOT belong here

  • Specific model names or experiment scenarios — those go in experiments/<name>/config.env.
  • Engine-specific launch flags — those go in the experiment's start_server.sh or run_bench.sh.

Adding a new platform

  1. Create platforms/<chip>.env with at least CHIP, ACCELERATOR, HARDWARE, ENGINE, DEFAULT_PORT, MODEL_ROOT.
  2. If the platform runs inside Docker, set DOCKER_IMAGE, CONTAINER_NAME, CONTAINER_PYTHON, and PATCH_ROOT (see kunlun_p800.env).
  3. If the platform runs natively on the host, set the relevant venv paths (see nvidia_h200.env).
  4. Add a row to the table above and write a quick-start experiment under experiments/<name>/.