qqtang/sskj
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
223 Commits 4 Branches 0 Tags
Commit Graph

38 Commits

Author SHA1 Message Date
Zhiyi Hong
9acf9fdfdb feat(pro6000): 部署/测试解耦 - deploy 层支持多节点与 vLLM，新增 6 个 profile 
- sskj.deploy runtime 支持 NODE_HOSTS 多节点编排(ssh 分发/本地 rank/LOCAL_NODE_RANK)
  与 ENGINE=vllm 启动(SERVER_CMD)，容器名按 rank 自动唯一
- scripts/common/deploy_cli.sh 新增 deploy_stop/status/multinode helper 与 node-rank 透传
- src/sskj/common/env.py 修复嵌套 ${VAR:-${OTHER}/path} 展开(平衡花括号扫描)
- deploy/profiles/pro6000/ 新增 6 个 profile: tp16/tp16_eagle/glm52(多节点)、
  sglang/vllm tp_dp_matrix、qwen3(单节点)
- 6 个实验 start/stop 脚本改为 deploy 薄包装，run_bench/adaptive 的 server 启停走
  deploy_render_args/deploy_start/deploy_stop，tp16 新增 matrix.json
- 首次入库 glm52_pro6000_sglang_multinode_tp16 实验目录；ops/README.md 补 pro6000 章节
- 实测通过: 单节点 dsv4 sglang/vllm 链路 + tp16 双节点启动/bench/清理
 2026-08-03 15:17:41 +08:00
yy-fighting
3761d75b00 feat(ops): add unified bench/deploy layers and P800 profile 2026-08-02 15:36:28 +08:00
shishi
e885fd0dc2 feat(adaptive): support tiered per-ISL TTFT SLO via get_ttft_slo_ms() 
- adaptive_bench_lib.sh: add default get_ttft_slo_ms() fallback (flat TTFT_SLO_MS),
  use it instead of hardcoded TTFT_SLO_MS in SLO comparison and logs,
  add ttft_slo_tiers_desc to run_manifest.json
- glm52_910c config.env: define tiered SLO for GLM-5.2:
  ≤2048:5000ms, ≤8192:8000ms, ≤32768:12000ms, ≤131072:20000ms, >131072:30000ms
  (~70-80% of DSv4-Pro values, since GLM-5.2 has simpler architecture)
- glm52_910c adaptive_config.env: update TTFT_SLO_MS comment noting tiered override

Backward compatible: experiments without get_ttft_slo_ms() keep flat 4000ms behavior.
 2026-07-30 10:47:56 +08:00
shishi
ad4fd2b878 fix: INVALID_WORKLOAD 只跳过当前shape而不中止所有配置 
原来 ADAPTIVE_FATAL_WORKLOAD=1 会设 abort_all=1,导致所有剩余TP/DP配置
全部被跳过。改为 continue 只跳过当前 shape,让后续 shape 和配置正常
继续执行。

这修复了 TP=8/DP=2 在 isl=4096 osl=128 因 KV cache 不足导致 workload
验证失败后,TP=16/DP=1 被跳过的 bug。
 2026-07-30 10:05:28 +08:00
shishi
98cdb67b66 fix(910c/glm52): 修复sglang0.5.2解析兼容性+TP=16设备挂载+health超时 
sglang 0.5.2 bench_serving 输出与解析器不兼容,致 add16 c=32 崩溃:
- 缺 total_throughput -> total_tps 恒0,无法检测吞吐增益
- 缺 p95_*(仅p99) -> TTFT SLO 失效
- gain previous<=0 返回 inf -> json.loads(inf) 崩溃

修复:
1. adaptive_concurrency.py: 缺失时从 ttfts/itls 数组补算 p95/p50;
   total_tps 回退 input+output throughput; gain 返回 Infinity
2. parse_backend.py: 同上补算逻辑; 补 from __future__ import annotations
   (py3.9 下 dict|None 语法无法 import)
3. start_vllm_docker.sh: --device davinci0~15 支持 TP=16;
   health 超时可配(默认480x5s=40min,TP=16编译16 graph约60min);
   补驱动挂载+/mnt; 修容器名双后缀
4. run_adaptive_concurrency_add16.sh: --tokenizer 替代 --model;
   TORCH_DEVICE_BACKEND_AUTOLOAD=0; CONTAINER_PYTHON 路径; 导出 ENGINE_TP/DP
5. config.env: 固定 CONTAINER_NAME/DOCKER_IMAGE/GPU_MEM_UTIL
6. TP8_vs_TP16_report.md: TP=8 vs TP=16 手动测速对比报告

验证: TP=8 add16 c=16->c=32 不再崩溃; TP=16 编译完成变 healthy 推理正常
 2026-07-28 16:43:05 +08:00
shishi
46e79d63e7 feat(platform): add Ascend 910C NPU platform support 
- platforms/ascend_910c.env: 8-card 910C config (16 dies, 64GB HBM/die),
  Ascend Docker Runtime, ASCEND_VISIBLE_DEVICES device selection
- scripts/common/platform.sh: auto-detect 910C via npu-smi + Huawei PCI IDs
- scripts/common/npu_smi_sampler.py: standalone npu-smi -> nvidia-smi CSV
  sampler so parse_backend.py needs no changes
- experiments/910c/glm52_910c_vllm_tp_dp_matrix/: GLM-5.2 (w4a8c8) experiment,
  model present on host, ready for smoke after image load
- experiments/910c/dsv4_910c_vllm_tp_dp_matrix/: DSV4-Flash experiment
  (placeholder MODEL_PATH, weights not yet downloaded)
- envs/ASCEND_910C_ENV_SETUP.md: full onboarding guide (permissions, image
  load, Ascend Docker Runtime, NPU monitor, known pitfalls)
- Both experiments: TP2/DP4 + TP4/DP2 + TP8/DP1, matrix.json capped at 128K
  context per 64GB HBM/die
 2026-07-27 22:00:05 +08:00
yy-fighting
489c5a5e61 fix(lib.sh): avoid cd side-effect in git_commit/git_dirty and safe JSON in write_metadata_json 2026-07-22 03:44:18 +00:00
Quantong Qiu
83f5ea2197 [Fix] skip longer contexts after C1 OOM 2026-07-21 14:19:38 +08:00
Quantong Qiu
5ffbd0a21a fix(bench): use default limits and back off after OOM 2026-07-20 11:48:10 +08:00
Quantong Qiu
8beed2411f update 8k and 32k 2026-07-17 16:47:25 +08:00
yy-fighting
8652a685e6 rewrite README, add new platform onboarding guide, fix broken scripts/common paths 
- rewrite README with project purpose, standard workflow, corrected index
- add docs/NEW_PLATFORM_GUIDE.md (new GPU onboarding SOP, GLM5.2 reuse)
- fix ../../scripts/common -> ../../../scripts/common in 42 experiment scripts
- refresh stale docs (EXPERIMENT_GUIDE, H200_QUICKSTART, ADAPTIVE_CONCURRENCY_USAGE, BENCHMARK_WORKFLOW)
- remove dead code (dp_proxy.py) and .bak leftovers
- add p800 adaptive results (tp4_dp2/tp8_dp1 metrics + summary)
- gitignore envs/charts and .tmp_charts
 2026-07-17 06:18:05 +00:00
SSKJ Dev
0591c401ab update stop method 2026-07-16 13:02:29 +00:00
SSKJ Dev
c2a3024a1b update SEARCH_INITIAL_BACKOFF_CONCURRENCIES 2026-07-16 08:58:31 +00:00
yy-fighting
0fdd3749c7 Add P800 sglang TP/DP matrix experiment 
- New experiments/p800/dsv4_p800_sglang_tp_dp_matrix: TP8/DP1, TP4/DP2,
  TP2/DP4 matrix with smoke results; TP2/DP4 documents the weight-loading
  OOM root cause (274 GiB INT8 weights sharded only across TP group).
- Launch args drop --ep-size/--chunked-prefill-size/--max-prefill-tokens/
  --max-running-requests; experts fall back to TP sharding.
- Move dsv4_p800_256k_4k_probe under experiments/p800/.
- scripts/common: jq-free parsing fixes in adaptive_bench_lib.sh and
  parse_backend.py.
- .gitignore: cover raw_outputs under nested platform experiment layout.
 2026-07-16 06:49:21 +00:00
yy-fighting
8f89cf6dc2 Merge branch 'main' of https://gitee.com/yy-fighting/sskj 2026-07-16 05:18:58 +00:00
SSKJ Dev
dcb8cd2131 feat(adaptive): add TTFT SLO early-stop and fix relative paths for h20 experiments 
- Add TTFT_SLO_MS and ENABLE_TTFT_SLO_STOP config to adaptive search
- Stop searching current shape when TTFT P95 exceeds SLO (default 3000ms)
- Fix relative paths in h20 scripts: ../../scripts/common -> ../../../scripts/common
- Update compare.py default TTFT limit to 4000ms
 2026-07-16 05:15:51 +00:00
yy-fighting
18ded8e15d Merge branch 'main' of https://gitee.com/yy-fighting/sskj 
# Conflicts:
#	experiments/dsv4_p800_long_context_matrix/README.md
#	experiments/dsv4_p800_long_context_matrix/config.env
#	experiments/dsv4_p800_long_context_matrix/run_bench.sh
 2026-07-16 05:13:38 +00:00
SSKJ Dev
90fc378d2b Add H20 TP×DP matrix experiments for vLLM and SGLang 
- Add dsv4_h20_vllm_tp_dp_matrix experiment (vLLM backend)
- Add dsv4_h20_sglang_tp_dp_matrix experiment (SGLang backend)
- Add nvidia_h20 platform config with H20-specific paths
- Fix platform.sh auto-detection to distinguish H20 from H200
- Fix vLLM Docker startup: remove --entrypoint override for CDI compat
- Fix vLLM 0.25.1 CLI args: remove redundant 'serve' from SERVER_ARGS
- Download ShareGPT dataset to local datasets dir
- Rename DSL to OSL across both experiments
 2026-07-16 03:43:58 +00:00
Quantong Qiu
5405d002fc chore: remove legacy benchmark files and scripts 
Delete obsolete files that are no longer maintained:
- experiments/legacy_bench_results/ (old benchmark reports and summaries)
- scripts/benchmark_dspark_0707/ (outdated dspark benchmark scripts)
- scripts/start_dsv4_dspark_8card.sh (deprecated deployment script)
 2026-07-14 10:47:09 +00:00
Quantong Qiu
e0ae0715b1 feat(adaptive): add RESUME_RUN_ID resume support and rename DSL to OSL 
- Add RESUME_RUN_ID env var for breakpoint resume in adaptive benchmarks.
  When set, the script reuses an existing result directory and skips
  already-tested (TP, DP, ISL, OSL) shapes based on adaptive_shapes.jsonl.
- Fix RUN_ID unbound variable in resume mode.
- Fix jq query to use -s (slurp) for jsonl files.
- Add resume skip logic to DRY_RUN mode as well.
- Rename DSL -> OSL across all adaptive benchmark files for consistency:
  - scripts/common/adaptive_bench_lib.sh
  - scripts/common/adaptive_concurrency.py
  - experiments/dsv4_h200_vllm_tp_dp_matrix/adaptive_config.env
  - experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh
  - experiments/dsv4_h200_sglang_tp_dp_matrix/adaptive_config.env
  - experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
  - experiments/ADAPTIVE_CONCURRENCY_USAGE.md
 2026-07-14 10:41:50 +00:00
iiGray
60826dda09 add run_adaptive_concurrency_add16 2026-07-13 17:26:33 +08:00
Root User
cf7a12932e bench: add adaptive run heartbeat monitor 2026-07-11 13:04:35 +00:00
Root User
5e95cee1a0 bench: add adaptive concurrency search 2026-07-11 06:11:40 +00:00
yy-fighting
59dbadb110 add dp_proxy for vLLM DP, write kv_cache_analysis.md, fix DP bench port 2026-07-09 10:19:30 +00:00
yy-fighting
04a384b265 add dsv4_h200_vllm_tp_dp_matrix experiment (TP×DP matrix with Y-only mode, 1M excluded) 2026-07-09 09:55:18 +00:00
yy-fighting
4658e36b1e Fix P800 baseline server startup issues 
- Fix LOG_DIR unbound variable in long-context matrix start_server.sh.
- Add BKCL RDMA env vars to w8a8_int8_baseline mode so cross-device
  communication initializes correctly (was missing BKCL_RDMA_NICS).
 2026-07-08 15:27:31 +00:00
yy-fighting
f6a45f1ed8 Add P800 long-context matrix and tune benchmark/health settings 
- Add experiments/dsv4_p800_long_context_matrix for 64k/128k/256k context
  grid on DeepSeek-V4-Flash-INT8 with Kunlun P800.
- Change bench_client_docker.sh to use --concurrency instead of
  --max-concurrency to match the user's long-context benchmark script.
- Add SGLANG_HEALTH_CHECK_TIMEOUT override to server_docker.sh for slow
  long-context server startups.
- Tune dsv4_p800_256k_4k_probe with lower max-running-requests and
  mem-fraction-static to avoid OOM on P800.
 2026-07-08 15:02:22 +00:00
yy-fighting
539b3bc539 Merge branch 'main' of https://gitee.com/yy-fighting/sskj 2026-07-08 14:45:00 +00:00
yy-fighting
80959cf765 docs: move SLO_STANDARDS.md from scripts/ to docs/; update references 2026-07-08 11:06:49 +00:00
yy-fighting
9b485c05f7 cleanup(scripts): remove unused legacy scripts; update README and workflow docs 2026-07-08 11:04:16 +00:00
yy-fighting
a127cf7ef0 refactor(common): centralize parse/compare/warmup; add TEMPLATE and EXPERIMENT_GUIDE 2026-07-08 10:48:43 +00:00
yy-fighting
be94b8e7ef Add P800 256k/4k long-context probe and baseline w8a8_int8 server mode 
- Add experiments/dsv4_p800_256k_4k_probe for testing 256k input + 4k output
  on DeepSeek-V4-Flash-INT8 with Kunlun P800.
- Add SGLANG_EXTRA_LAUNCH_ARGS support to scripts/common/server_docker.sh
  so probe can inject --context-length 270000.
- Add w8a8_int8_baseline server mode without EAGLE speculative decoding,
  matching the long-context P800 tuning.
- Switch dsv4_p800_sglang and the new probe to use w8a8_int8_baseline by
  default; original tuned mode remains available via SERVER_MODE=w8a8_int8.
 2026-07-08 10:30:05 +00:00
Quantong Qiu
20d04a31ae refactor: move final legacy result dspark_grid into experiments/legacy_bench_results/ 
- Move bench_results/dspark_grid_20260707-132641/ to experiments/legacy_bench_results/
- Remove empty bench_results/ directory
- Update README.md, BENCHMARK_WORKFLOW.md, legacy README, and parse_results.py default path
- Update internal report/evaluation paths
 2026-07-08 06:40:39 +00:00
Quantong Qiu
07835b9fed chore: remove obsolete scripts/test_block_sparse_attn.py after refactor 2026-07-08 06:01:31 +00:00
Quantong Qiu
f0d4c4753b fix(common): robust XPU platform auto-detection 
Capture xpu-smi output before grepping to avoid SIGPIPE failures under
set -o pipefail when grep -q exits early.
 2026-07-08 05:50:42 +00:00
Quantong Qiu
e72e34c2f7 fix(p800): managed-mode server startup and smoke-test defaults 
- Remove USE_FAST_BFP16_MOE=1 which caused torch parameter assignment
  crash in fresh containers for w8a8_int8 mode.
- Mount model path at its host absolute path inside the container so the
  bench client tokenizer can load it regardless of server --model-path.
- Default WARMUP to 0 and NUM_PROMPTS to 10 for a fast smoke test.
- Default to a single scenario; document how to run a full grid.
- Update experiment README with PLATFORM env var and override examples.
 2026-07-08 05:49:55 +00:00
Quantong Qiu
227ef603a6 refactor: experiments/ + platforms/ layout and P800 SGLang support 
- Restructure repo around experiments/<name>/ and platforms/<chip>.env.
- Add shared scripts under scripts/common/ for platform/server/bench-client logic.
- Add Kunlun P800 platform config and runtime patches.
- Add dsv4_p800_sglang experiment with INT8 smoke-test support.
- Update BENCHMARK_WORKFLOW.md and README.md with chip/engine recording rules.
- Add scripts/analysis/compare_experiments.py for cross-experiment comparison.
- Ignore experiments/*/results/ raw output directories by default.
 2026-07-08 05:15:59 +00:00
Benchmark Bot
eb431d3925 Initial commit: benchmark scripts and final reports 2026-07-08 02:17:13 +00:00