17 Commits

Author SHA1 Message Date
SSKJ Dev
90fc378d2b Add H20 TP×DP matrix experiments for vLLM and SGLang
- Add dsv4_h20_vllm_tp_dp_matrix experiment (vLLM backend)
- Add dsv4_h20_sglang_tp_dp_matrix experiment (SGLang backend)
- Add nvidia_h20 platform config with H20-specific paths
- Fix platform.sh auto-detection to distinguish H20 from H200
- Fix vLLM Docker startup: remove --entrypoint override for CDI compat
- Fix vLLM 0.25.1 CLI args: remove redundant 'serve' from SERVER_ARGS
- Download ShareGPT dataset to local datasets dir
- Rename DSL to OSL across both experiments
2026-07-16 03:43:58 +00:00
Quantong Qiu
5405d002fc chore: remove legacy benchmark files and scripts
Delete obsolete files that are no longer maintained:
- experiments/legacy_bench_results/ (old benchmark reports and summaries)
- scripts/benchmark_dspark_0707/ (outdated dspark benchmark scripts)
- scripts/start_dsv4_dspark_8card.sh (deprecated deployment script)
2026-07-14 10:47:09 +00:00
Quantong Qiu
e0ae0715b1 feat(adaptive): add RESUME_RUN_ID resume support and rename DSL to OSL
- Add RESUME_RUN_ID env var for breakpoint resume in adaptive benchmarks.
  When set, the script reuses an existing result directory and skips
  already-tested (TP, DP, ISL, OSL) shapes based on adaptive_shapes.jsonl.
- Fix RUN_ID unbound variable in resume mode.
- Fix jq query to use -s (slurp) for jsonl files.
- Add resume skip logic to DRY_RUN mode as well.
- Rename DSL -> OSL across all adaptive benchmark files for consistency:
  - scripts/common/adaptive_bench_lib.sh
  - scripts/common/adaptive_concurrency.py
  - experiments/dsv4_h200_vllm_tp_dp_matrix/adaptive_config.env
  - experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh
  - experiments/dsv4_h200_sglang_tp_dp_matrix/adaptive_config.env
  - experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
  - experiments/ADAPTIVE_CONCURRENCY_USAGE.md
2026-07-14 10:41:50 +00:00
iiGray
60826dda09 add run_adaptive_concurrency_add16 2026-07-13 17:26:33 +08:00
Root User
cf7a12932e bench: add adaptive run heartbeat monitor 2026-07-11 13:04:35 +00:00
Root User
5e95cee1a0 bench: add adaptive concurrency search 2026-07-11 06:11:40 +00:00
yy-fighting
59dbadb110 add dp_proxy for vLLM DP, write kv_cache_analysis.md, fix DP bench port 2026-07-09 10:19:30 +00:00
yy-fighting
04a384b265 add dsv4_h200_vllm_tp_dp_matrix experiment (TP×DP matrix with Y-only mode, 1M excluded) 2026-07-09 09:55:18 +00:00
yy-fighting
80959cf765 docs: move SLO_STANDARDS.md from scripts/ to docs/; update references 2026-07-08 11:06:49 +00:00
yy-fighting
9b485c05f7 cleanup(scripts): remove unused legacy scripts; update README and workflow docs 2026-07-08 11:04:16 +00:00
yy-fighting
a127cf7ef0 refactor(common): centralize parse/compare/warmup; add TEMPLATE and EXPERIMENT_GUIDE 2026-07-08 10:48:43 +00:00
Quantong Qiu
20d04a31ae refactor: move final legacy result dspark_grid into experiments/legacy_bench_results/
- Move bench_results/dspark_grid_20260707-132641/ to experiments/legacy_bench_results/
- Remove empty bench_results/ directory
- Update README.md, BENCHMARK_WORKFLOW.md, legacy README, and parse_results.py default path
- Update internal report/evaluation paths
2026-07-08 06:40:39 +00:00
Quantong Qiu
07835b9fed chore: remove obsolete scripts/test_block_sparse_attn.py after refactor 2026-07-08 06:01:31 +00:00
Quantong Qiu
f0d4c4753b fix(common): robust XPU platform auto-detection
Capture xpu-smi output before grepping to avoid SIGPIPE failures under
set -o pipefail when grep -q exits early.
2026-07-08 05:50:42 +00:00
Quantong Qiu
e72e34c2f7 fix(p800): managed-mode server startup and smoke-test defaults
- Remove USE_FAST_BFP16_MOE=1 which caused torch parameter assignment
  crash in fresh containers for w8a8_int8 mode.
- Mount model path at its host absolute path inside the container so the
  bench client tokenizer can load it regardless of server --model-path.
- Default WARMUP to 0 and NUM_PROMPTS to 10 for a fast smoke test.
- Default to a single scenario; document how to run a full grid.
- Update experiment README with PLATFORM env var and override examples.
2026-07-08 05:49:55 +00:00
Quantong Qiu
227ef603a6 refactor: experiments/ + platforms/ layout and P800 SGLang support
- Restructure repo around experiments/<name>/ and platforms/<chip>.env.
- Add shared scripts under scripts/common/ for platform/server/bench-client logic.
- Add Kunlun P800 platform config and runtime patches.
- Add dsv4_p800_sglang experiment with INT8 smoke-test support.
- Update BENCHMARK_WORKFLOW.md and README.md with chip/engine recording rules.
- Add scripts/analysis/compare_experiments.py for cross-experiment comparison.
- Ignore experiments/*/results/ raw output directories by default.
2026-07-08 05:15:59 +00:00
Benchmark Bot
eb431d3925 Initial commit: benchmark scripts and final reports 2026-07-08 02:17:13 +00:00