14 Commits

Author SHA1 Message Date
yy-fighting
8652a685e6 rewrite README, add new platform onboarding guide, fix broken scripts/common paths
- rewrite README with project purpose, standard workflow, corrected index
- add docs/NEW_PLATFORM_GUIDE.md (new GPU onboarding SOP, GLM5.2 reuse)
- fix ../../scripts/common -> ../../../scripts/common in 42 experiment scripts
- refresh stale docs (EXPERIMENT_GUIDE, H200_QUICKSTART, ADAPTIVE_CONCURRENCY_USAGE, BENCHMARK_WORKFLOW)
- remove dead code (dp_proxy.py) and .bak leftovers
- add p800 adaptive results (tp4_dp2/tp8_dp1 metrics + summary)
- gitignore envs/charts and .tmp_charts
2026-07-17 06:18:05 +00:00
Quantong Qiu
819850c4a1 Update README.md with new experiments and RTX 6000D platform
- Add all H200 experiments: TP/DP matrix, adaptive concurrency, custom bench,
  MTP vs default, long context matrix, 64k/256k probes
- Add RTX 6000D platform and experiments (sglang/vllm TP/DP adaptive)
- Update environment requirements section with RTX 6000D details
- Update platforms directory description to include RTX 6000D
2026-07-13 03:04:46 +00:00
yy-fighting
80959cf765 docs: move SLO_STANDARDS.md from scripts/ to docs/; update references 2026-07-08 11:06:49 +00:00
yy-fighting
9b485c05f7 cleanup(scripts): remove unused legacy scripts; update README and workflow docs 2026-07-08 11:04:16 +00:00
yy-fighting
19d336de08 feat: add max context length exploration experiment
- New experiments/dsv4_h200_max_context_length/
- start_sglang.sh / start_vllm.sh accept target length as argument
- run_bench.sh tests a ladder of input lengths (default 64k -> 1M)
  with --random-range-ratio 1.0 for exact length
- extract_metrics.py + parse_results.py produce results.json + report.md
- README.md documents usage and control variables
- Root README updated with entry and quick command
2026-07-08 08:35:56 +00:00
yy-fighting
5d1cd9c266 feat: add vLLM+DSpark vs vLLM-default TTFT comparison experiment
- New experiments/dsv4_h200_vllm_dspark_vs_default/
- start_dspark.sh: vllm serve with --spec-method dspark --spec-tokens 5
- start_default.sh: plain vllm serve baseline
- Unified scenario matrix (512/4000 input, 1/32/128 concurrency)
- parse_backend.py + compare.py focused on TTFT differences
- README.md + root README entries
2026-07-08 08:15:02 +00:00
yy-fighting
e6d9e99b6f feat: add SGLang vs vLLM controlled comparison experiment
- New experiments/dsv4_h200_sglang_vs_vllm/ with TP=8 on all 8 H200 cards
- Phase1 short-context throughput + Phase2 long context up to 200k
- Unified scenario matrix, warmup, parsing, and side-by-side comparison report
- H200 SGLang vs vLLM entry added to root README
2026-07-08 07:34:34 +00:00
yy-fighting
8b82f700e4 feat: add H200 SGLang baseline experiment and fix health check
- Add experiments/dsv4_h200_sglang (config, server start, orchestrator, parser, README)
- Fix start_server.sh to wait for HTTP 200 via curl --fail instead of treating 503 as ready
- Add H200 SGLang baseline to root README and update .gitignore for per-run raw_outputs/logs
- Remove stale experiments/dsv4_h200_dspark/DSPARK_FIX_PR_PREP.md
2026-07-08 06:56:45 +00:00
Quantong Qiu
20d04a31ae refactor: move final legacy result dspark_grid into experiments/legacy_bench_results/
- Move bench_results/dspark_grid_20260707-132641/ to experiments/legacy_bench_results/
- Remove empty bench_results/ directory
- Update README.md, BENCHMARK_WORKFLOW.md, legacy README, and parse_results.py default path
- Update internal report/evaluation paths
2026-07-08 06:40:39 +00:00
Quantong Qiu
0397b4a2b5 refactor: move legacy bench_results into experiments/legacy_bench_results/
- Keep only bench_results/dspark_grid_20260707-132641/ in original location
- Move all other historical results to experiments/legacy_bench_results/
- Add README.md explaining the archive contents
- Update root README.md and BENCHMARK_WORKFLOW.md path references
2026-07-08 06:36:19 +00:00
Quantong Qiu
a5d69dfd9a docs(h200): H200 quick-start guide, porting workflow, and example experiment
- Update platforms/nvidia_h200.env with ENGINE and configurable venv/paths.
- Update platforms/README.md with H200 notes and platform-addition guide.
- Add docs/H200_QUICKSTART.md with concrete H200 usage steps.
- Add experiments/dsv4_h200_dspark/ as a migration wrapper around the
  existing scripts/benchmark_dspark_0707/ grid benchmark, writing results
  into the new experiments/<name>/results/<RUN_ID>/ layout.
- Update BENCHMARK_WORKFLOW.md with a 'Adding a New Platform or Experiment'
  section and mention the H200 wrapper in Quick Start.
- Update main README.md to index the H200 experiment and link to the guide.
2026-07-08 05:55:11 +00:00
Quantong Qiu
227ef603a6 refactor: experiments/ + platforms/ layout and P800 SGLang support
- Restructure repo around experiments/<name>/ and platforms/<chip>.env.
- Add shared scripts under scripts/common/ for platform/server/bench-client logic.
- Add Kunlun P800 platform config and runtime patches.
- Add dsv4_p800_sglang experiment with INT8 smoke-test support.
- Update BENCHMARK_WORKFLOW.md and README.md with chip/engine recording rules.
- Add scripts/analysis/compare_experiments.py for cross-experiment comparison.
- Ignore experiments/*/results/ raw output directories by default.
2026-07-08 05:15:59 +00:00
Quantong Qiu
7de88b995a docs: 规范 benchmark 结果存储格式,要求每个实验同时产出 .md 报告和结构化 results.json 2026-07-08 03:31:47 +00:00
Benchmark Bot
eb431d3925 Initial commit: benchmark scripts and final reports 2026-07-08 02:17:13 +00:00