Without chunked prefill the server OOM/crashes (exit 137) during or
immediately after concurrent 64k-context scenarios. Speculative decoding
remains disabled as requested.
These were failed/intermediate smoke-test runs created while debugging the
managed server startup. Keep the repository clean; new runs can be reproduced
with experiments/dsv4_p800_sglang/run_bench.sh.
Add experiments/dsv4_p800_max_context_length/ to find the longest input
context that P800 SGLang INT8 can serve for DeepSeek-V4-Flash-INT8.
- Docker-based server start with dynamic --context-length.
- Single-request, single-output-token probes for each candidate length.
- Records success/failure, server args, and metrics per length.
- Generates results.json and report.md summarizing the max supported length.
- Smoke-tested at 4096 tokens successfully.
start_server.sh referenced LOG_DIR and RESULT_ROOT without defining them,
causing an unbound-variable failure under set -u. Mirror the initialization
from run_bench.sh so the script can run standalone.
- New experiments/dsv4_h200_max_context_length/
- start_sglang.sh / start_vllm.sh accept target length as argument
- run_bench.sh tests a ladder of input lengths (default 64k -> 1M)
with --random-range-ratio 1.0 for exact length
- extract_metrics.py + parse_results.py produce results.json + report.md
- README.md documents usage and control variables
- Root README updated with entry and quick command
- parse_backend.py for sglang_vs_vllm and dspark_vs_default:
- compute slo_status per scenario (TTFT P95 < 3000ms, TPOT mean < 50ms)
- append slo_status to results.json
- add SLO column to report.md
- BENCHMARK_WORKFLOW.md documents slo_status in schema and checklist
- sglang_vs_vllm/run_bench.sh: add phase1/phase2 server_args per backend
- dspark_vs_default/run_bench.sh: add server_args for dspark/default
- READMEs document where to find the recorded server args
- config.env: add SHAREGPT_DATASET, SHAREGPT_CONTEXT_LEN and SHAREGPT_SCENARIOS
- run_bench.sh: add run_sharegpt() and integrate it after Phase 1 for each backend
- parse_backend.py: support both phase{1|2} and sharegpt raw output filenames
- README.md: document ShareGPT scenarios
- New experiments/dsv4_h200_sglang_vs_vllm/ with TP=8 on all 8 H200 cards
- Phase1 short-context throughput + Phase2 long context up to 200k
- Unified scenario matrix, warmup, parsing, and side-by-side comparison report
- H200 SGLang vs vLLM entry added to root README
- Add experiments/dsv4_h200_sglang (config, server start, orchestrator, parser, README)
- Fix start_server.sh to wait for HTTP 200 via curl --fail instead of treating 503 as ready
- Add H200 SGLang baseline to root README and update .gitignore for per-run raw_outputs/logs
- Remove stale experiments/dsv4_h200_dspark/DSPARK_FIX_PR_PREP.md
- Keep only bench_results/dspark_grid_20260707-132641/ in original location
- Move all other historical results to experiments/legacy_bench_results/
- Add README.md explaining the archive contents
- Update root README.md and BENCHMARK_WORKFLOW.md path references
- Use bash array for SCENARIOS to avoid word-splitting
- Fix metadata config (tp=4, scenarios as array)
- Rewrite parse_results.py for sglang.bench_serving --output-details format
- Update .gitignore to keep experiments/*/results/*.json and *.md,
ignore only raw_outputs/ and logs/ subdirs
- First successful run: 20260708-062348
- Update platforms/nvidia_h200.env with ENGINE and configurable venv/paths.
- Update platforms/README.md with H200 notes and platform-addition guide.
- Add docs/H200_QUICKSTART.md with concrete H200 usage steps.
- Add experiments/dsv4_h200_dspark/ as a migration wrapper around the
existing scripts/benchmark_dspark_0707/ grid benchmark, writing results
into the new experiments/<name>/results/<RUN_ID>/ layout.
- Update BENCHMARK_WORKFLOW.md with a 'Adding a New Platform or Experiment'
section and mention the H200 wrapper in Quick Start.
- Update main README.md to index the H200 experiment and link to the guide.
- Remove USE_FAST_BFP16_MOE=1 which caused torch parameter assignment
crash in fresh containers for w8a8_int8 mode.
- Mount model path at its host absolute path inside the container so the
bench client tokenizer can load it regardless of server --model-path.
- Default WARMUP to 0 and NUM_PROMPTS to 10 for a fast smoke test.
- Default to a single scenario; document how to run a full grid.
- Update experiment README with PLATFORM env var and override examples.
