- New experiment: dsv4_p800_sglang_profile for PyTorch profiler testing
- Documents XPU cuptiActivityDisable bug (error 17) when saving traces
- Includes benchmark results (ISL=4k, OSL=1k, C=16, TP8/DP1)
- PROFILE_REPORT.md with full analysis of P800 vs H20 performance gap
- Add dsv4_p800_sglang_tp_dp_official experiment config
- Created adaptive_summary.md to summarize adaptive concurrency search results, including metrics such as TP, DP, and TPS.
- Added run_manifest.json to document experiment configuration details, including hardware specifications and search parameters.
- Introduced shapes.tsv to define input and output lengths for various configurations in the adaptive experiment.
- Introduced a new CUDA kernel for transposing and packing FP32 into UE8.
- Added binary file for the new kernel.
- Created multiple autotune configuration files for version 0.6.14 of FlashInfer, supporting various input shapes and configurations.
- Added model information JSON for DeepseekV4ForCausalLM, detailing its architecture and capabilities.
- New experiments/p800/dsv4_p800_sglang_tp_dp_matrix: TP8/DP1, TP4/DP2,
TP2/DP4 matrix with smoke results; TP2/DP4 documents the weight-loading
OOM root cause (274 GiB INT8 weights sharded only across TP group).
- Launch args drop --ep-size/--chunked-prefill-size/--max-prefill-tokens/
--max-running-requests; experts fall back to TP sharding.
- Move dsv4_p800_256k_4k_probe under experiments/p800/.
- scripts/common: jq-free parsing fixes in adaptive_bench_lib.sh and
parse_backend.py.
- .gitignore: cover raw_outputs under nested platform experiment layout.
- Add RESUME_RUN_ID env var for breakpoint resume in adaptive benchmarks.
When set, the script reuses an existing result directory and skips
already-tested (TP, DP, ISL, OSL) shapes based on adaptive_shapes.jsonl.
- Fix RUN_ID unbound variable in resume mode.
- Fix jq query to use -s (slurp) for jsonl files.
- Add resume skip logic to DRY_RUN mode as well.
- Rename DSL -> OSL across all adaptive benchmark files for consistency:
- scripts/common/adaptive_bench_lib.sh
- scripts/common/adaptive_concurrency.py
- experiments/dsv4_h200_vllm_tp_dp_matrix/adaptive_config.env
- experiments/dsv4_h200_vllm_tp_dp_matrix/run_adaptive_concurrency.sh
- experiments/dsv4_h200_sglang_tp_dp_matrix/adaptive_config.env
- experiments/dsv4_h200_sglang_tp_dp_matrix/run_adaptive_concurrency.sh
- experiments/ADAPTIVE_CONCURRENCY_USAGE.md
- Implemented a bash script to run six benchmark experiments sequentially.
- Added features for fault tolerance, out-of-memory recovery, and checkpoint management.
- Included detailed logging for monitoring experiment progress and results.
- Integrated GPU memory cleanup checks before each experiment.
- Provided functionality to resume experiments from checkpoints.
- Run vLLM server inside vllm/vllm-openai:latest container on H200.
- Run benchmark client inside lmsysorg/sglang:latest using
sglang.bench_serving --backend vllm, matching the sglang reference.
- Use RUNTIME_BASE for self-contained logs/pids/tmp instead of hardcoded
/data/user1/yy paths.
- Update model path to /data3/hf_models/DeepSeek-V4-Flash.
- Sample only low/high concurrency endpoints to match sglang matrix.
- Fix docker 'invalid reference format' caused by DOCKER_CLIENT_IMAGE not
being exported to bash -c subshells used by run_bench_serving.
- Add offline dummy ShareGPT dataset for random benchmark mode.
- Create configuration file for the custom benchmark experiment.
- Implement result parsing script to handle JSONL outputs from the benchmark.
- Develop run script to orchestrate the benchmark execution, including server management and health checks.
- Add server start script to launch a single vLLM service on all available GPUs.
- Fix docker invalid reference format by exporting DOCKER_IMAGE and
USE_DOCKER_CLIENT to bash subshells (run_bench_serving)
- Add persistent container mode: start_sglang_container.sh +
run_sglang_in_container.sh for faster config switching
- Add README.md with setup and migration guide
export -f only exports the function definition, not the variables it
references. MODEL_PATH and RESULT_BASE were empty inside timeout bash -c
subshells, causing docker run -v ::ro invalid spec. Add explicit export
for all variables used by run_bench_serving().
export -f only exports the function definition, not the variables it
references. DATASET_PATH was empty inside timeout bash -c subshells,
causing docker run -v ::ro invalid spec even after the dynamic vol_args
fix. Add explicit export after sourcing config.env.
When DATASET_PATH is empty, -v ::ro is passed to docker run, which
produces 'invalid spec: ::ro: empty section between colons'. Build
the volume args dynamically and only add the dataset mount when the
path is non-empty.
timeout(1) cannot execute shell functions directly; it needs a standalone
command. Export run_bench_serving via export -f and wrap both warmup and
scenario runs in bash -c so the function is visible in the subshell.
Source code was already committed; this adds only the structured final
artifacts from run 20260709-152419_matrix. Raw jsonl/logs/gpu_logs remain
ignored per .gitignore.
