- Fix docker invalid reference format by exporting DOCKER_IMAGE and
USE_DOCKER_CLIENT to bash subshells (run_bench_serving)
- Add persistent container mode: start_sglang_container.sh +
run_sglang_in_container.sh for faster config switching
- Add README.md with setup and migration guide
export -f only exports the function definition, not the variables it
references. MODEL_PATH and RESULT_BASE were empty inside timeout bash -c
subshells, causing docker run -v ::ro invalid spec. Add explicit export
for all variables used by run_bench_serving().
export -f only exports the function definition, not the variables it
references. DATASET_PATH was empty inside timeout bash -c subshells,
causing docker run -v ::ro invalid spec even after the dynamic vol_args
fix. Add explicit export after sourcing config.env.
When DATASET_PATH is empty, -v ::ro is passed to docker run, which
produces 'invalid spec: ::ro: empty section between colons'. Build
the volume args dynamically and only add the dataset mount when the
path is non-empty.
timeout(1) cannot execute shell functions directly; it needs a standalone
command. Export run_bench_serving via export -f and wrap both warmup and
scenario runs in bash -c so the function is visible in the subshell.
Add rule 10: commit experiment code together with final results.json
and .md reports; do not commit logs/raw_outputs/gpu_logs. Also update
the pre-commit checklist accordingly.
Source code was already committed; this adds only the structured final
artifacts from run 20260709-152419_matrix. Raw jsonl/logs/gpu_logs remain
ignored per .gitignore.
- config.env: switch to /data/models/DeepSeek-V4-Flash, USE_DOCKER=1,
CONTEXT_LENGTH=1M, CONCURRENCY_SAMPLES=2, add DATASET_PATH.
- start_sglang_docker.sh: add --ipc host --shm-size 16g for NCCL.
- start_sglang_dp.sh: use --model-path for Docker CLI.
- run_bench.sh: mount dataset into client container and pass --dataset-path
to avoid HF download on offline nodes.
- Document the rule in docs/EXPERIMENT_GUIDE.md section 7
- Update all config.env SCENARIOS to follow the 5x rule
- Migrate dsv4_h200_sglang, dsv4_h200_vllm, dsv4_p800_sglang
from global NUM_PROMPTS to per-scenario num_prompts
- Keep long-context phase2 as exception (low counts documented)
- Add legacy 3-field fallback in run_bench.sh loops
- P800 INT8 cannot sustain >1 concurrent long-context request without OOM;
set all scenarios to concurrency=1.
- Keep max_running=2 because max_running=1 triggers req_to_token_pool
allocation failures in this image.
- Reduce 64k max_context_len to 72000 (65536 + 4096 + pad).
- Make run_bench.sh resilient to container death: detect dead container and
skip remaining scenarios in the group instead of crashing the orchestrator.
- Update README to document the concurrency limitation.
Without chunked prefill the server OOM/crashes (exit 137) during or
immediately after concurrent 64k-context scenarios. Speculative decoding
remains disabled as requested.
These were failed/intermediate smoke-test runs created while debugging the
managed server startup. Keep the repository clean; new runs can be reproduced
with experiments/dsv4_p800_sglang/run_bench.sh.
Add experiments/dsv4_p800_max_context_length/ to find the longest input
context that P800 SGLang INT8 can serve for DeepSeek-V4-Flash-INT8.
- Docker-based server start with dynamic --context-length.
- Single-request, single-output-token probes for each candidate length.
- Records success/failure, server args, and metrics per length.
- Generates results.json and report.md summarizing the max supported length.
- Smoke-tested at 4096 tokens successfully.
start_server.sh referenced LOG_DIR and RESULT_ROOT without defining them,
causing an unbound-variable failure under set -u. Mirror the initialization
from run_bench.sh so the script can run standalone.
