qqtang/sskj
1
2
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
78 Commits 1 Branch 0 Tags
Commit Graph

4 Commits

Author SHA1 Message Date
Quantong Qiu
a1da95ef44 Scale P800 long-context matrix to concurrency=1 for stability 
- P800 INT8 cannot sustain >1 concurrent long-context request without OOM;
  set all scenarios to concurrency=1.
- Keep max_running=2 because max_running=1 triggers req_to_token_pool
  allocation failures in this image.
- Reduce 64k max_context_len to 72000 (65536 + 4096 + pad).
- Make run_bench.sh resilient to container death: detect dead container and
  skip remaining scenarios in the group instead of crashing the orchestrator.
- Update README to document the concurrency limitation.
 2026-07-09 05:59:28 +00:00
Quantong Qiu
46e05a57a4 Re-enable chunked prefill for P800 long-context matrix stability 
Without chunked prefill the server OOM/crashes (exit 137) during or
immediately after concurrent 64k-context scenarios. Speculative decoding
remains disabled as requested.
 2026-07-09 05:13:04 +00:00
Quantong Qiu
59f480b1bc Disable chunked prefill and speculative decoding in P800 long-context matrix 2026-07-09 04:47:59 +00:00
Quantong Qiu
90eecc4ede Add P800 long-context matrix experiment (64k/128k) for DeepSeek-V4-Flash-INT8 
- New experiment: experiments/dsv4_p800_long_context_matrix/
- Single backend: sglang-xpu in Docker on Kunlun P800
- Context groups: 64k (max_context_len=80000, max_running=10) and
  128k (max_context_len=140000, max_running=5)
- Scenarios: concurrency sweeps x output lengths 256/1024/4096
- Parameterized start_sglang.sh enforces max_running >= 2 to avoid
  P800 INT8 req_to_token_pool allocation failures
- Reuses proven P800 INT8 environment, NSA/EAGLE settings, and
  bench_serving invocations from dsv4_p800_max_context_length
 2026-07-09 04:45:04 +00:00