Gate: continuous scaling on shared endpoints; adaptive dwell; MIN_OK 5

User-directed changes:
- steady no longer pins the converged level: the endpoint is shared,
  other tenants move its capacity mid-run, so steady keeps judging
  forever (+1 when rate beats reference by 5%, -1 when 15% below,
  reference drifts by EWMA). Large drops are still handled by the
  failure channel's multiplicative x0.7; the +-1 path tracks drift.
- dwell fallback scales with the OBSERVED completion cadence:
  max(120s, 3x inter-completion gap EMA). A 25s timer judged 60s-per-
  request benches on one lone sample.
- MIN_OK floor raised to 5 (evidence = max(5, 2x level)).

Simulated capacity drift 8->3->8: gate follows down then recovers.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
sora 2026-09-15 03:46:14 +00:00
parent b81bff662d
commit 8fc7df5b26

View File

@ -162,8 +162,11 @@ class AdaptiveGate:
# bisect narrows to the knee; steady holds there. Any failure x0.7s
# immediately and restarts probing from the shrunken level.
GAIN_EPS = 1.1 # rate must beat the previous level by 10% to keep doubling
MIN_OK = 3 # baseline completions needed at a level before judging
MAX_AT_LEVEL_S = 25 # ... or this many seconds, whichever comes first
MIN_OK = 5 # baseline completions needed at a level before judging
# dwell fallback: JUDGE also when the level has been held this long (with
# >= 1 completion) -- scaled by observed inter-completion gap so a bench
# whose single request takes 60s is not judged on one lone sample at t=25s
DWELL_BASE_S = 120.0
# robust judging: sample count scales WITH the level (a 2-completion
# estimate at level 8 is pure quantization noise), plus a minimum dwell
# so one lucky tick cannot speak for the whole level
@ -256,6 +259,13 @@ class AdaptiveGate:
if ok:
self._interval_ok += 1
self._level_ok += 1
now = time.monotonic()
last = getattr(self, '_last_ok_t', None)
if last is not None:
gap = now - last
ge = getattr(self, '_gap_ema', None)
self._gap_ema = gap if ge is None else 0.6 * ge + 0.4 * gap
self._last_ok_t = now
else: # multiplicative decrease -- survival first
self._interval_fails += 1
# capacity moved (or we overshot): shrink now and restart the
@ -310,12 +320,17 @@ class AdaptiveGate:
lvl = max(1, int(self.limit))
# not enough evidence yet at this level: keep measuring.
# need = max(MIN_OK, level): rate noise shrinks only with
# need = max(MIN_OK, 2x level): rate noise shrinks only with
# samples proportional to the concurrency being judged
need_ok = max(self.MIN_OK, lvl * 2)
if self._level_ok < need_ok and dt < self.MAX_AT_LEVEL_S:
# dwell fallback scales with the OBSERVED completion cadence:
# a 60s-per-request bench needs minutes, not 25s, before a
# single-sample judgment is acceptable
need_dt = max(self.DWELL_BASE_S,
3.0 * (getattr(self, '_gap_ema', None) or 0.0))
if self._level_ok < need_ok and dt < need_dt:
return
# zero completions in MAX_AT_LEVEL_S: hang or overloaded -> hold
# zero completions so far: hang or overloaded -> hold
if self._level_ok == 0:
return
@ -356,17 +371,24 @@ class AdaptiveGate:
else:
self._enter_level('steady') # hit HI with gains: stay
elif self._mode == 'steady':
# capacity estimates are noisy: periodically re-probe upward
self._steady_ticks = getattr(self, '_steady_ticks', 0) + 1
if self._steady_ticks >= 12: # ~60s at PROBE_S=5
self._steady_ticks = 0
self._mode = 'probe'
self._prev = (lvl, rate)
# try one level up, not a full double, from settled state
# the endpoint is SHARED: other tenants change its capacity
# while we run -- keep judging forever, nudge +-1 against the
# reference rate instead of pinning the converged level
ref = self._good_rate or rate
if rate >= ref * 1.05 and lvl < self.HI:
self.limit = float(min(lvl + 1, self.HI))
self.stats['ramp_demand'] += 1
self._push_limit()
self._enter_level()
return
elif rate <= ref * 0.85 and lvl > self.LO:
self.limit = float(max(self.LO, lvl - 1))
self.stats['backoff_queue'] += 1
self._push_limit()
self._enter_level()
else:
# reference drifts with fresh measurements (slow EWMA)
self._good_rate = 0.7 * ref + 0.3 * rate
self._enter_level() # restart the measurement window
elif self._mode == 'bisect':
lo, hi = self._bis
if rate >= self._good_rate * 0.9: