One Bit per Dead Time

04 October 2026 • Yuriy Polyulya •

#distributed-systems #control-theory #systems-thinking #queueing-theory

Every pod on the dashboard reads 100% CPU. The replica count says 20, then holds. Two minutes later it says 40 and holds again. Two minutes after that it says 80. Nobody has touched the autoscaler, and nothing is broken. It is doing what it was built to do, one step at a time. The engineer on call has one question the dashboard cannot answer: how much load is actually arriving?

The autoscaler cannot answer it either. That inability is the subject of this post.

The numbers above come from a model of an ordinary service, which this series calls the specimen and uses in all four parts. Ten pods sit behind the Kubernetes Horizontal Pod Autoscaler, scaling on CPU at a 50% target. Each pod may use one core and serves a hundred requests a second at full use. A new pod takes 90 seconds from the order to its first served request. That is the provisioning window The Phase MAPE-K Usually Skips used for the time from a scaling decision to a new instance being live, kept here so the two results can be compared. Then traffic jumps to ten times its usual rate.

In that model the replica count roughly doubles every two minutes, from 20 to more than 300. Capacity catches up with demand after 345 seconds, and the count keeps climbing anyway, because a backlog piled up while capacity was short. It peaks at 317 pods, where 100 would have done the job.

Now the part that matters. Had traffic jumped to three times its rate instead of ten, the first two requests for capacity would have been identical: 20 pods, then 40. Had it jumped thirty times, the first five would have matched the tenfold run exactly. For the first several minutes of an overload, the controller’s behaviour carries no information about how large the overload is. It finds out only by adding capacity and watching what happens next.

The same pattern shows up in ordinary operations. A launch, a retry storm after a dependency recovers, or a batch job pointed at the wrong endpoint all produce it. The replica graph rises in stairs, each stair one provisioning delay wide. The usual reading is that the autoscaler is slow and needs tuning. This post argues for a narrower reading. The stairs are the controller measuring the overload, at a rate fixed by arithmetic that no setting in its policy changes.

Notation used across the series

where:

• $\tau$ - the dead time, from ordering a pod to its first served request; 90 seconds in the specimen
• $k$ - the size of an overload, as a multiple of the load before it; $K$ when it is drawn at random
• $u^\ast$ and $\rho$ - the utilization target and the utilization a service actually runs at
• $\ell$ and $r$ - a pod’s CPU limit and CPU request
• $g = \ell/(r\,u^\ast)$ - the growth factor: the most the controller can multiply its ready pods by in one round
• $W$ and $w = W/\tau$ - the downscale stabilization window, and the same window measured in dead times
• $T$ , $c$ and $D = T/c$ - a backlog threshold in messages per pod, the rate of one pod, and the standing delay every message waits at rest
• $\Phi(w)$ - the standing delay, in dead times, a backlog loop with window $w$ needs to rest
• $A$ and $A_{\text{fail}}$ - the retry amplification: units of work per first attempt once every attempt fails, and the same quantity whatever sets it, retry counts, budgets or limiters
• $\lambda$ , $C$ , $\tau_{\text{out}}$ and $\Delta$ - demand, capacity, the client timeout and the length of a spike
• $M$ and $b$ - a replica ceiling and an admission bound in seconds of capacity
• L1 · Bound, L2 · Fit and L3 · Estimate - the tag on each claim: a proof or a cited fact, a judgment or approximation the text argues for, and a worked example computed from the model; each part’s ledger names the layer of every law it records

What a Loop Cannot Know

This series takes one widely deployed autoscaler and asks four questions about it, one per part. Each question names something the loop cannot know from its own signal. Each part then prices the belief that lets engineers forget it.

1. Size. A utilization metric cannot report more load than the pods can carry. What does the loop do when the load it must size exceeds anything its metric can show? This post answers that.
2. Time. A loop driven by a backlog reacts to work that has already waited. What does a stabilization window, added to stop flapping, do to such a loop? Still open when this post ends.
3. State. A demand-rate signal sees what clients ask for now. It misses the work that earlier requests left behind. When does that gap hold a service in a failed state? Also open.
4. Where the excess goes. During the provisioning delay the replica count controls nothing. What bounds the damage then, and what does that bound cost? Also open.

    
    %%{init: {'theme': 'neutral'}}%%
flowchart LR
    classDef root fill:none,stroke:#333,stroke-width:3px;
    classDef branch fill:none,stroke:#ca8a04,stroke-width:2px;
    classDef leaf fill:none,stroke:#4a90d9,stroke-width:1.5px;
    D["Clients
first attempts and retries"]:::leaf
    S["Service
queue and ready pods"]:::root
    C["Controller
ratio, tolerance, window"]:::branch
    P["Provisioning
the dead time"]:::branch
    X["The excess
held, retried, failed or refused"]:::leaf
    D -->|"requests"| S
    S -->|"Size: CPU utilization, capped at the limit"| C
    S -->|"Time: the backlog, work that already waited"| C
    D -->|"State: the first-attempt rate, blind to debt"| C
    C -->|"order pods"| P
    P -->|"pods ready after the dead time"| S
    S -->|"Where the excess goes"| X
    X -.->|"timeouts become retries"| D

Each part has four beats. It starts from a belief an engineer can reasonably hold, with evidence that the belief is held. It reads the mechanism as it actually runs, from source. It derives a law and states the range where the law holds. It closes with the bill: what the law costs, when the advice reverses, and which observation on a real cluster would prove it wrong.

The specimen is Kubernetes at tag v1.34.0. It is the evidence for the argument rather than its subject. Every result is stated for the class of controller it belongs to: a sampled, quantized ratio controller acting through a provisioning delay. Each result is then shown on the controller most engineers already run.

Every number in this post comes from executed code. A plain port of the controller’s calculation, read from the v1.34.0 source and checked against eight test cases from the upstream Kubernetes test suite, drives a fluid model of the service. Each figure states the assumptions that produce it. The times, counts and ratios the model prints belong to this specimen and illustrate the laws; they are not measurements of any cluster. What carries over is the laws themselves, and the direction in which each assumption moves the numbers.

The Belief: the Metric Measures Load

L1 · Bound When a HorizontalPodAutoscaler is created without naming a metric, the API fills one in. It picks CPU utilization with a target of 80%. The constant is DefaultCPUUtilization = 80, in pkg/apis/autoscaling/annotations.go. The comment beside it says it applies “provided no other metrics are present”[1] . A default records what the designers expect most users to want. This one says: watch how busy the pods are, and keep them about four-fifths busy.

That choice rests on a belief worth stating plainly, because it is reasonable. A busier pod means more work is arriving. If utilization doubles, roughly twice the work is arriving, so roughly twice the pods are needed. The metric is read as a measurement of load. The target is read as a cost setting: run hotter to pay for fewer pods, or run cooler to keep headroom.

The documentation states the rule in the same spirit. Its algorithm section opens: “From the most basic perspective, the HorizontalPodAutoscaler controller operates on the ratio between desired metric value and current metric value”[2] . The formula that follows is the one most engineers can recite:

$$\text{desiredReplicas} = \left\lceil \text{currentReplicas} \times \frac{\text{currentMetricValue}}{\text{desiredMetricValue}} \right\rceil$$

L1 · Bound Published work reads the controller the same way. Rolik and Volkov study the over-regulation seen in horizontal pod autoscaling. They model it as a proportional regulator in a feedback loop with delay, and test proportional-integral-derivative designs against it[3] . Their abstract names the problem as “the over-regulation effect that occurs under certain conditions by horizontal pod autoscaling”.

Below saturation the belief holds well. If the metric can report the load, the ratio rule computes the right count in one step: a service at 100% of a 50% target on ten pods needs twenty, and the rule asks for twenty. The model confirms it. With the CPU limit set a hundred times above the request, so that the metric never saturates, a twofold, tenfold and thirtyfold step each peak at exactly the pods needed. The proportional reading of the controller is accurate in that regime.

Two things are left out. The one-line formula is not the calculation the controller runs, and the metric it divides by has a ceiling. The first matters for stability, and the second for everything else in this post.

The Rule Everyone Quotes and the Rule That Runs

The documentation does not hide the difference. Below the formula, in prose, it describes how pods that are not yet ready are set aside before the ratio is computed, and how pods whose metrics are missing are handled. The one-line rule is a summary. What follows is the calculation itself, read from replica_calculator.go and horizontal.go at v1.34.0[1] .

• L1 · Bound Grouping. Every pod the autoscaler targets is sorted into a group before any arithmetic happens. Pods being deleted and failed pods are ignored. Pending pods count as unready. Pods with no metric count as missing. For CPU, a running pod also counts as unready while it is inside its initialization period and either reports not-ready or has not yet produced a full metric window since it became ready.
• L1 · Bound The ratio. The usage ratio comes from the ready pods only: their summed usage over their summed requests, truncated to a whole percentage, divided by the target. With no unready or missing pods, the answer is ceil(usageRatio × readyPodCount). The currentReplicas of the documentation is, in this branch of the code, the number of ready pods.
• L1 · Bound Unready pods on a scale-up. If the ratio says scale up and some pods are not yet ready, those pods are added back with a usage of zero. The ratio is recomputed over all of them, and the new count is ceil(newRatio × podsInTheCalculation). Missing metrics are treated the same way on a scale-up. On a scale-down they count as full use of their request.
• L1 · Bound The hold. If the recomputed ratio falls inside the tolerance, or points the other way from the first ratio, the controller keeps the current count. The default tolerance is 10% either side of the target.
• L1 · Bound Limits. Two limits apply afterwards on the default path. A scale-up in one sync may not exceed max(2 × current, 4) replicas. A scale-down is held to the highest recommendation of the last five minutes. The controller syncs every 15 seconds.

One Sync, Walked Through

L3 · Estimate The port runs the calculation on concrete inputs. Take pods requesting one core, a 50% target, and every ready pod at its limit, which is the saturated case. Hold the ready pods fixed and sweep the number already on order, from none to many.

L1 · Bound The ratio over the ready pods is 2 at every point of the sweep, and what the sync orders changes only through the pods not yet ready. With ten pods ready and nothing on order, the sync orders ten more and the count doubles. With six already on order, sixteen replicas in all, the ordered pods dilute the utilization over all pods to 62%, the recomputed count is 20, and the sync orders four. At eighteen replicas the dilution brings the ratio inside the tolerance band and the controller holds, two short of twice the ready pods. Beyond that, the recomputed ratio points downward while the first one points upward, and the direction rule holds as well: at forty replicas with ten ready the utilization over all pods is 25%, and nothing is ordered. Every pod on order reduces what the sync orders by one, so no sync ever takes the total past twice the ready pods.

L1 · Bound One detail is worth keeping for later. The documentation states that “the original value for the average utilization is reported back via the HorizontalPodAutoscaler status, without factoring in the not-yet-ready pods or missing metrics, even when the new usage ratio is used”[2] . The number on the status is not the number the decision used. During the first round of an overload the status reads 100% while the decision runs on 62%, then 50%. An engineer reading the status sees a controller that ignores a saturated metric. The controller is in fact acting on a diluted one.

L1 · Bound None of this is new in v1.34.0. Comparing the same files at v1.30.0, the only differences are in how the tolerance is expressed. That release has one symmetric value. The later one has separate values for scaling up and down, with an alpha feature gate, off by default, that lets a single autoscaler override them. The ready-only ratio, the zero for unready pods, the grouping, the initialization rule, the stabilization maximum and the scale-up limit are identical at both tags.

What the Pull Requests Say It Was For

L1 · Bound The rule that unready pods count as zero arrived in 2016, in a change titled “HPA: Consider unready pods separately”[4] . Its description gives two reasons in its own words. Pods “frequently use extra CPU during initialization, so we want to consider them separately.” The effect is that “unready pods cause the autoscaler to be a bit more conservative” in “anticipation of the new pods later becoming ready and handling load.” The same description states the hold: a recomputation “cannot change the direction of the scale.”

L1 · Bound The rule about a full metric window came two years later, in two changes merged a week apart[5] [6] . The first describes the failure it fixed. Pods that burn CPU while starting trigger a scale-up. When they become ready, their start-up usage triggers another one, “even though existing pods can handle work, no problem.”

L2 · Fit Read together, these describe a controller designed to account for capacity it has already ordered. A pod that has been requested but is not yet serving is counted at zero usage, so the controller does not order it twice. That is a deliberate departure from the one-line rule. Neither description claims the design was meant to prevent instability under delay, and this post does not claim it either. The descriptions do say enough to explain why the instability of the next section never appears in the deployed controller.

What the Quoted Rule Does With Stale Data

Take the one-line rule at its word and feed it a replica count that is out of date. An engineer does exactly this when reasoning about the autoscaler from the formula. The thought runs: “utilization is at 100% on 10 pods, the target is 50%, so it will ask for 20.” A hand-written scaler that multiplies the replica count it last read by the latest ratio does the same in code.

Suppose the load is steady and needs exactly 10 pods at the target. The metric the rule divides by reflects the pods that were serving when it was measured, d sync periods ago. The multiplier is the current count. Write $N_t$ for the replicas after sync t and $u_t$ for the measured utilization:

$$N_{t+1} = N_t \cdot \frac{u_t}{u^\ast}, \qquad u_t = \frac{\lambda}{c\, N_{t-d}}$$

where:

• $\lambda$ - the arriving load, in requests per second
• $c$ - the rate one pod serves at full use
• $u^\ast$ - the target utilization
• $d$ - how many sync periods old the serving count behind the metric is

Taking logarithms turns the product into a sum with no approximation. Let $x_t = \ln N_t - \ln N^\ast$ be the deviation from the steady count $N^\ast = \lambda / (c\, u^\ast)$ . Then:

$$x_{t+1} = x_t - x_{t-d}$$

Proposition 1 -- Stale Ratio Rule: the one-line rule settles in one step with fresh data, oscillates forever at one stale period, and diverges at two or more

Proposition 1 (Stale Ratio Rule). The one-line ratio rule, applied to a replica count d sync periods old under steady load, settles in one step when d = 0. It oscillates with period 6 and constant amplitude when d = 1. It diverges for every d of 2 or more.

Proof. The characteristic equation of $x_{t+1} = x_t - x_{t-d}$ is

$$z^{d+1} - z^{d} + 1 = 0$$

At d = 0 the equation is $z = 0$ , so the deviation vanishes after one step. At d = 1 the roots are $e^{\pm i\pi/3}$ . Both lie on the unit circle and are distinct, so the deviation rotates with period 6 and never decays.

For d of 2 or more, the polynomial is monic with constant term 1, so the product of the moduli of its d + 1 roots is 1. A root on the unit circle must satisfy $|z| = 1$ and $|z - 1| = 1$ , which leaves only $z = e^{\pm i\pi/3}$ . The derivative $z^{d-1}\big((d+1)z - d\big)$ vanishes only at 0 and at $d/(d+1)$ , both inside the circle, so roots on the circle are simple. At most two of the roots lie on the circle. At least d - 1 of them are off it, and their moduli still multiply to 1. Since d - 1 is at least 1, at least one root lies strictly outside the unit circle, and the deviation grows geometrically. $\square$

L1 · Bound The general form of this equation carries a gain, $x_{t+1} = x_t - a\, x_{t-d}$ . Levin and May gave the condition for its equilibrium to be asymptotically stable[7] . Liz and two coauthors restate it as a bound on the gain that shrinks with the delay[8] :

$$a < 2\cos\!\left(\frac{d\,\pi}{2d+1}\right) = 2\sin\!\left(\frac{\pi}{2(2d+1)}\right)$$

The one-line rule has a gain of exactly 1. The bound is 2 at d = 0, 1 at d = 1, 0.618 at d = 2 and 0.445 at d = 3. The rule sits precisely on the edge at one stale period and beyond it at every longer delay, which is what the proof above shows directly.

L3 · Estimate With a load that needs 10 pods and a start at 8, the rule fed fresh data goes to 10 and stays there. One period stale, it runs 10, 12.5, 12.5, 10, 8, 8 and repeats forever. Two periods stale, it swings wider each cycle. After forty syncs the count is off by more than thirty orders of magnitude, a number with no physical meaning except that the loop has lost control.

L1 · Bound The same equation appeared in this blog before, as Proposition 22 of Self-Healing Without Connectivity. That proposition bounded the gain of a delayed healing loop by $K < 1/(1+d)$ . The bound is sufficient but not exact. At d = 1 it allows a gain of 0.5 where the exact bound allows 1. For the healing loop the conservative bound was the point, because a margin was wanted. Here the exact one matters, because the ratio rule sits on its edge. Delay in a feedback loop is a known source of oscillation in computing systems[9] . The one-line rule is a feedback loop with a gain of one, which leaves no room for any delay at all.

L2 · Fit The deployed controller does not fall into this trap. It never multiplies by a stale count. Each sync it recomputes from the usage of the ready pods and counts ordered pods at zero. In effect it computes the total demand the ready pods report and divides by the target, and the walked-through sync shows the effect. An engineer who reasons from the one-line rule therefore predicts an oscillation the controller does not have, and misses the weakness it does have.

L2 · Fit That substitution is the first lesson about how the loop is understood. The quoted rule is a model of the controller, and engineers use the model to reason about stability, tuning and incidents. Its errors are systematic. It sees a danger that the code removed in 2016, and it gives no hint of the price the code pays for removing it.

L2 · Fit Sufficient Abstraction and the Cost of Asking the Wrong Question Twice gave this kind of error its general form. An abstraction can be sufficient for one question and silent about another, and the information it discarded cannot be recovered by reasoning from the abstraction itself. The one-line rule is sufficient for “how many pods does this load need?” when the metric can see the load. It discarded the unready-pod rule, and with it every question about what the controller does while capacity is on its way. The only way back to that information is the raw source, which is why this post reads the code before it derives anything.

A Metric With a Ceiling

The weakness sits in the numerator. CPU utilization is usage divided by request, and usage cannot exceed what the pod is allowed to use.

Definition 1 -- Saturated Metric: a utilization signal whose value per pod cannot exceed the pod's allowance, so above that point it stops carrying the size of the load

Definition 1 (Saturated Metric). A utilization metric is saturated when every ready pod is using its full allowance. Its value per pod is then at its bound:

$$\frac{\text{usage}}{\text{request}} \le \frac{\ell}{r}$$

where:

• $\ell$ - the CPU limit of a pod
• $r$ - its CPU request, the denominator of utilization

While the metric is saturated, every arriving load above the pods’ combined capacity produces the same reading.

L1 · Bound With the limit equal to the request, a common setting for predictable scheduling, the ceiling is 100% utilization. A service carrying twice its capacity and a service carrying thirty times its capacity both read 100%. With the limit at twice the request, the ceiling is 200%, and the same collapse happens one level higher.

L2 · Fit CPU is the familiar case, but the ceiling belongs to the form of the metric. Any signal defined as a fraction of a pod’s own capacity has one. Connections in use over the pool size behave the same way, and so do worker threads busy over threads available, or memory in use over a container limit. Each measures how full a pod is. None can say how much more is waiting to get in, because the excess never reaches the pod to be counted. Every signal of this form saturates at the point where the autoscaler is needed most.

L2 · Fit Ashby’s law of requisite variety says a regulator can only counter as many distinct disturbances as its channel can distinguish[10] . A saturated utilization metric distinguishes every load below capacity and collapses every load above it into one value. The disturbances an autoscaler exists for are the ones larger than current capacity, and those are the ones its default channel cannot tell apart. This is an analogy, tagged as one, and the arithmetic below does not depend on it.

L1 · Bound The channel is also coarse below the ceiling. The calculation truncates utilization to a whole percentage before dividing, so at a 50% target the controller distinguishes about a hundred levels of load per pod up to capacity. Above capacity it distinguishes one. The truncation returns later in this post as a source of lost rounds.

The belief that the metric measures load holds below saturation. Above it, the metric reports only that the load is too large. To size the load the loop needs something else, and the controller can do just one thing: add pods and see whether the reading falls.

How Far One Recommendation Reaches

When the metric is saturated, the ratio the controller computes over its ready pods is fixed: the ceiling over the target. Call it the growth factor.

$$g = \frac{\ell}{r \cdot u^\ast}$$

L1 · Bound While pods are being added the new count is ceil(newRatio × podsInTheCalculation), with ordered pods at zero usage. That product equals the summed usage of the ready pods over the request and the target. It can never exceed g times the ready pods, rounded up to a whole pod, and the whole-percent truncation only lowers it. A hold can leave the count above that product, as the sweep in the walked-through sync shows, but a hold orders nothing. Ordering more pods does not raise it, because ordered pods contribute zero. The count rises again only when new pods become ready and start reporting usage of their own.

Definition 2 -- Round: the time from ordering a pod to the controller counting its usage, which is the provisioning delay plus one metric window

Definition 2 (Round). A round is the time from the sync at which a pod is ordered to the first sync at which the controller counts the pod as ready and uses its usage:

$$T_{\text{round}} \approx \tau + w_{\text{metric}}$$

where:

• $\tau$ - the dead time: from the order until the pod serves traffic, including scheduling, image pull and start-up
• $w_{\text{metric}}$ - the metric window reported with each CPU sample, because during the initialization period a pod’s sample is ignored until a full window has been collected since it became ready; the model uses 30 seconds

Alignment with the 15-second sync adds up to one sync to each round.

Proposition 2 -- At Most One Factor per Round: with a saturated metric, ready pods grow by at most g per round, so a k-fold overload needs at least ceil(log_g(k u* r / l)) rounds before capacity meets it

Proposition 2 (At Most One Factor per Round). Let a service run $N_0$ pods at target $u^\ast$ , and let the load step to k times its previous rate, with k large enough to saturate the metric. While the metric stays saturated, the controller multiplies its ready pods by at most g per round, up to rounding to a whole pod. Serving capacity therefore cannot meet the load in fewer than

$$R(k) = \left\lceil \log_g \frac{k\, u^\ast}{\ell / r} \right\rceil \quad \text{rounds}$$

Up to that rounding: whole pods replace $N_0$ and the needed count $N^\ast$ by $N_0 + \delta$ and $N^\ast + \delta$ , with $\delta = 1/(g-1)$ , inside the logarithm. Reaching capacity is not recovery. The backlog built during those rounds still has to drain, and the target utilization returns only at k times the original pods.

Proof. Before the step the pods carry $N_0 u^\ast$ pods’ worth of work at full use. After it they must carry $k N_0 u^\ast$ . Capacity at full use is $N \ell / r$ in the same units, so capacity meets the load only when $N \ge k N_0 u^\ast r/\ell$ . By the reach argument, no recommendation exceeds $\lceil gN \rceil < gN + 1$ for N ready pods, and the ready count changes only when ordered pods complete a round. Without rounding, after R rounds the ready count is at most $N_0 g^R$ , which reaches $N^\ast = k N_0 u^\ast r/\ell$ only for R at least the stated value. With rounding, $N + \delta$ grows by at most g per round, so $N_0 + \delta$ and $N^\ast + \delta$ take the place of $N_0$ and $N^\ast$ . In the specimen, with $N_0 = 10$ and g = 2, both forms give the same round count for every overload tested. $\square$

L2 · Fit The proposition is a bound, and the next question is how close the controller comes to it. The tolerance band sets the floor. The controller stops ordering only when the recomputed ratio over all pods falls to 1 + tolerance or below. So within a round, once the ready count settles, the ordered count reaches at least about $g/(1+\text{tol})$ times the ready pods. The truncation to a whole percent lowers that by up to one percentage point of utilization. Growth per round therefore lies between roughly g / 1.1 and g at the default tolerance.

L3 · Estimate The band does cost rounds in practice when the ready count rises in two steps within one round. Take a tenfold step at a 63% target. In the second round the controller sees 14 pods ready and orders 23, which is g times 14 rounded up. One sync later 16 pods are ready. The recomputed utilization over 23 pods is 69%, a ratio of 1.095 against the target, and that is inside the band. The controller holds at 23 where g times 16 would have been 26. The run needs five rounds and 585 seconds where Proposition 2 allows four.

L1 · Bound The scale-up limit of max(2 × current, 4) per sync is not what binds at moderate g. A round spans several syncs and the limit caps each one. What binds is that ordered pods count as zero, which caps the whole round at g. The controller’s design for anticipating capacity, the same feature that removed the instability of Proposition 1, is what sets the pace.

At a 50% target with the limit equal to the request, g is 2. Each round can at most double the ready pods, and a k-fold overload takes at least $\lceil \log_2 (k/2) \rceil$ rounds. Each round reveals at most one binary digit of the overload’s size.

Physical translation. A thermometer that stops at 100 degrees cannot tell a warm oven from a burning kitchen. A cook who needs to know which one it is has the controller’s options and no others: open the door a little, wait, and see whether the reading drops. Each wait costs a dead time, and each glance can at most double what the cook knows about the heat. The slowness belongs to the instrument, which tops out exactly where the measurement starts to matter.

Two Consequences of the Bound

Corollary 1 -- Dead Band: if the growth factor is inside the tolerance band, a saturated metric never triggers a scale-up at all

Corollary 1 (Dead Band). If the saturated reading divided by the target lies inside the tolerance band, so that

$$\frac{\lfloor 100\, \ell / r \rfloor}{100\, u^\ast} \le 1 + \text{tol}$$

then the controller holds its count however large the overload. With the limit equal to the request and the default tolerance, this happens at every target of 91% or above.

Proof. With every ready pod saturated and no pods on order, the first ratio is the left-hand side. When it lies inside the band, the calculation returns the current count. No pod is ordered, so the next sync sees the same inputs. $\square$

L3 · Estimate The port puts the edge between a 90% and a 91% target. Ten saturated pods draw a recommendation of 13 at an 80% target and 12 at every tested target from 85% to 90%. From 91% on they draw 10, the count they already have, and the controller stops. A 3x overload run at a 93% target never gets capacity at all, as the figure below shows.

L2 · Fit This is the extreme case of the same mechanism, and it is a configuration someone could choose for good reasons. A team that pays for every idle core might push the target toward 95% to cut cost. The cost setting works as intended while load stays below capacity. Above capacity the loop goes silent, and the status reports 100% utilization against a 95% target, close enough to look like a controller near its goal.

Corollary 2 -- Hottest Target: sizing a k-fold overload within R rounds requires a target no hotter than (l/r) k^(-1/(R+1))

Corollary 2 (Hottest Target). For a service to reach capacity under a k-fold overload within R rounds, its target must satisfy

$$u^\ast \le \frac{\ell}{r}\, k^{-1/(R+1)}$$

The condition is necessary, up to the rounding of Proposition 2, and not sufficient: at or below this target, the tolerance band or the whole-percent truncation can still cost a round.

Proof. Write a for $\ell / r$ . Proposition 2 requires $g^R \ge k u^\ast / a$ , with $g = a / u^\ast$ . Substituting gives $a^{R+1} \ge k\, (u^\ast)^{R+1}$ , which rearranges to the stated bound. $\square$

The model computes the hottest whole-percent target for each case and then runs the simulation at that target, with the limit equal to the request.

L3 · Estimate Most points land within their round budget. Three do not, and each one is instructive. At 3x and 93% the target falls in the dead band of Corollary 1, so the controller never scales. At 10x and 63% the tolerance band ends a round early, the case traced above. At 100x and 10% the controller’s order lands one pod short after truncation, and it needs a second round to make up for it. A target set at the edge of the bound is a target with no margin. The corollary tells an engineer where the edge is.

L2 · Fit The corollary turns the target into a design calculation. Start from the largest overload the service must survive and the time it may take, and divide that time by the round to get R. The hottest target that can meet the overload follows, with no simulation and no tuning. A target chosen as a cost setting, without that calculation, has fixed the recovery time as a side effect.

The Staircase

L3 · Estimate The running specimen makes the bound concrete. It has ten pods at a 50% target, a limit equal to the request, a 90-second dead time and a 30-second metric window, so a round costs about two minutes. The model runs the ported calculation every 15 seconds against a fluid queue.

L3 · Estimate Time to capacity grows with the logarithm of the overload. A hundredfold overload takes about twice as long to meet as a tenfold one, rather than ten times as long. At g = 2 the controller attains the bound at every overload in the figure, from one round at 3x to eight at 300x, and each time is the round count times two minutes, less up to one sync of alignment. That is the good news in the figure, and it is real: the time to meet an overload is gentle in the size of the shock.

L3 · Estimate The backlog is the bad news. The backlog is the work that arrived while capacity was short. It grows much faster than the time does, because each round lasts the same two minutes while the shortfall in each round grows with the overload. From 10x to 100x the time to capacity roughly doubles and the peak backlog grows about thirtyfold. A logarithmic number of rounds, each paid in full at the new rate, produces a backlog that grows faster than the overload itself.

The next figure shows what neither of these can: the three runs are indistinguishable until the smaller overload stops.

L3 · Estimate Each round shows two sub-steps, 16 then 20, and 32 then 40. They come from the 30-second averaging window. At the first sync after the step, part of the window still holds the old reading, so the first order is smaller. At the next sync the window is full and the walked-through calculation gives 20. Rounding, the averaging window and the sync alignment together account for every difference between the measured times and two minutes per round.

L3 · Estimate The figure repeats the times above and adds two things they cannot show. The first is the shared trunk: for 150 seconds the three runs are one line. The second is how far each run climbs past its dashed line. The 30x run meets its load once 159 pods are serving, above its need of 150, and has ordered 1,250 by 750 seconds, because pods ordered during the backlog keep arriving after the need has passed.

The Price of a Target

The target is usually chosen as a cost setting. Proposition 2 says it is also an information rate. The growth factor is the limit over the request over the target, so a hotter target means a smaller g and more rounds to size the same overload. A trade-off of that kind needs its achievable region stated before anything is chosen from it.

Definition 3 -- Target Achievable Region: each choice of target and limit is a point in steady-state pods against rounds to size an overload; the frontier is the set no other choice beats on both

Definition 3 (Target Achievable Region). For a design overload k, each choice of target $u^\ast$ and allowance $\ell / r$ maps to the point

$$\Big( \tfrac{1}{u^\ast},\; R(k) \Big)$$

Its first coordinate is the pods held per pod of steady demand. Its second is the rounds from Proposition 2. The achievable region is the set of all such points. A point is on the frontier if no other choice has both fewer pods and no more rounds.

For a thirtyfold overload in the running specimen, each choice of target and limit lands at one point of the region.

L3 · Estimate With the limit fixed at the request, every target is on the frontier, and the blue curve is steep at the cheap end. Moving from 50% to 80% saves three-eighths of the steady pods and costs eleven more rounds. At the default, a thirtyfold overload takes nearly half an hour to meet. That is the bill for reading the target only as a cost setting.

L3 · Estimate Once the limit can vary, the frontier changes shape. A 25% target with the limit at the request and a 50% target with the limit at twice the request have the same growth factor of 4. They take the same two rounds and the same 225 seconds. The first holds four pods per pod of demand and the second holds two. The 25% point is dominated. Headroom bought by lowering the target costs twice what the same information rate costs when bought through the limit.

L2 · Fit The limit does not come free. CPU above the request is borrowed from the node, which has it only if the scheduler left it free. The lever trades this loop’s information rate for headroom on the node. That trade belongs in the node’s capacity plan, where the risk of several services borrowing at once can be priced. A limit raised in one service’s configuration, with no such plan, moves the risk to whichever service is scheduled next to it.

L2 · Fit Read through Definition 3, the API default makes a choice on the user’s behalf. Eighty percent sits at the cheap end of the frontier: few idle pods, slow sizing. That suits a service whose load changes gradually, where the metric rarely saturates. It suits badly a service whose worst day is a step. Nothing in the default is wrong. It answers a question about cost, and the question about recovery time was never asked.

Scaling as Probing

L2 · Fit Dual Control and the Weaponized Probe took from Feldbaum the idea of an action that both regulates a system and tests the controller’s estimate of it[11] . A saturated autoscaler is such a controller without having been designed as one. Every scale-up regulates, by adding capacity. It also probes, by revealing whether capacity is now enough. The probe has a fixed resolution of one factor of g and a fixed price of one round of provisioning delay.

L2 · Fit This reading explains why the probe cannot be skipped. The controller has no other way to learn the size of an overload its metric cannot show. It also explains why a faster provisioning path helps more than any tuning of the controller’s policy. A shorter dead time leaves the amount learned per round unchanged and lowers what each round costs.

L3 · Estimate At g = 2, sizing a thirtyfold overload means learning that the load exceeds 2, 4, 8 and 16 times the starting capacity, in that order. That takes four rounds, four bits and about eight minutes in the specimen. At g = 1.25 each round learns about a third of a bit, since the base-2 logarithm of 1.25 is 0.32. That is why the default target needs fifteen rounds.

L2 · Fit A probe that multiplies also overshoots, by construction. The last round that crosses the need can land anywhere up to g times past it, because the controller cannot stop between factors. At g = 2 the round that crosses the need can order up to twice the capacity required. At g = 1.25 the overshoot from the probe itself is small, and the price is paid in rounds instead. The target sets an exchange rate between two errors: time spent short and capacity bought in excess.

L2 · Fit The information reading has a limit, and it should be stated. It describes what the controller can learn from its own signal. It says nothing about information the controller is never given. An operator who knows a launch is coming knows the overload’s size before the first round, and can set the replica count directly. The bound in this post applies to a loop left alone with a saturated metric, which is the loop on duty at three in the morning.

Overshoot, and Why the Bound Is Optimistic

L3 · Estimate Capacity meeting the load does not end the run. The backlog built during the rounds keeps every pod busy after capacity is sufficient, the metric stays saturated, and the controller keeps multiplying. In the tenfold run, capacity meets the load at 345 seconds, when the 64 pods ordered at 255 seconds begin serving. The count keeps climbing to 317 before the backlog drains. The overshoot ratio grows with the overload, from 1.33 at 3x to 6.46 at 300x.

L3 · Estimate The overshoot comes from the ceiling rather than from the controller’s arithmetic. Run the same steps with a limit a hundred times the request, so that the metric never saturates. The peak then equals the pods needed at every overload tested: 2x, 10x and 30x. The controller sees the load, computes the right count and stops there.

L2 · Fit Round arithmetic accounts for the size of the overshoot. While ready capacity is below demand or a backlog remains, the metric reads full and each round doubles the pods. Count the rounds until the backlog is gone and double that many times. The count gives 40 pods at 3x, exactly the simulated peak. At 10x it gives 320 against 317 simulated, at 30x 1,280 against 1,250, and at 300x 20,480 against 19,373. It never comes out below the simulated peak. It misses once, at 5x, where it predicts 160 and the run peaks at 80. The backlog left after the third round is small enough to clear before the next order, so the coarse count takes a doubling the run never does. The peak follows how long the backlog lasts, so anything that slows the drain raises it.

L3 · Estimate The figures so far assume a saturated pod still serves its full rate. Real pods under full load lose some of it to contention, memory pressure and context switching. The model can lower the efficiency of a saturated pod while its CPU still reads full:

L3 · Estimate Lower efficiency costs in two places. Capacity takes a round longer to meet the load, sometimes two, because each pod carries less. The backlog drains more slowly, so the metric stays saturated longer and the controller keeps ordering. At 40% efficiency a tenfold overload peaks at ten times the pods it needs. The CPU reading cannot tell a pod working at full efficiency from a pod thrashing, so the controller multiplies on both alike.

L3 · Estimate Proposition 2’s round count is therefore a lower bound on recovery, and the staircase’s overshoot is a lower bound on over-provisioning. A cluster whose pods degrade under load will fare worse on both counts than the staircase figures show. The fluid model also leaves out timeouts and retries, which add work during exactly the rounds when capacity is short. Every assumption the model makes points the same way: toward a faster, cleaner recovery than a real service gets.

L2 · Fit The limiting case is a service whose throughput per pod falls as pods are added. The Governance Tax described it past the peak of the scalability curve. Each added pod raises the CPU per pod, the controller reads that as under-capacity, and the cluster climbs to its configured maximum and stays there. The mechanism is the one in this section, with efficiency falling to the point where added capacity subtracts throughput. A ratio controller that cannot see efficiency has no way to tell that case apart from an ordinary overload.

Where This Sits Among Known Results

L1 · Bound None of the mechanism here is hidden. The handling of unready pods is documented. A common alternative is to scale on request rate, concurrency or queue depth instead of CPU. Knative’s autoscaler takes that route. Its default metric is concurrency, which counts requests in flight and waiting, so it does not saturate the way utilization does. It also enters a panic mode with a window one-tenth of its 60-second stable window. The documentation explains the default threshold: “panic mode will be started if traffic is twice as high as the current replica population can handle”[12] .

L1 · Bound The dead time itself is well documented outside Kubernetes. Gandhi and three coauthors measured a setup time of 260 seconds for an application server. They reported 30 seconds to a minute for a locally created virtual machine and 10 to 15 minutes for one obtained from a cloud provider, and observed that a reactive policy “lags in turning servers on” because of it[13] . The specimen’s 90 seconds sits at the short end of that range, so every time in this post is, if anything, optimistic.

L2 · Fit The proportional-regulator reading of Rolik and Volkov attributes over-regulation to the feedback loop, and their remedy changes the control law. This post locates the over-provisioning elsewhere: in a saturated signal, combined with a controller that anticipates ordered capacity and so cannot over-order within a round. The two readings are compatible. They point at different levers, and the levers have different costs.

L2 · Fit Three results were not found in the sources read for this post. The first is the round bound of Proposition 2 with its reading as an information rate. The second is the observation that the controller’s anticipation of ordered capacity is what caps each round. The third is the dead band and the hottest-target condition. They are offered as this post’s contribution, with a caution: absence from the sources read does not prove that nobody has written them down.

L2 · Fit An unbounded signal removes this post’s problem and introduces another. A signal that counts waiting work counts the backlog. The backlog is a quantity that integrates past shortfalls. How a loop driven by it behaves under the same provisioning delay is a different question, and this post does not answer it.

The Binding Constraint, in Sequence

L2 · Fit The Constraint Sequence Framework asks which constraint binds first and insists the rest wait their turn. For a saturated autoscaler the arithmetic fixes the order. The dead time binds first, because it sets the price of every round. The metric ceiling binds second, because it sets how much each round can learn. The tolerance band and truncation bind third, and only at the edge of a round budget. The scale-up limit per sync binds last, if at all, because a round spans several syncs.

L2 · Fit Tuning in the reverse order is common, and the sequence says why it disappoints. A scaling policy that allows more pods per sync changes the constraint that binds least. A lower target changes the second constraint at a steep price in idle pods. A faster image pull, a smaller start-up path or a warm pool changes the first one. It is also the only change that makes every round cheaper without making the steady state more expensive.

Falsification Criteria

A claim that cannot be wrong is not a claim. Each criterion below names the observation on a real cluster that would refute this post, with the prediction the model makes for it. The runs use a 16-core node, pods requesting and limited to 250 millicores, four pods at a 50% target before the step and a replica ceiling of 50. The predictions come from the same model, sized to that cluster.

F1 (the same first request).

• Condition: two load steps of different size, both large enough to saturate the metric: 3x and 5x.
• Prediction: both runs order 7 pods, then 8 within one sync, then 14 and 16 one round later.
• Refuted if: the orders differ between the two runs before the 3x run’s capacity meets its load.

F2 (at most one factor per round).

• Condition: the same two steps, timed from the step to the moment serving capacity meets the load.
• Prediction: 105 s for 3x and 225 s for 5x, one round apart, with peaks of 16 and 32 pods against 12 and 20 needed at the target.
• Refuted if: the ready count rises faster than twice per round, or the 5x run meets its load in the same round as the 3x run.

F3 (the ceiling causes the overshoot).

• Condition: the 5x step repeated with the CPU limit at 1,000 millicores, four times the request, so that the metric does not saturate.
• Prediction: the count rises to the 20 pods needed at the target and no further, and capacity meets the load at the step, because each pod can use four times its request.
• Refuted if: the overshoot persists with a metric that does not saturate.

F4 (the dead band).

• Condition: a 3x step with the target set to 93% and the limit at the request.
• Prediction: no scale-up for as long as the step lasts, with the status reporting 100% against 93%.
• Refuted if: the controller adds pods.

The Ledger of What the Loop Cannot Know

This series keeps one artifact across its four parts: a row per part, naming what the loop cannot know, the belief that hides it, the mechanism, the law and the bill.

Size

Belief: the metric measures load.

Mechanism: the controller computes from ready pods and counts ordered pods at zero, so a saturated reading caps each round at g times the ready pods.

Law: Proposition 2, at least $\lceil \log_g (k u^\ast r / \ell) \rceil$ rounds until capacity meets a k-fold load. Corollary 1, no scale-up at all when g lies inside the tolerance band. Corollary 2, the hottest target that can meet a k-fold load within R rounds. Layer 1 for the mechanism and the bounds; Layer 3 for the specimen’s times.

Bill: about two minutes per round in the specimen. Fifteen rounds for a thirtyfold overload at the API’s default target. Overshoot of 1.3 to 6.5 times the needed pods. All of it worse when saturated pods degrade.

What This Post Did Not Claim

Where Each Claim Holds

Claim. A saturated utilization metric lets the controller multiply ready pods by at most g per round.

• Assumption: the controller runs the v1.34.0 calculation on the default path, with no behavior block, and pods have equal requests.
• Failure mode: custom scale-up policies can cap a round below g, which makes the round count larger, never smaller.

Claim. A round costs the dead time plus one metric window.

• Assumption: pods report ready when they start serving, the metric window is 30 seconds, and the cluster has free node capacity for every pod ordered.
• Failure mode: a readiness probe that passes before a pod can serve shortens the counted round and lets the controller count capacity that does not yet exist.
• Failure mode: when the nodes are full, new pods wait for new nodes. The dead time then grows with the size of the order, in steps of one node’s worth of pods, and every round is longer than the specimen’s.

Claim. At targets of 91% and above, with the limit at the request, a saturated metric never triggers a scale-up.

• Assumption: the default tolerance of 10%, symmetric.
• Failure mode: a per-autoscaler tolerance, behind the HPAConfigurableTolerance feature gate (alpha and off by default at v1.34.0), moves the edge of the dead band to $1/(1+\text{tol})$ .

Claim. The staircase and overshoot numbers describe the specimen.

• Assumption: a fluid queue, a 90-second dead time, no timeouts or retries, and pods serving their full rate when saturated.
• Failure mode: every one of these is optimistic. Timeouts, retries and degradation make recovery slower and overshoot larger.

Reversal condition. The advice of this post is to read the target as an information rate and to spend effort on the dead time first. It reverses when the metric cannot saturate in the overloads that matter. That happens with a limit far above the request and node headroom to match, or with a signal that counts waiting work. The controller then sizes an overload in one step, and the costs that matter come from somewhere else.

What Is Not Claimed

Every number in this post comes from a model of one specimen, run against a port of the controller’s arithmetic. The staircase, the overshoot ratios, the hottest-target sweep and the dead band’s edge make the bounds checkable in one concrete case. Change the dead time, the metric window or the efficiency of a saturated pod and the numbers move. The bound of Proposition 2 and the conditions of its two corollaries do not.

It has not been claimed that the Kubernetes autoscaler is badly designed. Counting ordered pods at zero removes a real instability, although the pull requests that introduced it gave other reasons, and the slow learning this post prices is the cost of that choice rather than a defect in it. Nor has it been claimed that CPU is the wrong metric for every service. Below saturation it measures load well, and a service whose worst day arrives gradually may never meet the bound at all.

Two open questions are named on purpose. The reading of scaling as probing, one bit per round at a growth factor of two, is an interpretation and is tagged as one. And how close the controller comes to the bound when the growth factor sits near the tolerance band is shown here only in simulation; the band can cost a round, and this post has not derived how often it does.

One correction is made on purpose. Proposition 22 of Self-Healing Without Connectivity bounded the gain of a delayed loop by a condition that is sufficient but not exact. That was the right choice for a healing loop that wanted a margin, and the exact bound is used here only because the one-line rule sits precisely on its edge.

What Is Left Open

The deployed controller avoids the instability that the one-line rule predicts. It pays for that with a slow, fixed rate of learning when its metric saturates. Two ways out suggest themselves, and both lead to new questions.

A signal that does not saturate sees the size of an overload. A signal that counts waiting work, though, is driven by a backlog, and a backlog responds to delay in its own way. A signal that counts only what clients ask for sees demand cleanly. Demand, however, is not all the work a struggling service has to do. Neither question is answered here.

A third question sits underneath both. The bound of this post assumes the controller is the only thing deciding how much work the service accepts. During every round the service is short of capacity, and the excess goes somewhere: into a queue, into timeouts, into retries. This post priced the rounds and left the excess to the fluid queue, which holds everything and loses nothing. Real services lose things.

Read the Cognitive Map as plain text

The Rule as It Runs

1. The documentation opens with $\lceil \text{currentReplicas} \times \text{current} / \text{desired} \rceil$ and then describes, in prose, how unready pods and missing metrics are set aside. The calculation as run divides the usage of ready pods by the target and counts ordered pods at zero.
2. Fed a replica count one sync period old, the one-line rule oscillates forever with period 6; two or more periods old, it diverges. In logarithms it is $x_{t+1} = x_t - x_{t-d}$ , and its gain of 1 sits on the Levin-May stability bound at one period and beyond it at two.
3. The 2016 change that counts unready pods at zero gives its purpose as start-up CPU noise and anticipation of new pods becoming ready. That design removes the instability of the quoted rule.

A Metric With a Ceiling

4. Utilization is usage over request, and usage cannot exceed the limit. With the limit equal to the request, every load above capacity reads 100%.
5. While saturated, the controller can ask for at most $g = \ell / (r\, u^\ast)$ times the ready pods, because ordered pods count as zero. A k-fold overload therefore needs at least $\lceil \log_g (k\, u^\ast r / \ell) \rceil$ rounds, a round being the dead time plus one metric window.
6. The tolerance band ends some rounds early, so growth per round lies between about $g / 1.1$ and g. If g itself is inside the band, at targets of 91% and above with the limit at the request, the controller never scales.

The Bill

7. A hotter target saves pods and costs rounds: for a thirtyfold overload, 4 rounds at 50% and 15 at the API default of 80%. Meeting a k-fold load within R rounds requires a target no hotter than $(\ell / r)\, k^{-1/(R+1)}$ .
8. Each scale-up both adds capacity and reveals whether it is enough. At g = 2 each round learns at most one binary digit of the overload’s size; a shorter dead time makes every round cheaper.
9. The backlog keeps pods saturated after capacity suffices, so the loop overshoots: 3.17 times the needed pods at 10x. With pods that degrade under load, recovery is slower and overshoot larger, so the round count is a lower bound.

Compute it. Take your autoscaler’s target and your pods’ limit over request, and compute g. If g is 1.1 or less, the loop cannot scale on a saturated metric at all. Otherwise take the largest overload you plan for and compute the rounds, then multiply by your dead time plus your metric window. If the answer is longer than your users will wait, the fix lies in the dead time or the signal. The controller’s settings cannot supply it.

The controller learns the size of an overload the only way its metric allows: by paying for capacity and reading what comes back. At a 50% target it learns at most one bit per dead time. At the default, about a third of one.

---

^[1] The Kubernetes Authors (2025). kubernetes/kubernetes v1.34.0. pkg/apis/autoscaling/annotations.go; pkg/controller/podautoscaler/replica_calculator.go, horizontal.go and metrics/utilization.go. Released 27 August 2025.

^[2] The Kubernetes Authors (2026). Horizontal Pod Autoscaling, section “Algorithm details.” kubernetes.io documentation (kubernetes/website, main branch).

^[3] Rolik, O. & Volkov, V. (2024). Method of horizontal pod scaling in Kubernetes to omit overregulation. Information, Computing and Intelligent Systems, No. 5, 55-67. doi:10.20535/2786-8729.5.2024.315877.

^[4] kubernetes/kubernetes pull request #33593 (2016). HPA: Consider unready pods separately.

^[5] kubernetes/kubernetes pull request #67252 (2018). Fix HPA sample sanitization.

^[6] kubernetes/kubernetes pull request #68068 (2018). Change CPU sample sanitization in HPA.

^[7] Levin, S.A. & May, R.M. (1976). A note on difference-delay equations. Theoretical Population Biology 9, 178-187.

^[8] Liz, E., Tkachenko, V. & Trofimchuk, S. (2006). Global stability in discrete population models with delayed-density dependence. Mathematical Biosciences 199, 26-37.

^[9] Hellerstein, J.L., Diao, Y., Parekh, S. & Tilbury, D.M. (2004). Feedback Control of Computing Systems. Wiley-IEEE Press.

^[10] Ashby, W.R. (1956). An Introduction to Cybernetics. Chapman and Hall (Chapter 11, The Law of Requisite Variety).

^[11] Feldbaum, A.A. (1960-61). Theory of Dual Control, I-IV. Avtomatika i Telemekhanika.

^[12] Knative Authors (2026). Configuring the Knative Pod Autoscaler and Autoscaling metrics. knative.dev documentation (knative/docs, main branch).

^[13] Gandhi, A., Harchol-Balter, M., Raghunathan, R. & Kozuch, M.A. (2012). AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers. ACM Transactions on Computer Systems 30(4), Article 14.

---

Back to top