Free cookie consent management tool by TermsFeed Generator

Series: Capacity in Arrears: Autoscaling Under Dead Time, Saturated Metrics, and Retry Debt

An autoscaler adds capacity for the load it can measure, and an overload is the load it cannot.
A utilization metric is bounded by the capacity it reports on, so it reads the same at twice the load and at thirty times. This series follows one widely deployed autoscaler from its source code to its limits, and uses it to test four things engineers reasonably believe about a control loop: that its metric measures load, that a stabilization window makes it stable, that a cleaner signal is a safer one, and that a limit removes a risk. Each part prices what one belief leaves out. The results come from the controller's own arithmetic and from simulations whose assumptions each part states. Every part names the observation, on a real cluster, that would prove it wrong.

4 posts in this series

  1. 1. One Bit per Dead Time

    A CPU metric cannot report more load than the pods can carry, so a twofold overload and a thirtyfold one draw the same first request for capacity. This post reads the Kubernetes autoscaler's source to see what it computes, then derives what follows: it sizes an overload at most one multiplication per provisioning delay. At a 50% target that is one bit per dead time. At the API's default of 80%, a thirtyfold overload takes fifteen rounds.

    04 October 2026

  2. 2. The Delay a Window Demands

    Scale a consumer on its backlog with the default five-minute stabilization window, and it can swing between a handful of pods and its ceiling while demand stays flat. This post follows a queue through KEDA into the Kubernetes autoscaler and finds the condition for rest: a standing delay of about one provisioning dead time. At the Kafka default of ten messages per pod, a consumer handling ten messages a second carries one second of it.

    11 October 2026

  3. 3. Demand Is Not Debt

    A service with capacity at twice its demand can still serve almost nothing. Timeouts and retries leave work that no client is waiting for, and that debt needs capacity the demand figure never mentions. This post states when the failed state, once entered, becomes permanent, and shows that an autoscaler fed a clean demand signal can hold a service exactly there. With two retries, any target of a third or more is enough.

    18 October 2026

  4. 4. The Barrier the Scaler Cannot Hold

    During the provisioning delay an autoscaler has ordered capacity and controls nothing. This post prices the three ways to survive that interval: run colder, make clients retry less, or refuse work at the door. Under heavy-tailed demand the third bounds the backlog and moves its tail into rejected requests. Time can be bought, and buying it is what shrinks the excess. A limit can only move it.

    25 October 2026

← Back to all posts