Kubernetes
November 18, 2026 • 8 min read

Every autoscaler is right about its own layer

A cluster can run the Vertical Pod Autoscaler (VPA), the Horizontal Pod Autoscaler (HPA), KEDA and Karpenter at the same time. Each one makes correct decisions for the layer it controls, and the cluster can still pay for capacity it does not use and fail when load rises. The cause is that each autoscaler reads its own signals and acts on its own objects, and none of them sees the combined effect of all of them. Reducing cost while assuring stability requires a view across every scaling layer, and this post explains why.

Every autoscaler is right about its own layer
Anais Dotis
Anais Dotis
November 18, 2026
8 min read
Kubernetes

A cluster can run the Vertical Pod Autoscaler (VPA), the Horizontal Pod Autoscaler (HPA), KEDA and Karpenter at the same time. Each one makes correct decisions for the layer it controls, and the cluster can still pay for capacity it does not use and fail when load rises. The cause is that each autoscaler reads its own signals and acts on its own objects, and none of them sees the combined effect of all of them. Reducing cost while assuring stability requires a view across every scaling layer, and this post explains why.

This post uses three examples:

  1. A checkout service with an HPA that targets 70% CPU utilization, and it shows how a vertical change triggers a horizontal one.
  2. A node pool whose free capacity is split across many nodes, and it shows what fragmentation is and what it costs.
  3. A nightly Spark job named nightly-orders-etl, and it shows a workload that no scaling layer recognizes as a single workload. After defining the three scaling layers, the post covers how they interact, fragmentation, the invisible workload, what experienced teams do today and how groundcover's Autonomous Infrastructure approaches the problem.

The three scaling layers

Kubernetes scales workloads at three layers, and each layer has its own autoscalers:

  • Vertical scaling changes the CPU and memory requests of existing Pods. VPA recommends requests from historical usage and, with in-place Pod resize, can apply them without restarting Pods, as the previous post described.
  • Horizontal scaling changes the number of replicas. HPA adds or removes replicas to keep a metric near a target value, most often CPU utilization, which it calculates as usage divided by request. KEDA (Kubernetes Event-driven Autoscaling) scales on external event sources such as queue length, and it creates and manages an HPA through a resource called a ScaledObject.
  • Cluster scaling changes the number and type of nodes. Cluster Autoscaler adds nodes when Pods cannot be scheduled and removes nodes whose Pods can be placed elsewhere. Karpenter provisions nodes sized to pending Pods and consolidates the cluster by removing or replacing nodes.

Each autoscaler reads its own inputs and changes its own objects: VPA changes Pod requests, HPA changes replica counts, and Karpenter changes nodes. Their inputs overlap, however, and that overlap is where the interactions between layers begin.

How the layers interact

A vertical change can trigger a horizontal one, because HPA calculates utilization relative to the request. Suppose the checkout service runs four replicas, each using 0.6 cores and requesting 1 core. HPA reads 60% utilization against its 70% target and keeps four replicas, with 4 cores requested in total. If a sizing tool lowers each request to 0.5 cores, the same usage reads as 120% utilization, and HPA scales out to seven replicas to bring average utilization back under 70%. Requested CPU ends at 3.5 cores instead of 2, so most of the saving moves from the vertical layer to the horizontal layer instead of being realized. This is why the VPA documentation advises against running VPA and HPA on the same CPU or memory metric, but the interaction doesn’t stop at those two autoscalers.

Each layer also reacts to the output of the others. VPA changes a request, HPA responds to the new utilization, and Karpenter adds or removes nodes to fit the new replica count. The next VPA recommendation is then based on usage that the HPA and Karpenter changes already shaped. The Wand team, now part of groundcover, describes this pattern as adding reactive layers that each compensate for the blind spots of the previous one. The individual layers are not wrong, but they lack a shared view of the cluster, and fragmentation is one measurable result of that missing view.

Fragmentation

Resource fragmentation is free capacity that is split across nodes in pieces too small, or of the wrong shape, for the Pods waiting to be scheduled. Consider a node pool of 10 nodes, each with 1.5 CPU unreserved. The pool has 15 CPU free in total, but a Pod that requests 2 CPU fits on none of the nodes, so it stays pending and the cluster autoscaler adds an eleventh node. The cluster now pays for a new node while 15 CPU of capacity on its existing nodes remains unreserved.

Fragmentation also has a shape dimension, because CPU and memory are reserved independently. A node with 3 CPU and 1Gi of memory free cannot place a Pod that requests 1 CPU and 4Gi, and a node with the reverse profile cannot place a CPU-heavy Pod. Each of these nodes holds unreserved capacity that no pending Pod can use, which is often called stranded capacity. In both cases, the cost of fragmentation is new nodes added while the existing nodes hold enough free capacity in total.

Several common conditions increase fragmentation. Padded requests reserve capacity that is never used, frequent replica changes leave gaps of different sizes as Pods start and stop, and placement that ignores the remaining shape of each node leaves mismatched capacity. Consolidation by Karpenter or Cluster Autoscaler reduces fragmentation periodically, but it operates on requests, so it cannot correct requests that do not reflect usage.

What experienced teams already do

Experienced platform teams already reduce these interactions manually. They run HPA on custom metrics, such as requests per second or queue depth, instead of CPU utilization, so that request changes do not distort the scaling signal. They keep VPA in Off mode for workloads that HPA manages and apply its recommendations by hand, but only after working out how each change would shift the HPA's utilization calculation. A VPA recommendation applied as-is to a resource that HPA scales on triggers the same replica increase shown in the checkout example, so the recommendation is only a starting point. They design node pools around workload shape, separating memory-heavy and CPU-heavy services to limit stranded capacity, and they tune Spark configuration after reviewing past runs. These practices work, but different people maintain each one separately, and none of them reconciles the decisions of the three layers. Reconciling those decisions continuously requires a view of the whole cluster.

How groundcover approaches it

groundcover's Autonomous Infrastructure evaluates requests, replicas and nodes together, so it can reduce cost without moving that cost to another layer and without weakening stability. When an HPA or KEDA utilization target drives a resource on a workload, Autonomous Infrastructure leaves that resource unchanged and sizes only the other resource. Its changes therefore do not trigger the replica increase shown in the checkout example. Workloads that VPA manages in Recreate mode are excluded, so the two systems never act on the same Pods.

Requests that reflect real usage also give the cluster layer accurate inputs. When Karpenter or Cluster Autoscaler consolidates nodes based on requests that match how workloads actually behave, it can remove nodes that are not needed while keeping the headroom that overlapping peaks require. Node-level sizing reduces the padded requests that create stranded capacity, which produces the cost savings. It also preserves headroom for workloads that peak together, which assures stability.

groundcover also groups Pods into workloads across controllers, and the groundcover agent keeps workload labels and groupings consistent as Helm charts and naming conventions change. The first release of Autonomous Infrastructure manages Deployments, and the Spark example in this post illustrates the problem rather than a supported workload type. groundcover’s sensor collects the underlying data automatically: it runs as a DaemonSet on every node and uses eBPF to read resource usage for every container directly from the Linux kernel, at high resolution and without application code changes or exporters, so even Pods that exist for only a few minutes, such as Spark executors, are captured. With BYOC, the per-container history needed to see patterns across runs, nodes and layers stays in the customer's own cloud account, where retaining all of it is affordable.

Next: measuring which workloads are waiting

Each scaling layer makes accurate decisions within its own scope, and the cost and instability come from the interactions between layers. Vertical changes trigger horizontal ones, cluster decisions rest on inaccurate requests, free capacity fragments, and some workloads are invisible to every layer. Autonomous Infrastructure addresses those interactions with one view across requests, replicas and nodes, so that cost savings do not come at the expense of stability. All of these decisions, however, rely on utilization and request data, which show how busy a node is but not which workloads are waiting for CPU, memory or I/O. The next post covers cgroup v2 Pressure Stall Information (PSI) metrics, which measure that waiting directly.

Anais Dotis
Anais Dotis
 

8 min read |
Published on: Nov 18, 2026

Latest posts

Explore related posts

Sign up for Updates

Keep up with all things cloud-native observability.

We care about data. Check out our privacy policy.

No items found.
No items found.
No items found.
No items found.
No items found.
No items found.
No items found.