Kubernetes Limits: Mastering CPU and Memory Constraints

The scheduler will then try to ensure that that minimum amount is available by scheduling the Pod on a node with the requisite resource availability. Pod requests are compared to the available resources (such as available CPU and memory) on each node, as indicated by the node's .status.allocatable field. If a Pod's requested resources exceed the available resources on a node, the Pod will remain in a Pending state until resources become available.

Resource settings (requests and limits) directly influence scheduling decisions and node utilization. Monitoring node utilization is important for efficient resource management, and tools like the metrics server can provide real-time resource usage data to inform resource allocation decisions. Kubernetes also allows for resource overcommitment, where requests can be set lower than actual usage to maximize node utilization, but this must be managed carefully to avoid scheduling issues.

The problem with requests, though, is that they’re not always enough on their own to manage resource allocations properly. Whether requests solve your resource-management woes depends – as does everything – your workload requirements and on continuously tracking key Kubernetes metrics for visibility and control.

To illustrate the point, let’s look at two common ways of leveraging requests and/or limits to improve resource allocation in Kubernetes, ideally guided by robust Kubernetes application performance monitoring practices.

Playing it safe: setting high requests

As a rule of thumb, memory limits are encouraged. Defining high requests usually safeguards the node while not sacrificing app performance. Setting appropriate limits requests is essential for balancing safety and efficiency, as they control CPU and memory allocation, affect scheduling, and help prevent resource overuse to maintain cluster stability. Monitoring memory resource usage is crucial to ensure that requests and limits are set appropriately and to avoid resource exhaustion. The caveat of this approach is that applications that don’t actually utilize the memory they request can leave other workloads under-provisioned because there aren’t enough resources left over for them. This leads to an underutilized node, which can increase expenses, especially when node autoscaling is implemented, making strong Kubernetes observability practices and tooling critical to detect waste. Additionally, when troubleshooting memory issues or OOM kills, always check for memory leaks in your application, as they can cause persistent memory problems and impact stability.

Set low requests, and set limits… only for spiky applications

In the case of a spiky workload, with a varying resource consumption, it’s possible to create a margin between the requests and the limits, while relying on Kubernetes health checks and probes to ensure Pods remain healthy as they scale resource usage up and down. Containers can use more CPU than their request if additional resources are available on the same node, but they cannot exceed their CPU limit due to kernel enforcement. Similarly, ephemeral storage is another resource that can be limited and should be monitored for spiky workloads to prevent unexpected evictions or failures.

The result is a cluster where nodes are less likely to go underutilized and Pods are less likely to have to wait to be assigned because your requests are not too high. At the same time, however, the limits serve as a protection against those workloads that may become especially disruptive to other containers.

Kubernetes supports three Quality of Service (QoS) classes: Guaranteed, Burstable, and BestEffort. Burstable is the most common class, allowing pods to burst above their requests if additional resources are available, but they will be evicted before Guaranteed pods under resource pressure. BestEffort pods have no resource guarantees and are the first to be evicted, making them suitable only for non-critical batch jobs that can be interrupted.

However, creating a large margin between request and limits can lead to a very nasty problem called memory overcommit. In this case, workloads lie to the node about their minimum requirements, but once assigned to it, they completely drain it out of resources - quickly leading to unpredictable instabilities and crashes. This is the classic scenario where a kid at six flags wants to go on the rollercoaster, but doesn’t reach the minimum height requirement so he stands on his parent’s feet to fake the extra two inches and gets in. 

Quality of Service (QoS) Classes: How Kubernetes Prioritizes Pods

Kubernetes uses Quality of Service (QoS) classes to determine how pods are prioritized for resource allocation, especially when nodes experience resource pressure. These classes—Guaranteed, Burstable, and BestEffort—are assigned based on the resource requests and limits specified in your pod definitions.

Guaranteed pods are given the highest priority. To achieve this class, you must set both CPU and memory requests and limits to the same value for all containers in the pod. This ensures that the pod receives exactly the amount of cpu and memory resources it requests, making it ideal for critical workloads that require consistent resource utilization and cannot tolerate performance fluctuations.

Burstable pods are assigned when at least one container in the pod has a memory or CPU request that is less than its limit. This allows the pod to use more resources than requested if available, but it may be throttled or evicted if the node runs low on resources. Burstable pods are a good fit for applications with variable resource usage patterns, where occasional spikes in cpu and memory requests are expected but not guaranteed.

BestEffort pods have no resource requests or limits set. These pods are the lowest priority and are the first to be evicted when the node runs out of resources. BestEffort is suitable only for non-critical workloads where resource guarantees are not necessary.

Understanding and leveraging QoS classes is essential for effective resource management in Kubernetes. By carefully setting resource requests and limits, you can ensure that your most important workloads are protected and that overall resource utilization across your cluster remains balanced.

Namespace ResourceQuota and LimitRange: Enforcing Boundaries in Multi-Tenant Clusters

In multi-tenant Kubernetes environments, it’s crucial to prevent any single team or application from monopolizing cluster resources. This is where Namespace ResourceQuota and LimitRange come into play, providing guardrails for resource allocation and usage.

ResourceQuota allows administrators to set hard resource limits—such as total CPU, memory, or the number of pods—within a namespace. This ensures that the sum of all resource requests and limits in that namespace cannot exceed the defined quota, preventing resource hogging and promoting fair access to shared resources. For example, if a namespace has a memory quota of 8Gi, all memory requests and limits for pods in that namespace must stay within this boundary.

LimitRange complements ResourceQuota by defining default and maximum resource requests and limits for containers within a namespace. This means that if a developer forgets to specify resource requests or limits in their container specification, Kubernetes will automatically apply the defaults set by the LimitRange. It also prevents users from requesting excessive resources for a single container, which could otherwise lead to performance degradation for other workloads.

By combining ResourceQuota and LimitRange, cluster administrators can enforce consistent resource requests and limits, reduce the risk of overcommitment, and maintain stable performance across all tenants. This approach is essential for maintaining a healthy, multi-tenant Kubernetes cluster where resources are shared equitably and efficiently.

Avoiding OOM Kills: Protecting Your Workloads from Memory Surprises

One of the most disruptive events in Kubernetes is an Out-of-Memory (OOM) kill, which occurs when a container exceeds its memory limit and the Linux kernel is forced to terminate it. OOM kills can lead to application downtime, data loss, and poor performance, especially if they affect critical workloads.

To minimize the risk of OOM kills, it’s vital to set accurate memory requests and memory limits for each container. By aligning memory requests with memory limits, you ensure that the Kubernetes scheduler only places pods on nodes with enough available memory, and that containers cannot consume more memory than allocated. This approach helps maintain predictable memory usage and prevents a single container from causing memory pressure that could impact other workloads.

Regularly monitoring memory usage with Kubernetes-native monitoring tools or third-party solutions allows you to spot trends and adjust resource requests and limits as your application’s actual usage patterns evolve. If you notice that a container frequently approaches its memory limit, consider increasing its memory requests and limits to provide more headroom and avoid unexpected OOM kills.

Effective memory management is about balancing the need for more memory with the risk of overcommitting resources. By setting appropriate resource limits and requests, and by keeping a close eye on memory usage, you can protect your workloads from memory surprises and ensure reliable, stable operation across your Kubernetes cluster.

What about autoscaling?

Another creative way to try to leverage limits while minimizing the risk of running out of resources is to configure horizontal or vertical pod autoscaling for your clusters or even use kubectl scale deployment to manually adjust replicas. Although each type of autoscaling works differently, they both serve the purpose of automatically making more resources available to your nodes (and, by extension, to your Pods) in situations where total cluster resources run low.

In addition, the vertical pod autoscaler (VPA) can analyze historical resource usage and provide recommendations for resource requests, which you can review and apply manually for safe sizing guidance without automatically changing your workloads unless configured to do so.

In theory, then, autoscaling can function against inadequate limit settings. The problem with this idea in practice, though, is that horizontal and vertical pod autoscalers “don't act instantly - certainly not on the timescale of seconds.” Your app may already be dropping requests by the time autoscaling rides in to save the day.

This doesn’t mean you shouldn’t use autoscaling. You certainly should in situations where it makes sense. But it is to say that you shouldn’t assume auto scaling has your back in the event that limit settings turn out to be insufficient; you still need sound Kubernetes deployment strategies for safe rollouts.

The bottom line: when it comes to K8s limits, know your app

Whichever approach you take, what matters most is knowing your application. You can’t determine which requests or limits to set unless you know what the consumption patterns of your application are likely to be. This requires tracking workload behavior over time, learning which resources the workload consumes during normal and peak activity, and then rightsizing requests and/or limits based on the actual needs of your applications. Profiling each application individually ensures that resource allocation is efficient and cost-effective. Additionally, Kubernetes allows for the overcommitment of resources, meaning you can set requests lower than actual usage to maximize node utilization, while limits provide a safety mechanism to prevent any single application from monopolizing node resources. Regularly evaluating and adjusting these settings is key, as well as configuring resilient Kubernetes liveness probes to restart unhealthy containers.

At the end of the day, Kubernetes is only as effective as the configurations you apply to it. It’s not an observability tool, so it can’t automatically determine which resources your apps need to run reliably. That task is on you, and it requires you to glean application insights with the help of a robust observability platform which will help you understand the resource consumption behavior of workloads in Kubernetes and beyond, including complex Kubernetes on‑premises environments with unique scaling constraints - so that you can make informed decisions about limit and request settings.