Kubernetes Limits: Mastering CPU and Memory Constraints

The scheduler will then try to ensure that that minimum amount is available by scheduling the Pod on a node with the requisite resource availability.

The problem with requests, though, is that they’re not always enough on their own to manage resource allocations properly. Whether requests solve your resource-management woes depends – as does everything – your workload requirements.

To illustrate the point, let’s look at two common ways of leveraging requests and/or limits to improve resource allocation in Kubernetes.

Playing it safe: setting high requests

As a rule of thumb memory limits are encouraged. Defining high requests usually safeguards the node while not sacrificing app performance. This is a wise preventative measure since it signals to the server that this application will utilize a lot of memory before it actually starts consuming it. The caveat of this approach is that applications that don’t actually utilize the memory they request while other workloads remain under-provisioned because there aren’t enough resources left over for them. This leads to an underutilized node which can increase expenses, especially when node autoscaling is implemented. 

Set low requests, and set limits… only for spiky applications

In the case of a spiky workload, with a varying resource consumption, it’s possible to create a margin between the requests and the limits. 

The result is a cluster where nodes are less likely to go underutilized and Pods are less likely to have to wait to be assigned because your requests are not too high. At the same time, however, the limits serve as a protection against those workloads that may become especially disruptive to other containers.

However, creating a large margin between request and limits can lead to a very nasty problem called memory overcommit. In this case, workloads lie to the node about their minimum requirements, but once assigned to it, they completely drain it out of resources - quickly leading to unpredictable instabilities and crashes. This is the classic scenario where a kid at six flags wants to go on the rollercoaster, but doesn’t reach the minimum height requirement so he stands on his parent’s feet to fake the extra two inches and gets in. 

What about autoscaling?

Another creative way to try to leverage limits while minimizing the risk of running out of resources is to configure horizontal or vertical pod autoscaling for your clusters. Although each type of autoscaling works differently, they both serve the purpose of automatically making more resources available to your nodes (and, by extension, to your Pods) in situations where total cluster resources run low.

In theory, then, autoscaling can function against inadequate limit settings. The problem with this idea in practice, though, is that horizontal and vertical pod autoscalers “don't act instantly - certainly not on the timescale of seconds.” Your app may already be dropping requests by the time autoscaling rides in to save the day.

This doesn’t mean you shouldn’t use autoscaling. You certainly should in situations where it makes sense. But it is to say that you shouldn’t assume auto scaling has your back in the event that limit settings turn out to be insufficient. 

The bottom line: when it comes to K8s limits, know your app

Whichever approach you take, what matters most is knowing your application. You can’t determine which requests or limits to set unless you know what the consumption patterns of your application are likely to be. This requires tracking workload behavior over time, learning which resources the workload consumes during normal and peak activity and then setting requests and/or limits accordingly.

At the end of the day, Kubernetes is only as effective as the configurations you apply to it. It’s not an observability tool, so it can’t automatically determine which resources your apps need to run reliably. That task is on you, and it requires you to glean application insights with the help of a robust observability platform which will help you understand the resource consumption behavior of workloads in Kubernetes and beyond - so that you can make informed decisions about limit and request settings.