Privilege Escalation in Kubernetes: Risks, Detection & Prevention

In theory, Kubernetes clusters are like apartment buildings where every tenant (or workload) lives separately, having keys only to their own apartment.

But in practice, various types of mistakes can lead to a problem known as privilege escalation in Kubernetes. When this happens, Pods, service accounts, and other resources gain permissions that they shouldn’t have. They cross the boundaries that are supposed to keep the components of Kubernetes clusters neatly segmented from each other.

For Kubernetes admins, knowing how to detect, prevent, and mitigate privilege escalation risks is critical for enforcing effective security, as well as ensuring the efficient use of cluster resources. Read on for guidance as we dive deep into what privilege escalation in Kubernetes means, what causes it, how to detect it, and best practices for managing privilege escalation risks.

What is privilege escalation in Kubernetes?

In Kubernetes, privilege escalation is any type of incident in which a Kubernetes resource (such as a Pod or service account) gains permissions that admins did not intend for it to have.

Privilege escalation typically occurs when a configuration issue creates a “loophole” that a resource can exploit to bypass or elevate the permissions assigned to it. We’ll take a look at specific causes of privilege escalation in a bit.

Why privilege escalation in Kubernetes is a critical security risk

First, let’s talk about why privilege escalation presents a major problem from a security standpoint in Kubernetes.

The root of the issue is straightforward: When a Kubernetes resource can escalate its privileges, it gains the ability to interfere with other resources or workloads. This can lead to three specific types of problems:

• The disruption of other workloads’ operations: Privilege escalation could potentially allow a rogue workload to stop other Pods or services from operating normally, either by depriving them of the resources they need or shutting them down completely.
• Exfiltration or manipulation of sensitive data: In some cases, a workload that achieves privilege escalation can read and/or write sensitive data that should only be available to other workloads, creating data privacy and security risks.
• Host takeovers: In worst-case scenarios, privilege escalation in Kubernetes can lead to “container escape,” which means workloads or services take control of node operating systems. When this happens, they can effectively access any data available to the host. They could also execute commands or code, including malicious ones.

Alongside other types of key Kubernetes security threats, such as malware inside containers, privilege escalation is a leading cause of security incidents in Kubernetes.

Impacts of privilege escalation in Kubernetes

How privilege escalation happens in Kubernetes clusters

At a high level, privilege escalation occurs whenever a Kubernetes resource gains the ability to bypass or rewrite the access controls that are intended to restrict its permissions. Common causes of privilege escalation scenarios in Kubernetes include the following.

RBAC misconfigurations and excessive permissions

Kubernetes has a built-in Role-Based Access Control (RBAC) framework that admins can use to define access controls and permission settings on a resource-by-resource (or namespace-by-namespace) basis. RBAC is the foundation of effective access control in Kubernetes.

That said, mistakes in RBAC policies can trigger privilege escalation. For example, if engineers accidentally bind a resource to a role that has expansive permissions (like cluster-admin), the resource will effectively have admin-level access to an entire cluster. It’s also possible for RABC misconfigurations to grant resources the ability to rewrite the RBAC roles that apply to them, effectively granting them total access to the cluster.

Pod Security Standards misconfigurations

While Kubernetes RBAC policies enforce access controls and permissions over Kubernetes resources, Pod Security Standards (PSS) are a complementary security framework that define runtime-level permissions for workloads; in other words, they establish a Pod security context.

This is another context where mistakes can cause privilege escalation. Misconfigurations such as allowing a Pod to run as root, or to access host-level networking resources, can grant workloads the ability to perform actions during runtime that interfere with other workloads or disrupt hosts.

Overprivileged containers due to Security Context misconfigurations

An overprivileged container is one that is allowed to run with privileges (typically, those assigned to the root account in Linux) that normally wouldn’t be available to a standard workload.

This can occur as a result of RBAC or PSS misconfigurations. It can also happen due to missing or misconfigured security context settings. Security contexts define runtime permissions for Pods or containers. They can be used to enable containers to do things like run as a specific user (via the runAsUser option) or run in privileged mode (which effectively means they run as the root user).

A common reason for assigning permissions like these to a Pod or container is to allow it to access hardware resources (like a GPU) that normally wouldn’t be available to a containerized workload. However, admins may accidentally enable overprivileged container settings to achieve a specific goal without understanding the broader security implications.

Token abuse by service accounts

Attackers can sometimes elevate privileges for a Kubernetes resource by abusing tokens. Typically, this process works as follows:

• Attackers compromise a container by exploiting a vulnerability or security misconfiguration.
• They extract service account tokens from the compromised Pod.
• They use the tokens to create new Pods or impersonate users.

Essentially, token abuse attacks allow malicious actors to take tokens that are supposed to govern permissions for a single workload, then rewrite permissions settings for other workloads.

Kernel and container runtime vulnerabilities

Beyond mistakes in the various frameworks that define access settings for Kubernetes resources, vulnerabilities in the Linux kernel, as well as in the runtime that Kubernetes uses to execute containers, could trigger privilege escalation risks.

The root cause of this type of issue is flaws within kernel or runtime code that attackers can exploit to circumvent access controls or execute malware. These vulnerabilities are typically fixed via patches whenever security researchers discover them, but if admins don’t apply updates regularly to their Kubernetes nodes and clusters, there is a risk that they’ll operate using vulnerable kernels or runtimes.

Kubernetes components that are most vulnerable to privilege escalation

While privilege escalation in Kubernetes can occur in a variety of ways, the components that most commonly serve as vectors for privilege escalation include:

• The RBAC framework
• Pod Security Standards
• Security Contexts

As we explained above, configuration mistakes within policies or code associated with any of these frameworks can trigger privilege escalation vulnerabilities.

As for the types of Kubernetes resources most exposed to privilege escalation, they mainly include Pods, containers, and service accounts. In situations where access controls are misconfigured, attackers can potentially elevate the privileges of these three types of resources in order to access data or perform actions that should not be available to them.

How to detect privilege escalation attempts in Kubernetes clusters

One of the tricky things about Kubernetes privilege escalation is that Kubernetes lacks a built-in means of detecting and reporting it. Indeed, to Kubernetes, privilege escalation attempts usually appear to be “normal” activity because they involve using standard Kubernetes components, as opposed to deploying resources that would not normally be present in a Kubernetes cluster.

That said, there are types of activity that admins can monitor as a way of identifying privilege attempts, including:

• Sudden Role and ClusterRole changes: Unexpected changes to these policies could mean that an attacker has exploited an RBAC misconfiguration to elevate privileges.
• Secret requests: Attempts to access Secrets via watch or list requests may be a sign of privilege escalation, since viewing Secrets is a common goal of attackers who have successfully elevated privileges.
• Containers running in privileged mode: If a container executes with the privileged: true setting defined, it could be the case that a malicious actor modified code to elevate the container’s privileges.
• Unexpected commands or code execution: Commands that enable the execution of code inside a container (like kubelet exec), as well as the creation of unusual processes inside a container, may mean that an attacker has bypassed permissions and is attempting to execute malicious code.

You can track most of these activities using the Kubernetes auditing capability, which records the types of events described above (auditing won’t report unusual processes inside a container, but you can detect those using tools that leverage eBPF to track process activity). Auditing makes it easy to detect anomalous activity quickly, without having to inspect configurations or track processes manually.

Mitigation strategies for privilege escalation in Kubernetes

Practices like the following help prevent Kubernetes privilege escalation:

• Enforce Pod Security Standards: As we mentioned, Pod Security Standards help prevent privilege escalation by blocking pods from running with dangerous settings such as privileged mode, host namespace access, or unrestricted Linux capabilities.
• Harden RBAC permissions: Hardening prevents privilege escalation by limiting users and service accounts to only the Kubernetes API actions and resources they explicitly need.
• Configure secure container Security Contexts: Avoiding insecure settings within Security Contexts, like allowing containers to run in privileged mode, mitigates the risk of containers accessing sensitive resources.
• Use admission controllers to block risky configurations: Admission controllers are add-ons that can inspect and block malicious or risky Kubernetes API server requests.
• Restrict host access from Pods: Setting hostPID: false and hostIPC: false prevents Pods from interfering with host-level processes.

Best practices for preventing privilege escalation in Kubernetes

Beyond the specific practices we just described for blocking privilege escalation attempts, the following, more general best practices can reduce the risk of configuration errors and vulnerabilities that might trigger privilege escalation in Kubernetes:

• Enable automatic security updates for nodes: Automatic updates help ensure that node kernels and container runtimes are patched against vulnerabilities that may enable privilege escalation.
• Apply the principle of least privilege: As a best practice, follow the principle of least privilege by assigning each Kubernetes resource the minimum level of access necessary for it to perform its job.
• Isolate privileged workloads: For workloads that require special privileges (like running in privileged mode), deploying them in a separate namespace and/or on a separate node reduces the risk that they can interfere with other workloads.‍
• Enable host-level security controls: Security frameworks that run outside Kubernetes but within Linux (like SELinux and AppArmor) can help prevent malicious activity in the event that privilege escalation occurs. So can practices like mounting sensitive data in read-only mode.

Runtime observability for detecting privilege escalation in Kubernetes with groundcover

Given that privilege escalation can occur in many ways and affect diverse Kubernetes components, detecting privilege escalation attempts requires comprehensive visibility into what’s happening across the cluster.

This is where groundcover excels. By continuously monitoring every node, Pod, container, service account, and other resource, groundcover clues teams in quickly to anomalous behavior that could reflect privilege escalation attacks. In turn, engineers can take action to minimize the impact of potential breaches.

Keeping Kubernetes privileges where they should be

In an ideal world, every Kubernetes resource would have the permissions it needs to do what it needs – no more and no less. With the right practices and tools on your side to manage privileges and mitigate privilege escalation attempts, achieving that goal is possible.