Kubernetes Ingress: How to Manage Access, Security, and Scale

Although certain Kubernetes clusters (like those used for dev/test purposes) may run as cloistered islands, most host workloads that need to be able to accept traffic from the outside world – or at least, the world outside a Kubernetes cluster. Hence the importance of Kubernetes Ingress, one of the features that makes it possible to route incoming network traffic to applications within a Kubernetes cluster.

And hence, too, the importance for Kubernetes admins of knowing how to use Ingress in Kubernetes, how to keep Ingress traffic secure, and how to manage and monitor Ingress. Below, we unpack all of the above and more so that you’re ready to make the very most of Ingress as a networking solution for Kubernetes.

What is Kubernetes Ingress?

In Kubernetes, Ingress is an object within the Kubernetes API that routes network traffic from outside the cluster to one or more Services hosted within the cluster. Usually, Ingress uses HTTP/HTTPS to manage traffic.

To understand this definition fully, it helps to know what a Service is. A Service is essentially a network endpoint that is associated with one or more Pods inside a Kubernetes cluster. Services are a way of establishing a fixed network identity for workloads hosted in Kubernetes. This is important because the internal IP addresses of Pods and containers (meaning the ones they receive within the cluster) can change over time as they shut down and restart or move between nodes. With a Service, you can assign a persistent network address to Pods (and the containers inside them), which other workloads can then use to communicate with the Pods.

If an application that is external to a Kubernetes cluster wants to communicate with one or more Pods, you can use Ingress to direct traffic to them, provided you’ve configured a Service for the Pod or Pods.

Note that Ingress requires the presence of an ingress controller within your cluster. An ingress controller is a special application that accepts incoming traffic and decides how to route it based on rules defined in the ingress controller’s configuration. Usually, ingress controllers run inside dedicated Pods and are implemented using external tools, like NGINX, HAProxy, or Envoy. Kubernetes doesn’t have a built-in ingress controller (although Ingress itself is a native feature).

Ingress vs. LoadBalancer vs. NodePort

Ingress isn’t the only way to map external network traffic to a specific workload in Kubernetes. The other main options include:

• LoadBalancer: A LoadBalancer operates as a Service with an externally routable IP address – so external workloads can connect directly to it. Unlike Ingress, however, a LoadBalancer Service can only manage traffic for one application at a time. However, LoadBalancers don’t require an ingress controller, so they are simpler to deploy. (Note that Ingress and a LoadBalancer Service can both perform load balancing by distributing traffic between Pods, so you don’t need to use a LoadBalancer service if you require this capability – despite what the terminology might seem to imply.)
• NodePort: A NodePort makes a workload accessible to external traffic by opening a network port and assigning it to that workload. It’s a simple way of accepting incoming traffic, and it provides less control and security than Ingress. NodePorts don’t require an ingress controller to be present in a cluster.

In most respects, Ingress is the most powerful way to accept incoming traffic. It’s also more efficient from a configuration and resource consumption standpoint due to its ability to manage traffic for multiple services (so there’s less to configure, and less overhead associated than you’d have if you ran separate LoadBalancers or NodePorts for each application you wanted to expose). That said, LoadBalancer or NodePort Services may be better than Ingress for simple use cases; for example, if you have just one application in your cluster that needs to accept external traffic, setting up Ingress for it might be overkill.

As an example of Kubernetes Ingress in action, imagine a use case where an admin wants to route traffic to different Services based on URL paths. Here’s an example of an Ingress configuration that achieves this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-app-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/proxy-read-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - example.com
      secretName: example-tls-secret  # Must contain the TLS certificate and key
  rules:
    - host: example.com
      http:
        paths:
          - path: /api(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080
          - path: /(.*)
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 80

This example (which uses NGINX as the ingress controller) routes traffic as follows:

• Traffic with /api/* in its path routes to api-service on port 8080
• All other requests go to frontend-service on port 80

This example also encrypts traffic using a Secret named example-tls-secret.

Key features of Kubernetes Ingress

As you may have surmised by now, Ingress is more than simply a way to get traffic from an external endpoint to an endpoint inside Kubernetes. The full range of key features of Ingress include:

• Routing incoming traffic to Services within a cluster.
• The ability to route traffic to multiple Services at the same time based on hostnames or URL paths.
• Load balancing, for distributing traffic across multiple instances of a Service.
• SSL termination, encryption, and decryption.

Other methods of handling incoming traffic don’t provide as much control.

Benefits of using Kubernetes Ingress

The main benefit of using Kubernetes Ingress instead of an alternative method of accepting incoming traffic (like a LoadBalancer or NodePort service) is that Ingress provides more features. As noted above, Ingress offers full support for features like traffic encryption and decryption, which are not always available using other techniques.

Ingress also tends to be more efficient, as we mentioned, because a single Ingress configuration can handle all external traffic routing needs for your cluster. Although Ingress requires an ingress controller (which adds overhead to the cluster), it’s still usually more efficient from a resource perspective than trying to run multiple LoadBalancer or NodePort Services.

Common Kubernetes ingress controllers

A variety of open source networking solutions are available that can serve as Kubernetes ingress controllers. Popular options include:

• NGINX Ingress Controller: The NGINX Ingress Controller is one of the most widely used and well-supported options, maintained by the Kubernetes community and F5 NGINX. It’s known for its stability, rich configuration options, and ability to handle complex routing and TLS termination efficiently.
• Istio Ingress Gateway: The Istio Ingress Gateway is part of the Istio service mesh and integrates tightly with its traffic policies and observability stack. It’s best suited for environments that already use or plan to adopt service mesh capabilities.
• HAProxy Ingress: Built on the HAProxy load balancer, this controller emphasizes high performance, low latency, and fine-grained traffic control. It’s particularly strong in environments requiring advanced TCP and HTTP load balancing capabilities.
• Traefik: Traefik is a modern, cloud-native ingress controller with built-in service discovery and automatic reconfiguration. It’s easy to use, integrates seamlessly with Let’s Encrypt for HTTPS, and provides an intuitive dashboard for observability.
• Kong Ingress Controller: Kong builds on the Kong API gateway to provide ingress and API management features. It stands out for its extensibility, plugin ecosystem, and strong focus on security, authentication, and rate limiting.

Security considerations for Kubernetes Ingress

Since Ingress allows external applications on the wilds of the Internet to connect to resources inside a Kubernetes cluster, it’s important to ensure that Ingress is configured in a secure way. The following practices can help:

• Avoid ambiguous routing rules: When routing traffic based on hostnames or URLs, avoid ambiguities that attackers could exploit to reach a Service that shouldn’t be available to them.
• Use encryption: In general, it’s a best practice to enable encryption as part of Ingress. This mitigates the risk of attackers intercepting data in transit.
• Update your ingress controller: Ensure that your ingress controller software is up-to-date to protect against vulnerabilities in the ingress controller itself.
• Manage secrets securely: If your Ingress configuration includes Secrets, ensure that they are managed properly, such as by storing them in a dedicated Secrets repository.

Challenges in managing Kubernetes Ingress

The main challenge associated with managing Kubernetes Ingress is that Ingress configurations can be complex. As we noted above, it can be less complex overall to have a single Ingress compared to managing multiple LoadBalancer or NodePort Services, but each Ingress still tends to include a lot of parameters and complexity.

To reduce the complexity, try to keep routing and rewrite rules simple (while avoiding ambiguities that may lead to security issues). It can also be helpful to version-control your Ingress configurations, since this makes it easier to prevent configuration drift and revert to earlier configurations in the event that you encounter an issue.

Best practices for Kubernetes Ingress management

To simplify the overall process of managing Ingress in Kubernetes, consider these best practices:

• Choose the right ingress controller: Ingress controllers vary in terms of their performance and how many options they offer. While a full comparison of ingress controllers is beyond the scope of this article, suffice it to say that you should review available options carefully to decide which one is best for your needs.
• Monitor Ingress: To detect problems with network performance, like dropped packets or failure to route traffic to a specified Service, monitor traffic both within and outside your cluster. 
• Enable encryption by default: Unless you have a specific reason not to encrypt traffic – which is probably not the case – turn on encryption by default.
• Rotate certificates: Bolster the security of encryption by rotating certificates regularly.
• Manage Ingress configurations as code: As mentioned above, define Ingress configurations as code, then use a version-control system (like Git) to track them over time.
• Centralize Ingress behavior: Certain aspects of Kubernetes network management (like authentication and authorization) can be handled at the Service level. But it’s usually preferable to handle them through Ingress, which provides a more centralized and efficient approach. (The exception is if you need different behavior from each Service.)

Troubleshooting Kubernetes Ingress

If you run into a problem with Ingress, such as traffic not being routed as you expected, the following troubleshooting strategies can help determine the cause of the issue:

• Check Ingress status: Use the command kubectl get ingress ingress-name to confirm that an expected Ingress resource actually exists. You can also run kubectl describe ingress ingress-name to collect details on the status of an Ingress.
• Validate Ingress configuration: Passing the Ingress configuration code through a YAML linting tool can help catch typos that may be causing your controller to fail to interpret the configuration properly.
• Verify controller is running: Use kubectl describe pods pod-name to check on the status of the Pod that hosts your ingress controller.
• Check your controller version: Be sure your controller is up-to-date. Outdated controllers are not only potentially insecure, but they may also contain bugs or not support all of the features defined in your Ingress.
• Check network status: To rule out issues with network connectivity, log into nodes or containers in your cluster and use commands like ping or wget to be sure they are able to send traffic to each other, as well as to external endpoints.
• Check logs: Check the logs of your ingress controller. The log location and details vary from one controller to the next, but most controllers will log information like failed connections.

Monitoring and observability for Kubernetes Ingress

You can monitor and observe Kubernetes Ingress by focusing on two different types of data sources.

1. Generic performance metrics

First, look at generic metrics related to network performance, like bandwidth, dropped packet rate, and latency. You can collect this data using network monitoring tools.

2. Ingress controller metrics

Depending on which ingress controller you use, the controller may expose additional metrics – and often, they will be more granular than those you can get through generic network monitoring. For example, you may be able to track connection counts on a workload-by-workload basis.

How groundcover simplifies Kubernetes Ingress monitoring

With groundcover, you can take Kubernetes Ingress monitoring to the next level.

In addition to offering a simplified, automated way of collecting all relevant data about Ingress performance (regardless of which ingress controller you use), groundcover makes it easy to visualize data about the health and responsiveness of your Ingress resources. It also contextualizes Ingress data with other critical insights – like the status of Pods and nodes – to accelerate troubleshooting and root cause identification.

Getting more from Kubernetes Ingress

Ingress isn’t the only way to route incoming traffic to workloads with Kubernetes. But it’s the most powerful way – which is why you should consider Ingress for any use case that requires advanced network configuration and management. Just be sure you’re also simultaneously prepared to handle the security, management, and monitoring aspects of working with an ingress controller.

FAQ

How can teams troubleshoot performance issues in Kubernetes Ingress?

The best way to get details about performance issues related to Kubernetes Ingress is to look at logs and metrics provided by the ingress controller you use. These will typically provide insights like how many connections the Ingress is supporting or which types of traffic are triggering errors.

It may also be helpful to examine metrics related to the overall health of your cluster, such as CPU and memory utilization. In some cases, lack of adequate resources may contribute to poor Ingress performance.

What metrics should be monitored to ensure Kubernetes Ingress reliability?

For starters, monitor the CPU and memory utilization of the Pod that hosts the ingress controller, since a lack of resources for the controller could cause it to be unable to handle traffic reliably. You can also look at metrics related to network performance, like total requests, error rate, and latency. These serve as indicators that the Ingress is not performing well.

How does groundcover improve observability for Kubernetes Ingress?

groundcover simplifies Ingress observability in several ways. First, it automatically collects relevant data about the health and status of an Ingress, eliminating the need for admins to provide this data manually. Second, it alerts teams to problems with an Ingress (or with Kubernetes networking in general), helping them to manage issues proactively. Finally, groundcover contextualizes Ingress health and performance data with insights drawn from all other layers of a cluster – Pods, nodes, Services, and more – which is critical for troubleshooting issues that involve multiple variables (like a situation where an ingress controller is failing because the Pod hosting it is out of memory due to a problem with the host node’s configuration).