Kubernetes Services & Load Balancing: Keep Your Microservices Happy and Reachable

All Pods with the label my-app would then receive traffic pointing to the Service.

The service will direct the traffic to the targetPort.

But that’s not all, the code snippet above is an example for a ClusterIP Service - K8s default service type, there are additional Service types available, each with its own unique super powers.

Service types

As mentioned, there are multiple Service types that you can specify when defining a Service, the options are:

 • ClusterIP: This is the default type. It makes Pods accessible only from within your cluster.

 • NodePort: NodePorts make your Pods accessible on a fixed port on each Node in the cluster. You can manage both internal and external requests using NodePort. NodePort can be utilized to implement your own load balancing solution.

 • LoadBalancer: This type makes your Service externally accessible using a cloud provider's external load balancer, which will route traffic to the Pods in your Service automatically. The features supported by this type depends on the cloud vendor implementation.

 • ExternalName: With this type, you rely on an external DNS service to route traffic as it returns a custom CNAME record.

A cool use case for ExternalName objects is to route traffic outside of the cluster or to ingresses that expects specific host names (great trick when working with API gateways!). 

As you can see, each Service type is suited for different use cases. ClusterIP makes sense for Services that only need to be reachable from within your cluster and don’t pose unique requirements.  NodePort is a basic way to integrate with custom load balancing solutions, And if you need a more granular control over the Service object and want to integrate it deeper with your cloud assets (such as TLS certificates) and use your cloud vendor capabilities, you might choose to use the LoadBalancer type instead.

Headless vs. regular Services

But what if you want to be able to directly connect to the Pods directly? The solution in that case is to configure a headless Service, which you can set up by choosing a ClusterIP Service type and setting None for the cluster IP specification.

With a headless Service, there is no cluster IP address that endpoints can use to reach your Service. Instead, discovery requests to your Service will return a list of IP addresses for each Pod in the Service. 

Why would you want to do that, you ask? The most common reason is if you want to hand off the decision of which pod should serve a request to the requesting application. An example for such a case is when contacting databases governed by a StatefulSet, those usually expose the option to contact a specific replica in the StatefulSet - by setting up a headless Service.

The role of kube-proxy in Kubernetes Services and load balancing

So what’s the underlying magic that makes sure traffic to a Service is directed to the correct pods, and in a distributed way?

The answer is kube-proxy and depending on whether it operates on top of IPVS or Iptables (more on that in future articles) - it is responsible for directing the traffic designated to the Virtual IP Representing the Service Object to one of the targeted pods.

Whether using iptables or IPVS, kube-proxy will  implement the routing logic to randomly and evenly  distribute the Service received traffic to the targeted pods.

So when a group of pods targeted by a Service is scaled up or down, Kubernetes will magically supply basic load balancing to those instances out of the box - Very nice!

By the way, we cannot ignore the fact that eBPF is revolutionizing this field as well, and there are solutions replacing kube-proxy with eBPF-based networking. 

Best practices for monitoring and managing Kubernetes Services

As you just saw, Services are highly flexible objects. And the fact that Kubernetes automatically provides load balancing for most Service types makes Services all the more wonderful.

But like most Kubernetes objects, Services are complex, and a lot of things can go wrong. To minimize your chances of running into issues with Services, consider the following best practices:

 • Monitor your networking plane: As you’ve seen, whether the Service object is governed by kube-proxy or by a custom networking plane, there are crucial underlying components in Kubernetes that affect our applications directly - and those must be monitored independently  in order to attest that application connectivity functions as expected.

 • Monitor your Services using a tool that delivers Pod-level visibility into Service performance: This is crucial because if you only monitor traffic from Services point-of-view, you won't know which individual Pod or Pods within the Service may be causing regressions.

 • Scale your Services with cost in mind: Obviously, you want your Services to be responsive. But if your Deployments are "too" good at handling requests, there's a chance that it's because they are under-utilized. In that case, you could scale back the Pods in order to reduce your overall costs.

 • Know which Service types to use when: Identifying the Service type that serves you the best can help simplifying and optimizing cluster inter-connectivity, and although ClusterIP is a good default, other Service objects might be a better fit for your needs.

These pointers will help you get the very most out of Kubernetes Services and ensure that your microservices have the network connectivity they need, where and when they need it.