Service Mesh: Strategies, Hidden Pitfalls & Smarter Scaling in Kubernetes

Service meshes in Kubernetes, which help to manage network traffic that flows between microservices, are by no means strictly necessary, but they can offer a lot of convenience by simplifying many aspects of Kubernetes administration, such as service discovery, load balancing, observability, and security.

Read on for a deep dive into the role service meshes play in Kubernetes, how to decide whether you need a service mesh, and how to get the most value out of one if you do deploy it.

What is a service mesh?

A service mesh is a type of infrastructure component that handles communications between services. It’s called a mesh because it essentially creates links between each of the services running within an application hosting stack.

The main purpose of a service mesh is to operate as an intermediary for service-to-service communication. With a service mesh deployed, each service can send a request to the mesh whenever it needs to talk to another service. The mesh then interprets the request, modifies it if necessary (which may be the case if, for example, sensitive data needs to be removed from the request for security purposes), and then forwards it to the appropriate service. The mesh also receives responses and forwards them back to the original service that issued a request.

Service meshes are an add-on component in Kubernetes, not a native feature. To deploy a service mesh, you have to install external software such as Istio, Traefik Mesh, Consul or Linkerd, to name just a few popular open source service meshes.

Service mesh vs. API gateway

A service mesh is not the only type of infrastructure layer that can serve as an intermediary between services. API gateways do the same type of work.

The main difference between service meshes and API gateways, however, is that API gateways handle traffic between an application and external services (i.e., the type of traffic known in networking jargon as north-south traffic). In contrast, service meshes deal with internal traffic (or east-west traffic) by operating as intermediaries for the microservices that compose applications. Service meshes are also sometimes used to handle traffic between different applications in the same hosting environment – such as multiple apps in a Kubernetes cluster – but this is still internal traffic, since it doesn’t involve packets that originate from external applications.

So, while service meshes and API gateways perform similar types of work, they deal with distinct use cases. Note, too, that you can – and often would – deploy both a service mesh and an API gateway for Kubernetes, since each component would help manage a different type of network traffic.

Key components and architecture of a service mesh

A typical service mesh architecture includes two main components:

• The control plane, which acts as the “brain” of the service mesh. The control plane operates as a central hub that is responsible for receiving incoming service requests, processing them, delivering them to the right receiving service, and then sending the response back to the originating service. Actions related to data transformation, observability, and security also occur within the control plane, since all service-to-service traffic flows through it.‍
• The data plane, which integrates with individual microservices and provides an interface by which they can interact with the control plane. To achieve this integration, most service mesh data planes deploy sidecar containers, but some (like Istio deployed in “ambient” mode) use alternative approaches that may reduce resource consumption.

Why a service mesh matters in cloud-native environments

To be clear, you definitely don’t need a service mesh to deploy microservice applications in cloud-native environments like Kubernetes. It’s definitely possible for microservices to communicate directly with each other, without a service mesh acting as an intermediary.

However, managing traffic as it flows across a collection of microservices can quickly become very difficult without the help of a service mesh. For starters, you would have to deal with the issue of ensuring that each microservice can discover other microservices. You’d also have to make sure that each microservice sends and receives requests in a way that other microservices can understand. Both of these tasks would require code within each microservice – so you’d end up burdening your developers by requiring them to include service-to-service communication logic for each microservice they release.

What’s worse, you’d also need to ensure that you can properly observe and secure each microservice. That’s hard to do when there is no central hub from which you can track and manage communications. You could try to include logic within each microservice, telling it how to expose observability data and how to deal with security issues – but here again, this would be a ton of work for developers.

With service meshes, however, you get a simple, scalable way to handle service discovery, traffic routing, observability, and security, without the need to build any of these functions into microservices directly.

The bottom line: You may be able to administer a Kubernetes cluster easily enough without the help of a service mesh if it includes just a handful of discrete services. But it becomes very difficult to operate Kubernetes without a service mesh once your total service count exceeds about a half-dozen.

Core use cases of service mesh

As we’ve hinted at, service meshes assist with four core use cases:

• Service discovery: Service meshes provide a way for services to locate one another without having to track each other’s specific identities. This is important in dynamic environments like Kubernetes, where services constantly spin up and down, and their IP addresses often change. With a service mesh, service X can effectively connect to the mesh and say, “here’s a request for service Y – please send it for me” without having to know information like service Y’s IP address or which node it’s running on.
• Traffic management and load balancing: Service meshes assume responsibility for deciding which traffic should flow where when microservices talk to each other. Not only can they ensure that each request reaches the right service, but they can also do things like balance load across services (if there are multiple instances of the same service) or rate-limit how many requests a given service is allowed to send or receive (which can help prevent services from becoming overwhelmed with traffic).
• Observability: Instead of having to collect network performance metrics from each microservice directly, service meshes can operate as centralized observability data hubs. They don’t include features for interpreting the data (that’s where observability platforms like groundcover come in), but they can report information like how long it takes requests to flow from one service to another, how many requests fail, how response duration rates vary between services, and so on. Without a service mesh, you’d have to collect this data directly from each microservice, which would be very difficult because microservices don’t typically generate these types of data on their own.
• Security: Similarly, service meshes can help with security by providing a central hub for the enforcement of security policies. They can block service-to-service requests that appear malicious, for example, or anonymize sensitive information that one service is trying to send to another. Here again, it would be challenging to implement this type of functionality within each individual service, but it’s relatively easy to let a service mesh assume responsibility for service-to-service security.

For the record, let’s make clear that you don’t need to use service meshes for all of these use cases. For instance, if you just want to use a service mesh for observability and plan to route traffic and balance load in other ways (such as by using custom logic that your developers bake into your microservices), that’s fine. But in general, most admins prefer to outsource as many aspects of service-to-service communication as they can to service meshes.

Service mesh deployment models and process overview

To deploy a service mesh in Kubernetes, you typically start by installing the control plane, often using a tool like Helm. From there, you need to install the data plane. This process can vary depending on which type of data plane deployment model you choose. Here’s a look at the two main options.

Sidecar data plane deployment

The first, and more common, approach is to use a data plane deployment model in which agents run in sidecar containers. These sidecar containers operate alongside each microservice, allowing them to intercept requests and forward them to the service mesh control plane.

Sidecar data planes are relatively simple to install. Usually, you just need to include a container for the service mesh in each application you deploy.

Agentless service meshes

The other approach is an agentless service mesh deployment model for the data plane. Instead of relying on sidecar containers to facilitate communication between individual services and the control plane, an agentless service mesh implements data plane functionality at a different layer – often, via node operating systems. The exact details of how this works vary from one agentless mesh to another, but as an example, one approach is to use eBPF to implement the data plane. This means that data plane functionality can be handled in the kernel space of the node operating system, without the use of sidecar containers.

Agentless service meshes can be more complex to set up, but they usually are a lot faster. The reason why is simple: Running a bunch of sidecar containers consumes a lot of memory and CPU. But with an agentless mesh, the data plane can operate at the node or operating system level, where there is less redundancy and therefore fewer wasted resources. The eBPF-based approach saves even more CPU and memory because kernel-space code runs much more efficiently than user-space software.

Benefits of using a service mesh

The benefits of service meshes can be summed up as follows: Service meshes eliminate the need to manage service discovery, traffic, observability, and security on a service-by-service basis. Instead, admins can offload this work to a centralized service mesh hub.

This simplifies life for Kubernetes administrators because it means they don’t have to worry about managing how each individual service behaves. They can simply configure policies within the service mesh control plane, telling it how to manage traffic, how to collect observability data, and so on.

Service meshes also offer major benefits for application developers because they eliminate the need for developers to include custom logic in each microservice, telling it how to discover other microservices, route traffic to them, and so on. When a service mesh handles these tasks, individual services don’t need as much custom code.

Challenges and pitfalls in service mesh adoption

While service meshes provide many benefits, they can have their drawbacks.

The biggest, arguably, is the added overhead. Service meshes consume CPU and memory, so running a service mesh in your cluster means that there are fewer resources for your actual applications to use. The performance overhead can be especially significant if you opt for a sidecar data plane deployment model, since these are more resource-hungry, as we explained above.

Service meshes are also one more thing that you need to install, manage, and update – so in that sense, they can increase the administrative burden facing Kubernetes engineers.

Of course, service meshes usually more than make up for their overhead and administrative complexity by streamlining administration tasks. But in situations where you have a small number of microservices running in your cluster, it’s possible that a service mesh will create more problems or pitfalls than it solves.

Best practices for service mesh implementation

If you decide a service mesh is right for your cluster, consider these best practices to get the most value from it: 

• Choose the right mesh: There are a variety of service meshes available, including both commercial and open source options. They vary in terms of the features they provide, their installation process, and how easy they are to manage. Review your options to choose a mesh that aligns with your needs, preferences, and level of expertise.
• Choose the right data plane model: As we mentioned, sidecar data planes are easier to deploy, but they tend to perform less well than agentless meshes. Consider your performance goals and choose a data plane model accordingly.
• Consider GitOps-based management: To streamline administration of the service mesh, it can be helpful to store configuration code, traffic policies, and so on in Git – a practice that enables so-called GitOps. In addition to storing all configuration data in a central place, GitOps makes it easy to revert a change quickly if necessary and to store multiple versions of your service mesh configuration.
• Observe your service mesh: Like any other type of software, service meshes can fail or perform inadequately, which is why collecting observability data from the mesh itself is critical.

The role of observability in service mesh success

We just mentioned the importance of observability for service meshes. But this is such an important topic that it merits its own section.

There are two different facets of observability to think about in the context of service meshes. The first is observing your service mesh itself, including both the control plane and data plane components, to ensure they are performing adequately. You’ll want to know about issues like exhaustion of CPU and memory on the Pods that host your service mesh, or a crashed sidecar within the data plane.

Second, it’s also important to think about how you’ll leverage the service observability data that service meshes collect. As we mentioned, service meshes can serve as central hubs for generating metrics about service communication health. But it’s up to you to ensure that you send this data to a tool where you can easily interpret it. Effective interpretation means not just collecting and analyzing all relevant service-to-service communications metrics, but also being able to contextualize this data alongside other information – like the health and status of the Pods and nodes hosting each service, which is critical for understanding whether a communication problem is caused by the network or an issue with a service itself.

How groundcover elevates service mesh observability

Call us biased, but if you ask us which particular platform to use to assist in service mesh observability, we’d confidently answer “groundcover.”

As an end-to-end observability solution built specifically for Kubernetes and other cloud-native platforms, groundcover provides all of the data you need to track service-to-service communication performance and troubleshoot problems quickly. It integrates with popular service meshes to collect relevant observability data, then lets you analyze and visualize the data alongside other vital metrics from all components of your cluster.

Services meshes as a step toward Kubernetes success

Again, no one is saying you absolutely have to use a service mesh to use Kubernetes. But in general, service meshes come highly recommended. They make the management, observability, and security of service-to-service communications much more feasible in clusters of any significant scale. Once you try a service mesh, you’re likely to find that you can never go back.

FAQ

Is a service mesh necessary for all Kubernetes deployments?

No. Service meshes are beneficial in many Kubernetes deployment scenarios. But for clusters that include just a handful of services, service meshes may add more complexity than they are worth. Service meshes can also reduce cluster performance because they consume resources, so they may not be ideal in situations where admins are pursuing very aggressive performance optimization.

What security features are built into modern service meshes?

Core security features in service meshes include the ability to authenticate services and encrypt traffic flowing between them, typically using mutual TLS (mTLS). Service meshes can also enforce security policies configured by admins, such as rules that block certain types of traffic. And service meshes include authentication and authorization features for controlling admins’ access to the service mesh software itself.

Can groundcover integrate with existing service mesh setups for unified monitoring?

Yes. groundcover can integrate directly with popular service meshes, such as Istio and Consul. Integration allows groundcover to collect relevant observability data from the service mesh. In turn, groundcover includes this data within the dashboards available to users for monitoring service health and communications.

This means that when you use groundcover, you don’t need to worry about add-ons or extra configuration for service mesh observability. Groundcover handles the hard work for you.