Imagine trying to drive a car without a speedometer, fuel gauge, temperature gauge and the other essentials that help you understand at a glance what's happening to the vehicle. You'd probably struggle to drive safely. You might even find yourself running out of gas or stranded on the side of the road with an overheated engine.

Similarly, you can't operate a Kubernetes cluster very effectively if you lack a means of visualizing what's happening in the cluster quickly and easily. Tracking the state of your nodes, Pods, and so on in order to identify and address performance issues requires continuous data visualization, along with alerts that tell you when something looks off.

This is where Grafana, a leading open source data visualization and analytics tool, comes in. Grafana is to Kubernetes and other software platforms what the dashboard is to a car: A solution that helps you understand the state of your software environment on an ongoing basis.

Grafana does this by helping to visualize the metrics that Kubernetes produces. Because Grafana's flexibility allows you to create multiple visualizations for multiple types of metrics, you can easily get the insights you need to achieve deep observability, even in the most complex of environments.

To prove the point, this article walks through what Grafana does and why it matters. We also explain how to set up Grafana in Kubernetes and how to make the most of Grafana as part of your Kubernetes observability stack.

What is Grafana?

In case it's not clear by now, Grafana is an open source tool designed to help visualize and interpret monitoring data. Its key functionality centers on what are known as Grafana dashboards, which are customizable data visualizations that you can view side-by-side based on data sources of your choice.

In addition, Grafana allows you to set up alerting conditions. It will then fire off alerts when your data meets a certain condition or passes a threshold.

Thus, Grafana helps teams make sense of monitoring data – especially when they're dealing with complex environments (like those based on Kubernetes), where they typically have many data sources that they need to analyze collectively in order to detect anomalies and errors. You can use Grafana to help observe virtually any type of workload, but its power really shines when you're dealing with complex, cloud-native environments.

To be clear, Grafana doesn't collect monitoring data for you. For that, you'll need other tools that you integrate with Grafana, as we explain below. But once you have your monitoring data, Grafana helps you figure out what it all means so that you can put it to use for the purpose of optimizing workload performance.

Installing Grafana in Kubernetes

Beyond the fact that Grafana offers some powerful observability features, the icing on the Grafana cake is its ease of setup and deployment. Although the setup process can vary depending on which type of environment you're deploying Grafana to, it's typically pretty simple.

For example, to set up Grafana in Kubernetes, you'd follow these steps:

  1. Create a Grafana manifest file. You can find the boilerplate code here.
  2. Deploy the file into your cluster with the command

And there you go. Grafana should now be up, running and available through your browser at localhost:3000 (or whichever port you configured for running Grafana when you created the manifest).

Grafana configuration for Kubernetes

The steps above cover Grafana deployment on Kubernetes, but in order to connect Grafana to Kubernetes data sources so that you have data to visualize, you'll need to perform some extra configuration.

First, install kube-state-metrics by running:

Then, install the Kubernetes Metrics Server with:

Kube-state-metrics and the Kubernetes Metrics Server generate data that helps you monitor the state of your Kubernetes cluster.

To get the data into Grafana, you'll also need to set up a metrics backend. VictoriaMetrics and Prometheus are both good choices. We won't go through the details of setting up these metrics backends here, but suffice it to say that you can find solid guides for deploying VictoriaMetrics and Prometheus online.

Data sources for Grafana

Above, we explained how to set up Grafana and collect simple Kubernetes metrics with it. But Kubernetes cluster metrics represent just the tip of the iceberg when it comes to the data sources you can analyze and visualize with Grafana. Grafana supports a wide variety of additional data sources.

Data sources that you can use with Grafana fall into two categories:

 • Built-in data sources, which Grafana supports out-of-the-box with minimal setup or configuration needed. Grafana offers more than a dozen built-in data sources, such as Prometheus, Loki, Elasticsearch and all of the monitoring services (like Cloudwatch) on the major public clouds.

 • Data sources made available via plugins. There are currently more than 150 data sources that you can connect to Grafana using plugins, and if that list isn't long enough for you, you can create your own Grafana plugin to move data from your favorite source into Grafana dashboards.

So, whether you want to visualize data from Kubernetes or virtually any other type of environment or tool, there's a way to get the data into Grafana via either a built-in or custom data source.

Creating custom Grafana dashboards for Kubernetes

To use Grafana to maximum effect in your Kubernetes environment, you'll want to create custom dashboards that display data in ways that make most sense for your needs.

You can do this easily through the Grafana UI (which, again, will be available at a URL such as localhost:3000). Start by clicking Dashboards, then click New and New Dashboard to begin building a custom dashboard.

You'll be able to select from a wide range of Grafana visualization options. Common options include:

 • Time series, which displays graphs of data over time.

 • Table, which shows data in a tabular format.

 • Heatmaps, a way of displaying histograms over time.

 • Lists, which you can use to display lists of alerts, dashboards and other components inside Grafana.

With so many options, Grafana makes it easy to visualize data in whichever ways make most sense based on the type of data you're dealing with and your use cases for the data. Plus, as we mentioned above, Grafana lets you display multiple dashboards side-by-side, which is great when you have multiple data sources to monitor and/or if you want to visualize the same data source in multiple ways at once.

Monitoring Kubernetes with Grafana alerts

Grafana also gives you a lot of flexibility when it comes to setting up alerts. You can configure a variety of alerting rules from within the Grafana UI.

To create an alert, navigate to Alerting and click Alert rules, then Create alert rule. From there, you'll be asked to give a name to your new alert. You'll then have to choose which data source the alert is based on. You can select Loki or Prometheus (assuming you have both configured as Grafana data sources).

The rest of the alert configuration depends on which data source serves as the basis for your alerts. But to take Prometheus as an example, you'd typically configure fields such as:

 • Alert: A name for your alert.
 • Expr
: The data expression that describes a condition that needs to be present for an alert to fire.
 • For
: Defines a duration of time for which an alert's conditions need to be met before an actual alert fires.
 • Labels
: Labels that you define to describe an alert.

You can also set up Alertmanagers, which tell Grafana where to send an alert. Popular Alertmanagers include Slack, Microsoft Teams and Pagerduty, to name just a few.

Best practices for using Grafana with Kubernetes

To maximize the effectiveness of Grafana as part of a Kubernetes monitoring stack, consider these best practices:

 • Know what to monitor: Trying to create a Grafana dashboard for every metric that exists in a Kubernetes cluster is not realistic. Even if you set up all those dashboards, you wouldn't be able to monitor them all. Instead, define which metrics and which types of visualizations are most important, then create dashboards accordingly. Typically, creating somewhere between one and two dozen dashboards in total is a good number to target.

 • Keep dashboards simple: In general, it's better to build multiple simple dashboards and view them side-by-side than to try to pack complex visualizations into a single dashboard.

 • Use one metric per dashboard: Although there may be cases where you want to track multiple metrics (like CPU and memory) together in order to monitor how they correlate, it's a best practice in most cases to create dashboards based on a single metric.

 • Use consistent dashboard design: To the extent possible, strive to use the same types of dashboard designs and layouts. This makes it easy to compare dashboards and understand context when monitoring Kubernetes with Grafana.

 • Tag dashboards: Although Grafana offers a dashboard search feature that you can use to find dashboards, applying tags to each dashboard makes it easy to navigate between them without having to search.

Kubernetes monitoring made simple with Grafana

When it comes to making sense of Kubernetes metrics, it's hard to think of a better tool than Grafana. It's highly flexible and adaptable, it's easy to set up and it makes it a breeze to configure visualizations and alerts tailored to your needs.

In fact, we think Grafana is so great that we chose to build groundcover's custom dashboard offering based on the Grafana stack. We hope you'll find it just as compelling for meeting your Kubernetes monitoring needs.

Sign up for Updates

Keep up with all things cloud-native observability.

We care about data. Check out our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.