Grafana for Kubernetes: Unlocking the Power of Metrics and Dashboards

Kube-state-metrics and the Kubernetes Metrics Server generate data that helps you monitor the state of your Kubernetes cluster.

To get the data into Grafana, you'll also need to set up a metrics backend. VictoriaMetrics and Prometheus are both good choices. We won't go through the details of setting up these metrics backends here, but suffice it to say that you can find solid guides for deploying VictoriaMetrics and Prometheus online.

Data sources for Grafana

Above, we explained how to set up Grafana and collect simple Kubernetes metrics with it. But Kubernetes cluster metrics represent just the tip of the iceberg when it comes to the data sources you can analyze and visualize with Grafana. Grafana supports a wide variety of additional data sources.

Data sources that you can use with Grafana fall into two categories:

 • Built-in data sources, which Grafana supports out-of-the-box with minimal setup or configuration needed. Grafana offers more than a dozen built-in data sources, such as Prometheus, Loki, Elasticsearch and all of the monitoring services (like Cloudwatch) on the major public clouds.

 • Data sources made available via plugins. There are currently more than 150 data sources that you can connect to Grafana using plugins, and if that list isn't long enough for you, you can create your own Grafana plugin to move data from your favorite source into Grafana dashboards.

So, whether you want to visualize data from Kubernetes or virtually any other type of environment or tool, there's a way to get the data into Grafana via either a built-in or custom data source.

Creating custom Grafana dashboards for Kubernetes

To use Grafana to maximum effect in your Kubernetes environment, you'll want to create custom dashboards that display data in ways that make most sense for your needs.

You can do this easily through the Grafana UI (which, again, will be available at a URL such as localhost:3000). Start by clicking Dashboards, then click New and New Dashboard to begin building a custom dashboard.

You'll be able to select from a wide range of Grafana visualization options. Common options include:

 • Time series, which displays graphs of data over time.

 • Table, which shows data in a tabular format.

 • Heatmaps, a way of displaying histograms over time.

 • Lists, which you can use to display lists of alerts, dashboards and other components inside Grafana.

With so many options, Grafana makes it easy to visualize data in whichever ways make most sense based on the type of data you're dealing with and your use cases for the data. Plus, as we mentioned above, Grafana lets you display multiple dashboards side-by-side, which is great when you have multiple data sources to monitor and/or if you want to visualize the same data source in multiple ways at once.

Monitoring Kubernetes with Grafana alerts

Grafana also gives you a lot of flexibility when it comes to setting up alerts. You can configure a variety of alerting rules from within the Grafana UI.

To create an alert, navigate to Alerting and click Alert rules, then Create alert rule. From there, you'll be asked to give a name to your new alert. You'll then have to choose which data source the alert is based on. You can select Loki or Prometheus (assuming you have both configured as Grafana data sources).

The rest of the alert configuration depends on which data source serves as the basis for your alerts. But to take Prometheus as an example, you'd typically configure fields such as:

 • Alert: A name for your alert.
 • Expr: The data expression that describes a condition that needs to be present for an alert to fire.
 • For: Defines a duration of time for which an alert's conditions need to be met before an actual alert fires.
 • Labels: Labels that you define to describe an alert.

You can also set up Alertmanagers, which tell Grafana where to send an alert. Popular Alertmanagers include Slack, Microsoft Teams and Pagerduty, to name just a few.

Best practices for using Grafana with Kubernetes

To maximize the effectiveness of Grafana as part of a Kubernetes monitoring stack, consider these best practices:

 • Know what to monitor: Trying to create a Grafana dashboard for every metric that exists in a Kubernetes cluster is not realistic. Even if you set up all those dashboards, you wouldn't be able to monitor them all. Instead, define which metrics and which types of visualizations are most important, then create dashboards accordingly. Typically, creating somewhere between one and two dozen dashboards in total is a good number to target.

 • Keep dashboards simple: In general, it's better to build multiple simple dashboards and view them side-by-side than to try to pack complex visualizations into a single dashboard.

 • Use one metric per dashboard: Although there may be cases where you want to track multiple metrics (like CPU and memory) together in order to monitor how they correlate, it's a best practice in most cases to create dashboards based on a single metric.

 • Use consistent dashboard design: To the extent possible, strive to use the same types of dashboard designs and layouts. This makes it easy to compare dashboards and understand context when monitoring Kubernetes with Grafana.

 • Tag dashboards: Although Grafana offers a dashboard search feature that you can use to find dashboards, applying tags to each dashboard makes it easy to navigate between them without having to search.

Kubernetes monitoring made simple with Grafana

When it comes to making sense of Kubernetes metrics, it's hard to think of a better tool than Grafana. It's highly flexible and adaptable, it's easy to set up and it makes it a breeze to configure visualizations and alerts tailored to your needs.

In fact, we think Grafana is so great that we chose to build groundcover's custom dashboard offering based on the Grafana stack. We hope you'll find it just as compelling for meeting your Kubernetes monitoring needs.