How Lemonade improves application performance monitoring & plans to replace Datadog’s APM with groundcover

Background on Lemonade

Lemonade Inc., founded in 2015, sells renter, homeowner, car, pet and term life insurance through a digital platform that uses automation and analytics to keep its labor costs low, even as it has grown to more than 1 million paying customers. This approach extends to the company's DevOps team, primarily located in Israel, which supports some 200 developers with five DevOps engineers.

"We're getting a head start," Avital said. "Redesigning the stack is mostly us looking five years ahead. ... And the key idea there is not for groundcover to be the best data analytics platform but a really great troubleshooting platform that helps you surface problems and show you what went wrong before you even know what to look for."_{-Avital Tamir}

Challenges

Lemonade is a consumer product company, supporting real time services such as online insurance quotes and claims. Lemonade has shifted into a microservices architecture and uses AWS’s EKS managed Kubernetes solution to deploy many different services, and a complex tech stack that spans across many code languages and platforms, posing a challenge for any monitoring and observability solution.

As part of its migration to Kubernetes, Lemonade is still actively shaping its microservices architecture, deprecating monolithic components that are part of its business logic. These components are harder to instrument and observe since they are not actively maintained by the R&D team.

Lemonade’s observability data volumes are massive, which quickly translates into significant costs with its main observability vendor - Datadog. Lemonade manages its cost surges by moving some of its observability data to internally managed monitoring stacks

These internally managed solutions require manual labor and detailed integrations that are maintained by the Lemonade DevOps team.

The Lemonade team sought out a way to provide deep observability into their Kubernetes stack that would provide value to the DevOps, SRE and development teams without requiring code instrumentation or complex integrations.

Solutions & Benefits

Lemonade has been running groundcover on their production K8s clusters since March 2022. The relationship started when Lemonade joined as one of groundcover’s first design partners, early on in the company’s journey.

"We've been watching groundcover grow from drawings on napkins … and it feels like it's just getting started," Avital said. "I really would like to see it mature over the next year into a product that can actually rival some of the big observability players."

Lemonade has been using Datadog for infrastructure monitoring and APM across their stack for a long time as the company grew over the last five years at a compound annual growth rate of 150% and from 23 employees at its founding to more than 1,000. With its rapid growth so did the cost of observability in terms of software licensing and developer time for manual instrumentation. "The billing for Datadog just kept growing and growing. And looking a few years ahead, we kind of figured we need to start doing something that's a little more cost effective," Avital said. "Not only that but also to open up the possibility of storing metrics for long periods of time … with high cardinality and without going bankrupt."

Lemonade started using groundcover’s observability platform which installs an eBPF agent via a Daemonset on Kubernetes clusters and uses information flowing out of the Kubernetes API to put that data in context for each application. groundcover's agent also performs some analytics at the cluster level before sending data to its central repository. This reduces traffic on the network, trims the amount of data that must be stored for long-term analysis and prioritizes alerts according to criticality.

Why groundcover

{{quote-block}}

groundcover provided Lemonade with a high cardinality application and infrastructure metrics solution that allowed the different teams better visibility into their Kubernetes infrastructure and deployments.

“Lemonade now uses groundcover's dashboards displayed on TV screens to show developers data about their apps in test and development environments. Not having to instrument applications separately for each programming language has allowed Lemonade's DevOps team to roll it out, update it and feed data to its packaged dashboards for these environments quickly and easily so far”

By using groundcover, Lemonade also got simpler visibility into active issues in its K8s stack that requires attention. groundcover’s auto-detected issues also provides K8s and metrics, logs and traces context that helps teams troubleshoot faster.