Flowcore breaks free of legacy vendor lock-in and reduces observability costs by 87% with groundcover

Introduction

Flowcore is a next-generation data management platform that enables the use of data products and the emergent practice of data-driven development. The R&D team is composed of 3 developers, led by Julius Biskopstø, Co-Founder and CTO at Flowcore, and are all involved in monitoring and optimizing their application and infrastructure performance.

The nature of the company’s product drives their cloud-agnostic approach, as they must be able to deploy and test their product in the environment in place of their end-users, and with a wide variance between different customers. To facilitate this flexibility, Flowcore uses Kubernetes to manage their environment with 31 nodes in total - 20 in their production cluster and 11 in their staging cluster. The company is steadily growing its customer base and scaling their environments rapidly.

For Julius, the ideal observability solution must natively support both the required flexibility of their environment and their fast-scaling nature.

The Problem

Julius and his team experimented with Datadog’s APM solution, which was relatively easy to deploy, but required extensive and complex instrumentation. Overall, using the solution created significant challenges for them in two main areas:

1. Developer hours: Datadog required hours upon hours of maintenance, which, as a small team, had a direct impact on the team’s ability to focus on their actual responsibilities - continuously developing and improving their product.
2. Financial uncertainty: Datadog’s pricing model lacked transparency and created constant concern around upcoming costs. “Flowcore is developing a very distributed system that has a lot of microservices on which we need to have high throughput observability and high throughput metrics," says Julius. “Using that with Datadog would spike the price up extremely fast because of how they price custom metrics”.

Why groundcover?

As the Flowcore team came to a clear conclusion that they could not continue working with that solution, they tried out multiple APM solutions, many of which claimed to be “zero setup” solutions, but in reality, ended up being significantly time consuming to deploy. New Relic, Honeycomb, Dynatrace, Instana, and other solutions that were tried by the team were found to be too complex to integrate into their distributed system, required extensive configuration to gain meaningful insights, and, most notably, were offered at unpredictable, often prohibitive, costs.

Julius came across an old video review of groundcover by Viktor Farcic, which convinced him it could stand out in the face of all the other solutions he and his team had come across. The first thing that appealed to him was the transparent and predictable pricing model, which is based on the number of nodes and completely unaffected by the volume of monitoring data. That was enough to give groundcover a try.

“groundcover’s model enables us to confidently introduce new services and metrics for enhanced observability, without the concern of impacting our financial margins. This certainty is invaluable for a growing startup, granting us the freedom to innovate, expand our services, and add new features with confidence.”
- Julius Biskopstø, Co-Founder & CTO, Flowcore

The Impact

Julius and his team were amazed by the ease of installation and instant visibility over all their logs, metrics, traces and Kubernetes events. With Datadog, they needed to use very complex annotations to get their metrics during setup. Then, they also needed to update these annotations across all their applications. As they were using Helm charts, this was not a huge issue for them but still consumed developer hours. Similarly, they needed to thread carefully with any use of custom metrics and make sure to apply ignores, so as not to spike the bill. With groundcover, none of this was required, as it is built on VictoriaMetrics, requiring much simpler annotations, and there’s no need to limit custom metrics since data volume doesn’t affect the price.

“It literally took like under ten minutes to get everything up and running”
- Julius Biskopstø, Co-Founder & CTO, Flowcore

These factors were a complete game-changer for Flowcore.

groundcover significantly reduced the time and effort required for system monitoring and issue resolution. The team freed up dozens of developer hours per month, which were used to set up and maintain the monitoring of their 100+ services. This time is now being used to speed up the development of their platform-as-a-service solution. 

Furthermore, the ability to monitor every aspect of their infrastructure, in real-time and without the need to selectively choose which area of their environment to monitor, enabled proactive problem-solving and system optimization, rather than reactive firefighting.