Why Flip AI Developed a Custom Large Language Model for Enhancing Its Observability Platform

Observability has become a key term in IT discussions, referring to the process of monitoring a company’s systems to identify issues or determine the root causes of problems. These issues can frequently disrupt sites or applications, potentially leading to costly downtime.

While many startups and established firms are working to address this challenge, Flip AI is introducing a unique approach to the observability landscape. This early-stage startup has developed its own large language model (LLM) specifically aimed at enhancing system monitoring.

Recently, Flip AI announced that its product is now generally available, along with a noteworthy $6.5 million seed funding round.

CEO and co-founder Corey Harrison points out that despite the plethora of available tools, companies often rely on labor-intensive processes to track data across various systems. Together with co-founders CTO Sunil Mallya and CPO Deap Ubhi, Harrison recognized the opportunity to leverage intelligence and automation to significantly reduce resolution times.

“Large enterprises often utilize multiple tools, yet they struggle during the troubleshooting phase,” Harrison explained. This challenge is especially pronounced in larger organizations where a multitude of tools and disparate data sources complicate the process of pinpointing the root cause without extensive manual querying.

By utilizing a large language model trained on over 100 billion tokens of DevOps-specific data — including logs, metrics, traces, and configuration files — Flip AI aims to accelerate the troubleshooting process and recovery times. “We’ve developed our own large language model; we’re not using OpenAI or similar solutions,” Harrison said. “It functions similarly to human queries across systems, allowing for a seamless analysis.”

The result is a powerful tool that can perform root cause analysis in under a minute, often within just seconds, while maintaining the integrity of the data with minimal read access required.

Harrison acknowledges that no model is infallible; however, Flip AI provides transparency regarding how the model derives its conclusions, enabling developers to validate its findings. “Even if the root cause analysis isn’t 100% accurate, we’ve localized the error, executed queries, and retrieved sample data, accomplishing 90% of the work for you,” he explained.

Creating a proprietary LLM is an ambitious undertaking, but both Mallya and Ubhi bring valuable experience from Amazon, where Mallya directed Amazon Comprehend, the company’s natural language processing service, and Ubhi served as director of product management. Harrison himself has a strong technical background, including a recent role as SVP of operations and chief of staff to the NFL Commissioner.

With 20 employees across San Francisco and Bangalore, India, the company is currently focused on balancing customer demand with its growth strategy. Harrison notes the importance of fostering diversity within the tech industry, a topic he reflects on frequently. “Understanding my background and the diverse individuals who supported my journey, I am committed to ensuring Flip AI embodies a similarly diverse culture,” he emphasized.

The recent $6.5 million seed investment round was led by Factory, with participation from Morgan Stanley Next Level Fund and GTM Capital.