Kolena, a promising startup focused on developing tools for testing, benchmarking, and validating the performance of AI models, has successfully secured $15 million in a funding round led by Lobby Capital, with participation from SignalFire and Bloomberg Beta. This latest round brings Kolena’s total funding to $21 million. The new capital will be used to expand the company’s research team, collaborate with regulatory bodies, and enhance its sales and marketing initiatives, as co-founder and CEO Mohamed Elgendy shared in an email interview.
"The applications for AI are vast, yet both creators and the public lack trust in the technology," Elgendy stated. "We need to ensure that the deployment of this technology enhances digital experiences rather than detracts from them. While we can’t reverse the trend, our industry can make informed decisions regarding its development."
Kolena was founded in 2021 by Elgendy, Andrew Shi, and Gordon Hart, who spent roughly six years collaborating in AI divisions at major companies like Amazon, Palantir, Rakuten, and Synapse. The founders aimed to create a "model quality framework" that offers unit testing and end-to-end testing for AI models within a customizable, enterprise-friendly format.
“Our primary goal was to introduce a new framework for model quality, not merely a tool that simplifies existing methods,” Elgendy explained. “Kolena enables continuous scenario-level or unit testing and provides comprehensive testing of entire AI and machine learning products, not just isolated components.”
Kolena’s platform is designed to offer insights that help identify gaps in test data coverage for AI models. It also incorporates risk management features that monitor risks linked to deploying specific AI systems. Through Kolena’s user interface, clients can create test cases to assess a model's performance and uncover reasons behind any shortcomings, while also comparing performance across various models.
“With Kolena, teams can manage tests tailored to specific scenarios encountered by the AI product, instead of relying on a generic metric like an accuracy score that might obscure critical performance details,” Elgendy noted. “For instance, a model achieving 95% accuracy in car detection isn’t inherently superior to one at 89%. Each model has unique strengths and weaknesses for different conditions, such as varying weather or occlusion levels.”
If Kolena functions as intended, it could provide invaluable support to data scientists who invest considerable time in building models for AI applications.
Surveys indicate that AI engineers spend only 20% of their time analyzing and developing models, while the remainder goes toward sourcing and cleaning training data. Additionally, research reveals that only about 54% of AI models progress from pilot to production due to difficulties in developing accurate and high-performance models.
Kolena faces competition from several startups and established players, including Amazon, Google, and Microsoft, which are also developing tools for testing and validating AI models. Emerging startups like Prolific, which raised $32 million to train and stress-test AI models using a crowdsourced network, Robust Intelligence, and Deepchecks, are creating their own solutions to enhance AI reliability. Moreover, Bobidi incentivizes developers for testing AI models.
However, Elgendy asserts that Kolena stands out by allowing customers complete control over the data types, evaluation logic, and other elements that constitute an AI model test. He highlights Kolena’s privacy-centric design, which does not require customers to upload their data or models; instead, Kolena only retains model test results for future benchmarking, which can be deleted upon request.
“Mitigating risk in AI and machine learning systems necessitates thorough testing prior to deployment, yet many enterprises lack robust tools and processes for model validation,” Elgendy said. "Today’s ad-hoc testing practices often lead to unsuccessful machine learning proof of concepts. Kolena prioritizes exhaustive model assessment, providing machine learning managers, product directors, and executives with unparalleled insight into test coverage and product-specific requirements, empowering them to enhance quality from the outset.”
Headquartered in San Francisco, Kolena has a team of 28 full-time employees. While the company did not disclose its current client count, Elgendy mentioned that Kolena is selectively partnering with "mission-critical" organizations for now, with plans to introduce team bundles for mid-sized companies and emerging AI startups in Q2 2024.
Ranking
