Navigating the Pitfalls of Generative AI: A New Approach to Risk Management
Generative AI holds incredible potential, but it also comes with significant risks—particularly when it involves third-party models. These tools can generate misinformation, display biases, and potentially violate copyright laws. A recent survey from MIT Sloan Management Review and Boston Consulting Group found that third-party AI tools account for over 55% of AI-related failures in organizations. It’s no wonder many companies are hesitant to fully embrace this technology.
What if Generative AI Came with a Warranty?
This was the innovative idea proposed by Karthik Ramakrishnan, an entrepreneur and electrical engineer, during his time at Deloitte. After co-founding two AI-focused companies—Gallop Labs and Blu Trumpet—Ramakrishnan recognized that a lack of trust and the inability to quantify risk were significant barriers to widespread AI adoption.
“Every enterprise is actively seeking ways to leverage AI for increased efficiency and market competitiveness,” Ramakrishnan shared in an email interview. “However, many organizations are implementing third-party AI models without fully understanding the underlying product quality. The rapid advancement of AI raises ongoing risks and potential harms.”
To address these concerns, Ramakrishnan partnered with Dan Adamson, an expert in search algorithms and seasoned startup founder, to establish Armilla AI. The company aims to provide warranties on AI models used by corporate clients.
How Can Armilla Provide Warranties?
You may wonder how Armilla manages to offer warranties on models that often operate as "black boxes," hidden behind licenses, subscriptions, and APIs. Ramakrishnan explained their approach centers on rigorous benchmarking and strategic customer acquisition.
Armilla evaluates both open-source and proprietary models, assessing their quality based on current global AI regulations. The company tests for issues like hallucinations, racial and gender biases, and overall security across various theoretical applications, utilizing its proprietary assessment technology.
If a model meets Armilla's standards, the company offers a warranty that reimburses clients for any fees paid to use that model. “We provide enterprises with confidence in the technology they acquire from third-party AI vendors,” Ramakrishnan stated. He likened their assessment process to penetration testing commonly applied to new technologies.
I also inquired whether specific models would be excluded from evaluation for ethical reasons—such as facial recognition algorithms from vendors with questionable practices. Ramakrishnan responded:
“Assessing problematic AI models would contradict our ethical values and undermine our business model, which relies on establishing trust. Legally, we refuse to engage with clients whose models are banned by the EU, including certain facial recognition and biometric categorization systems.”
A Unique Approach to AI Warranty
While the concept of warranties and insurance for AI isn't entirely new—Munich Re introduced an AI insurance product, aiSure, in 2018—Ramakrishnan argues Armilla's methodology is distinct.
“Our assessments cover various aspects like KPIs, processes, performance, and qualitative as well as quantitative criteria—all done swiftly and cost-effectively,” he elaborated. “We assess AI models against requirements outlined in regulations such as the EU AI Act and New York's Local Law 144 regarding AI hiring bias. We are also prepared for additional regulations, like Canada’s AI and Data Act, as they emerge.”
Launched in late 2023 and backed by insurance carriers such as Swiss Re, Greenlight Re, and Chaucer, Armilla has attracted around ten clients, including healthcare firms leveraging generative AI for processing medical records. Ramakrishnan noted that Armilla's client base has been doubling month over month since Q4 2023.
“Our primary customers are enterprises and third-party AI vendors,” Ramakrishnan said. “Enterprises use our warranty to safeguard against risks associated with third-party AI vendors, while vendors leverage our warranty as proof of their product's trustworthiness, expediting their sales cycles.”
Challenges and Opportunities Ahead
Warranties for AI models make logical sense, yet I question whether Armilla can keep pace with the rapidly evolving AI regulatory landscape. Laws surrounding AI use—like New York City’s hiring algorithm bias law and the EU AI Act—could place the company at risk for substantial payouts if its assessments are not foolproof.
Ramakrishnan addressed this concern head-on:
“Given that regulations continuously develop across numerous jurisdictions, grasping the nuances of legislation worldwide is crucial. There's no universal approach we can adopt, so we must adapt as needed. While this complexity is challenging, it also creates a competitive advantage for us.”
Based in Toronto with a growing team of 13 employees, Armilla recently raised $4.5 million in a seed round led by Mistral (distinct from the AI startup of the same name), with participation from Greycroft, Differential Venture Capital, Mozilla Ventures, and others. With total funding now reaching $7 million, Ramakrishnan indicated that the funds will help expand Armilla’s warranty offerings and develop new products.
“Insurance will play a pivotal role in managing AI risk, and Armilla is at the forefront of creating insurance solutions that enable companies to deploy AI safely,” Ramakrishnan concluded.
Ranking
