Experts in media and technology assert that open-source large language models (LLMs) could significantly influence generative AI within enterprises, potentially more so than proprietary models like OpenAI's ChatGPT or Anthropic's offerings.
Despite extensive experimentation and numerous proof-of-concept projects with open-source models, established companies have been slow to disclose their real-world applications. To address this, we reached out to major open-source LLM providers, including Meta, Mistral AI, IBM, Hugging Face, Dell, Databricks, AWS, and Microsoft.
Our inquiries revealed 16 notable deployment cases (see list below). While this number is modest, industry analysts expect to see a surge in open-source applications by the end of this year.
Delays in Open-Source LLM Adoption
One reason for the slow uptake of open-source models is their recent inception. Meta launched the first significant open-source model, Llama, in February 2023, three months after OpenAI released ChatGPT in November 2022. Mistral AI's Mixtral model, which has garnered praise in various benchmarks, was released just a month ago.
As a result, real-world deployments are only beginning to surface. Open-source advocates acknowledge that the prevalence of closed models currently outweighs that of open-source models, but predict that the gap will narrow.
Challenges with Open-Source Models
There are limitations to current open-source models. Amjad Masad, CEO of the software startup Replit, pointed out that the feedback mechanism for model development is ineffective due to the difficulty in contributing to enhancements. However, many may have underestimated the extent of experimentation occurring among open-source developers, who have created thousands of derivative models—some demonstrating equal or superior performance against closed models on specific tasks, such as FinGPT and BioBert.
The Limited Value of Large Public Models for Enterprises
According to Matt Baker, Dell's SVP of AI Strategy, "Large public models have little to no value for private companies." Baker explained that these models tend to be overly generalized and do not facilitate easy integration of private enterprise data, which constitutes about 95% of the AI work done by organizations.
As a result, many enterprises are exploring open-source solutions for customer support and code generation, using custom code often incompatible with generic closed-model LLMs.
Slow Movement Due to Enterprise Considerations
Andrew Jardine from Hugging Face noted that enterprises often hesitate to adopt LLM applications because they prioritize data privacy, customer experience, and ethical implications. Companies typically pilot internal use cases before branching out to external applications. While closed models saw significant deployments toward the end of 2023, open-source implementations are anticipated to accelerate this year.
Despite advantages, some enterprises find open-source cumbersome. Working with established APIs from providers like OpenAI is often seen as simpler than managing open-source licensing and governance issues.
Bridging the Gap Between Open and Closed Models
Jardine emphasized that the distinction between open and closed models is increasingly blurred. For instance, many companies, including a large pharmaceutical firm, utilize a closed LLM for internal tasks and an open-source model for specific functions such as identifying sensitive information. This choice reflects a desire for greater data control.
The Case for Open-Source Adoption
As model adaptation and cost factors shift rapidly, businesses will likely seek the flexibility to switch between open-source and closed models to mitigate risk. Companies often lean towards open-source models to retain control over sensitive data while fine-tuning them for specialized applications.
Several companies, such as Intuit and Perplexity, are already developing generative AI orchestration layers that allow for the seamless integration of various models for specific tasks, regardless of their open or closed status. Although deploying open-source models on a large scale may require more effort, they can offer cost savings over time, especially for organizations with existing infrastructure.
Many enterprises quietly leverage open-source models. For instance, automotive firms and airlines are experimenting with applications powered by Databricks’ lakehouse platform, which incorporates open-source LLMs.
Challenges in Identifying Open-Source Deployments
Defining bona fide enterprise use cases can be challenging. While many developers and startups create applications using open-source LLMs, we aimed to highlight established enterprises with significant applications. We defined an enterprise as having at least 100 employees and focused on end-users rather than suppliers of LLM technology.
Defining "open source" also presents challenges, as seen with Meta's Llama, which has a restricted license. Moreover, while companies like Writer have developed their own LLMs, only certain models are open source, complicating the classification.
Below are examples of documented enterprise deployments of open-source LLMs:
1. VMWare: Utilizes the Hugging Face StarCoder model for enhanced code generation in a self-hosted environment, ensuring the security of proprietary code.
2. Brave: The privacy-centric browser employs the Mixtral 8x7B model for its conversational assistant, Leo.
3. Gab Wireless: Implements Hugging Face models to screen messages for children's safety, blocking inappropriate content.
4. Wells Fargo: Uses Meta's Llama 2 model for various internal applications, enhancing employee resources.
5. IBM: Incorporates open-source LLMs in its AskHR app, which assists employees with HR-related inquiries and is also utilized in new consulting services.
6. The Grammy Awards: Collaborates with IBM for “AI Stories,” utilizing Llama 2 to generate custom insights and content for fans.
7-9. Masters Tournament, Wimbledon, US Open: Leverage IBM technology for real-time commentary and highlight generation during events.
10. Perplexity: A startup focused on reinventing search, using open-source LLMs for response summarization in its search engine.
11. CyberAgent: Employs open-source LLMs from Dell for Japanese language modeling tailored to user needs.
12. Intuit: Combines its own LLMs with open-source models to enhance the capabilities of its Intuit Assist feature.
13. Walmart: Develops conversational AI applications, having initially utilized Google's open-source BERT model.
14. Shopify: Deploys Llama 2 in Sidekick, an AI tool that streamlines tasks for e-commerce business owners.
15. LyRise: A talent-matching startup creating a chatbot for recruiting, utilizing Llama for interactions.
16. Niantic: Uses Llama 2 to drive character interactions in the mobile game Peridot.
Despite existing challenges in defining and tracking these deployments, the interest in open-source LLMs is surging. As more companies explore their potential, we anticipate the number of public use cases will grow. We will continue to update this list as new information surfaces.
Ranking
