Could a furry camelid claim the title for the biggest AI story of 2023? If we’re referring to Llama, Meta’s large language model that revolutionized the AI research landscape in February, followed by the commercial Llama 2 in July and Code Llama in August, I would argue yes.
I can already hear the backlash. “What? ChatGPT is clearly the biggest AI story of 2023!” I can almost hear the crowd shouting. “OpenAI’s ChatGPT launched on November 30, 2022, and reached 100 million users by February! It brought generative AI into popular culture!”
But hear me out. In my view, ChatGPT is undeniably a generative AI game-changer. As Forrester analyst Rowan Curran noted, it was “the spark that set off the fire around generative AI.”
However, February marked a pivotal moment when Meta released Llama, the first significant free 'open source' Large Language Model (LLM), sparking a heated debate about open-source AI that has persisted throughout the year. While other tech giants, LLM companies, and policymakers scrutinized the safety and security of AI models with open source access—coupled with costly computational challenges—open source AI gained significant traction.
According to Meta, since the release of Llama, the open-source AI community has created over 7,000 derivatives on the Hugging Face platform, resulting in notable models like Koala, Vicuna, Alpaca, Dolly, and RedPajama. While there are other open source models—such as Mistral, Hugging Face, and Falcon—Llama was the first backed by a major tech company’s data and resources.
One could liken ChatGPT to 2023’s blockbuster film, Barbie, while Llama and its open-source counterparts resemble an expansive Marvel Universe, filled with spinoffs that promise a lasting impact on the AI landscape.
This trend will lead to “more real-world, impactful generative AI applications, further cementing the open-source foundations of the field,” said Kjell Carlsson, head of data science strategy and evangelism at Domino Data Lab.
Long-term Impact of Open Source AI
The era of closed, proprietary models was heralded by ChatGPT. OpenAI, founded in 2015 as an open research initiative, shifted its stance in 2023. Ilya Sutskever, OpenAI’s co-founder, admitted to The Verge that sharing research was a misstep motivated by competitive and safety concerns.
In contrast, Meta’s chief AI scientist Yann LeCun advocated for the commercial release of Llama 2, emphasizing that large language models are crucial infrastructure that needs to remain open. Carlsson acknowledged that comparing ChatGPT and Llama is somewhat like comparing apples to oranges, with Llama 2 being revolutionary due to its open-source nature and commercial licensing, enabling fine-tuning and on-premises operation.
While he recognized ChatGPT as pivotal in bringing LLMs to public awareness, he argued that its underlying models, GPT 3.5 and 4, are limited in scope and should primarily be used for proof of concept.
Matt Shumer, CEO of Otherside AI, noted that Llama's impact likely stems from the groundwork laid by ChatGPT. He stated, “There are hundreds of companies started in the last year that would not have been possible without Llama and the subsequent developments.”
Former Neeva CEO Sridhar Ramaswamy commended Llama 2 as the first genuinely capable open-source AI model, likening it to the potential cloud model landscape where a few companies could monopolize AI capabilities. Instead, Meta made Llama accessible.
The Early Llama Leak and Its Aftermath
Launched in February, Llama stood out for its varied model sizes, from 7 billion to 65 billion parameters. Llama’s 13B model performed better than GPT-3 in many NLP benchmarks, and its largest model rivaled top-tier models like PaLM and Chinchilla. Initially, Meta provided model weights to select academics and researchers, including Stanford for its Alpaca project.
Following a leak on 4chan, developers worldwide gained access to a GPT-level LLM, resulting in a surge of derivatives. In July, Meta made Llama 2 freely available for commercial use, with Microsoft integrating it into its Azure cloud service.
This was a crucial moment as discussions about AI regulation escalated. In June, U.S. Senators expressed concerns about the Llama leak, highlighting potential misuse in various harmful activities.
Despite these challenges, Meta reinforced its commitment to open-source AI. In a June meeting, Mark Zuckerberg emphasized integrating generative AI into all Meta products and reaffirmed the company’s dedication to an "open science-based approach" to AI research.
Meta: Champion of Open Research
Meta has consistently championed open research, significantly by fostering an ecosystem around the PyTorch framework. As 2023 nears its end, Meta will celebrate the tenth anniversary of the Fundamental AI Research (FAIR) initiative, aimed at advancing AI through open research for the public good.
Joelle Pineau, VP of AI research at Meta, joined the organization in 2017 due to its commitment to open science, which she emphasized as part of Meta’s core values. “The reason I joined without looking anywhere else is the commitment to open science,” she reflected.
However, Pineau noted a shift in the motivation behind open research. While initial goals focused on enhancing research quality, recent developments have significantly spurred productivity across the AI ecosystem, empowering numerous startups with alternative models.
Yet, she cautioned that Meta does not guarantee constant open releases; each project undergoes meticulous risk-benefit analysis.
Reflections on Llama: Precision Over Perfection
Angela Fan, a Meta FAIR research scientist involved in Llama and its follow-ups, emphasized the careful groundwork that went into creating the Llama models. “Even though the technology is still developing, we can create interesting tools and develop consistent integration across our apps,” she stated.
Meta seeks continuous feedback from its developer community and startups employing Llama for various applications, aiming to refine future iterations.
Fan highlighted the importance of accuracy and meticulous preparation in Llama’s development. It’s not merely about grand breakthroughs; it’s about executing numerous smaller tasks effectively over time.
Advocacy for Open Source AI
Vipul Ved Prakash, co-founder of Together—a startup recognized for creating the RedPajama dataset—concurred with the belief that Llama and open-source AI are the game changers of 2023. By developing high-quality models within a vast network of companies and organizations, the costs are effectively distributed, helping startups mitigate model-building expenses.
However, as regulators begin to scrutinize open-source AI, advocates emphasize the need to safeguard access to these models. At a recent Safety Summit in the U.K., concerns about the risks of advanced AI systems were predominant.
Despite these concerns, a coalition of open-source AI proponents, including LeCun and Google Brain co-founder Andrew Ng, released a statement asserting that open AI is “an antidote, not a poison.”
Ongoing Debate: Llama vs. ChatGPT
The Llama versus ChatGPT debate continues to spark discussions among experts. While some firmly believe ChatGPT remains the dominant story of the year, others see Llama's influence as a precursor to transformative changes in AI.
Nikolaos Vasiloglou, VP of ML research at RelationalAI, insisted, “Hands down, ChatGPT is a game-changer,” thanks to its engineering and operational efficiency. John Lyotier, CEO of TravelAI, echoed this sentiment, highlighting ChatGPT's role in making AI accessible to everyday users.
Conversely, Ben James, CEO of Atlas, noted that Llama has revitalized research and innovation in a way that ChatGPT did not, suggesting long-lasting implications.
Ultimately, both models have contributed richly to the AI landscape. While ChatGPT ignited widespread interest in generative AI, Llama is poised to shape its future impact. As Curran articulated, the phenomenon of generative AI in 2023 would not have occurred without ChatGPT, yet the drive for innovative applications will be significantly fueled by the open-source community.
In the long run, both proprietary and open-source models will coexist. However, without open-source models like Llama 2—widely adopted by enterprise developers—the generative AI landscape would be considerably less advanced and overly niche. The open-source community plays a crucial role in driving significant long-term advancements in this exciting field.
Ranking
