Allen Institute for AI Unveils 'Truly Open Source' LLM to Propel a 'Critical Shift' in AI Development

The Allen Institute for AI (AI2), a non-profit organization founded in 2014 by the late Microsoft co-founder Paul Allen, has introduced OLMo—an open-source large language model (LLM) described as “truly open-source” and a “state-of-the-art” alternative to restrictive, closed models. This initiative marks a significant shift in the landscape of AI development.

Unlike other models that only share code and weights, OLMo offers comprehensive resources, including training code, training data, and associated toolkits, alongside evaluation tools. Released under an Open Source Initiative-approved license, all OLMo components, including code, weights, and intermediate checkpoints, are available under the Apache 2.0 License.

The announcement comes at a crucial time when open-source AI is rapidly advancing, striving to catch up with proprietary models like OpenAI's GPT-4 and Anthropic's Claude. For instance, the CEO of Paris-based startup Mistral recently confirmed the emergence of a new open-source AI model approaching GPT-4 performance. Additionally, Meta recently launched an enhanced version of its code generation model, Code Llama 70B, while anticipation builds for the upcoming Llama LLM iteration.

However, the open-source AI sector faces criticism from some researchers, regulators, and policymakers. A notably controversial opinion piece in IEEE Spectrum claimed that “Open-Source AI is Uniquely Dangerous.”

The OLMo framework promotes a “completely open” approach, providing full access to pretraining data, training code, model weights, and evaluation processes. This includes inference code, training metrics, training logs, and a development evaluation suite featuring over 500 checkpoints for each model, tracked throughout the training process under the Catwalk project.

AI2 researchers plan to continuously improve OLMo by introducing various model sizes, modalities, datasets, and capabilities. “Many language models today lack transparency,” stated Hanna Hajishirzi, project lead and senior director of NLP Research at AI2. “Researchers can’t fully understand a model’s workings without access to training data. Our framework empowers researchers to scientifically study LLMs, essential for developing safe and trustworthy AI.”

Nathan Lambert, an ML scientist at AI2, emphasized that OLMo represents a new paradigm in LLMs. “OLMo enables fresh approaches to ML research and deployment, facilitating scientific development at every stage of the process,” he said.

The open-source AI community has responded enthusiastically to OLMo's release. Jonathan Frankle, chief scientist at MosaicML and Databricks, hailed it as “a giant leap for open science.” Hugging Face’s CTO also remarked on social media that the model is “pushing the envelope of open source AI.”

Meta's chief scientist, Yann LeCun, highlighted in AI2's press release that “Open foundation models drive innovation in generative AI, and a vibrant open-source community is key to shaping the future of AI.”

Correction: An earlier version inaccurately referred to AI2's LLM as the first truly open-source model; several fully open-source models exist. We apologize for this error.