The Falcon AI model team has introduced its latest iteration, Falcon 2, touted to outperform Meta’s Llama 3. Developed by the Technology Innovation Institute (TII), Falcon 2 features 11 billion parameters and comes in two forms: a base version and one specifically optimized for visual tasks. The Falcon team focused on creating smaller yet highly effective models, enabling developers and businesses to implement them more economically. For perspective, the original Falcon model boasted a massive 40 billion parameters, along with a 180 billion parameter version that faced challenges with “problematic” outputs due to its size.
Both Falcon 2 models are designed to operate efficiently on a single GPU, which means that developers and businesses can scale the models without needing extensive infrastructure. Remarkably, they can also run on standard laptops. “AI is continually evolving, and developers recognize the many benefits of smaller, more efficient models,” remarked Hakim Hacid, TII’s executive director. “These models not only reduce computing power requirements but also align with sustainability goals, providing increased flexibility and seamless integration into emerging edge AI infrastructures.”
The Falcon 2 11B VLM represents a significant advancement as the first multimodal version of the model, adept at processing images alongside text. This capability allows it to identify objects within images, making it a valuable asset across various industries. In healthcare, for instance, it can assist clinicians in analyzing X-ray results, while in finance, it helps in the examination of charts and graphs to unearth trends.
Both versions of Falcon 2 support multiple languages, including English, French, Spanish, German, and Portuguese, further increasing their utility in diverse applications.
### Falcon 2 vs. Llama 3: A Showdown of Open Source Models
Upon its release, TII asserted that Falcon 2 surpasses Meta's Llama 3. The performance of Falcon 2 11B outshined the 8 billion parameter version of Llama 3 across several key evaluation benchmarks. Specifically, it received scores of 78.30 on the WinoGrande benchmark, 82.91 on HellaSwag, and 52.56 on TruthfulQA, compared to Llama 3 8B’s scores of 77.35, 82.09, and 43.09, respectively. However, it is worth noting that Llama 3’s smaller model excelled on the MMLU test, achieving a score of 66.6 compared to Falcon 2’s 58.37.
TII publicly launched only the 8 billion version and a larger 70 billion parameter version of Llama 3. The prior release of a 13 billion parameter model with Llama 2 remains comparable to Falcon 2 in size, but model size alone does not dictate performance.
“While Falcon 2 11B has showcased exceptional capabilities, we remain committed to the ethos of open-source innovation,” stated Faisal Al Bannai, secretary general of the Advanced Technology Research Council and advisor on advanced technology matters to the UAE president. “With various multimodal models anticipated to enter the market in differing sizes, we aim to provide developers and organizations that prioritize privacy access to one of the premier AI models available.”
Falcon 2 operates on an open-source basis, permitting any user to leverage its capabilities. Businesses can utilize the model to enhance their AI applications, provided they adhere to its acceptable use policy. The base version of Falcon 2 is accessible through Hugging Face, while the visual version will be available starting May 14.
### Future Developments: Falcon 2 Mixture of Experts (MoE)
TII is also planning to innovate further by developing a Mixture of Experts (MoE) version of Falcon 2. MoE models work collaboratively, generating responses based on inputs from various smaller systems to produce enhanced answers. While TII has not disclosed specifics on the MoE version, the strategy promises to boost the model’s accuracy and decision-making capabilities.
The adoption of the MoE approach could help Falcon 2 mitigate the limitations commonly associated with current transformer architectures in language models. Seth Dobrin, founder and CEO of Qantm AI, emphasized this potential during a February appearance on a podcast. He stated that if TII applies the MoE strategy in its latest models, it could result in improved performance without a significant increase in parameters. This method is reminiscent of techniques employed in GPT-4 and other leading language models.
Despite the advancements of Falcon 2, Dobrin noted that TII has yet to introduce a Falcon model tailored to support Arabic, which represents an exciting opportunity to enhance its presence in the region and access a substantial market. The continued commitment to maintaining the Apache open-source license ensures that Falcon 2 stands out as one of the highest-performing models available in the realm of truly open-source large language models.
Ranking
