Apple Unveils Open-Source Multimodal LLM: October Release Highlights

In October 2023, researchers from Apple and Columbia University quietly released an open-source multimodal large language model (LLM) named Ferret. This release included both the model's code and weights, but it was designated for research purposes only and not for commercial use. Initially overlooked, the situation has shifted as recent developments in open-source models—such as Mistral's innovations and Google’s Gemini model debuting on the Pixel Pro—have sparked discussions about the potential for local LLMs on small devices.

The conversation intensified following Apple's announcement of a significant breakthrough in LLM deployment on iPhones. The company published two new research papers outlining techniques for creating 3D avatars and enhancing AI inference efficiency. These advancements promise to deliver more immersive visual experiences and pave the way for complex AI systems operating on consumer devices like the iPhone and iPad.

Many in the AI community are now recognizing the importance of Apple's foray into the open-source LLM space. Bart de Witte, who leads a European non-profit dedicated to open-source AI in medicine, noted on X, “I somehow missed this. Apple joined the open-source AI community in October. Ferret’s release demonstrates Apple’s commitment to impactful AI research, establishing its position as a leader in multimodal AI… I look forward to the day when local LLMs (LLLMs) run on my iPhone as part of a redesigned iOS.”

Tristan Behrens, a German AI music artist and advisor, also highlighted this development on LinkedIn, writing, “Well, Christmas is tomorrow. But did you realize that Apple (yes, Apple!) recently released a multimodal LLM, including code and weights?”

Tech blogger Ben Dickson added, “What is the AI development in 2023 that you expected the least? For me, it was Apple releasing open-source LLMs (albeit with a non-commercial license).” He noted Apple’s historic stance on closed systems and emphasized the company's strategic move into the LLM market, aligning itself with the open-source trend.

This move aligns with the current landscape, where companies like Anthropic and OpenAI are reportedly seeking significant new funding to bolster their proprietary LLM efforts. According to Reuters, Anthropic is in discussions to raise $750 million from Menlo Ventures, while Bloomberg highlighted that OpenAI is exploring a fresh funding round with a valuation potentially exceeding $100 billion.