Apple Fully Embraces Generative AI: Launching a 30 Billion Parameter Multimodal Model

In contrast to tech giants like NVIDIA and Microsoft, Apple has disappointed investors this year, experiencing a stock price drop of over 10% and losing its title as the "most valuable company in the world." The primary factor behind this decline is Apple's struggle to keep pace in the rapidly evolving field of artificial intelligence (AI). In response, Apple has recently shifted focus from its car project to generative AI, announcing key developments in this area.

Apple's research team recently published a paper on "MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training," introducing their multimodal large model, MM1. This milestone emphasizes Apple's strides and capabilities in AI. Featuring up to 30 billion parameters, the MM1 model excels in contextual learning and multi-image reasoning, achieving strong performance on various multimodal benchmarks. Researchers are actively testing the model to identify critical factors that influence its effectiveness.

Notably, the research indicates that the resolution of images and the number of image labels significantly affect performance, surpassing the impact of visual language connectors. This underscores the importance of diverse pre-training datasets in optimizing model efficacy. MM1 is distinguished by its impressive scale and innovative architecture, which incorporates dense models and mixtures of expert models. These advancements validate the research team's strategy of combining large-scale pre-training with targeted data selection to bolster learning capabilities.

The team asserts that MM1 sets a "state-of-the-art" standard in pre-training metrics and remains "competitive" after fine-tuning against established multimodal benchmarks. Its architecture enhances contextual learning and multi-image reasoning, streamlining prompt engineering with fewer samples. While MM1’s performance has yet to exceed that of Google’s Gemini or OpenAI's GPT-4, it represents a notable advancement for Apple in the AI landscape.

Additionally, Apple has acquired AI startup DarwinAI, recognized for developing smaller and faster AI systems vital for efficient on-device processing. Following the acquisition, DarwinAI’s online presence has ceased, with many employees joining Apple’s AI division. Alexander Wong, an AI researcher from the University of Waterloo, has also joined Apple as director of the AI team, further reinforcing Apple's commitment to AI innovation.