Apple Unveils Apple Intelligence: The New Multi-Modal Generative AI Service for Mac, iPhone, and iPad Users

Today at Apple Inc.’s annual Worldwide Developers Conference (WWDC 2024) in Cupertino, California, the company unveiled Apple Intelligence, its most ambitious generative AI initiative to date. This highly anticipated service combines various AI models to deliver personalized, private, and secure features across Mac, iPhone, and iPad devices.

“It’s aware of your personal data without collecting it,” stated Craig Federighi, Apple’s Senior Vice President of Software Engineering, during the keynote. He emphasized that the service prioritizes privacy and security by operating on-device and using private clouds, depending on the model employed. “You are in control of your data, where it is stored, and who can access it,” he added.

Seamless Integration with Apple Platforms

Apple Intelligence is intricately woven into iOS 18, iPadOS 18, and macOS Sequoia. It utilizes the power of Apple silicon to comprehend and generate language and images, facilitate actions across applications, and leverage personal context—streamlining daily tasks. The service will be freely available on these platforms later this summer.

Custom Emoji and Image Creation Tools

The service employs large language models (LLMs) to analyze and generate text, alongside diffusion models used in popular image generators like Stable Diffusion and Midjourney. A standout feature for consumers is "Genmoji," a generative AI tool that allows users to create custom emoji in Messages based solely on text input, even representing their friends and family from Contacts.

In addition, Apple is launching "Image Playground," a generative AI image creation app that integrates with Messages and Keynote. Users can generate imagery from text prompts—perfect for crafting invitations for gatherings. Image Playground offers three styles—Animation, Illustration, or Sketch—allowing users to choose concepts from various categories and include people from their photo library.

Advanced Features in Notes and Photos

Apple Intelligence enriches the Notes app with "Image One," capable of analyzing notes and creating corresponding images. Meanwhile, the Photos app has been upgraded to allow users to create smart photo albums by typing in descriptions, like “child learning to fish.” The app scans the user’s library to find and curate matching photos with appropriate music.

The Photos app now also features advanced search capabilities for natural language queries such as “Maya skateboarding in a tie-dye shirt.” The new Clean Up tool can remove distracting background objects from photos.

Enhanced Siri Functionality

Apple Intelligence enhances Siri with new functionalities, including tools for rewriting, summarizing, and tailoring recommendations across all apps. Siri can also generate images based on Contacts’ photos and autonomously perform app actions.

Siri, now 14 years old, has received significant updates, including the ability to accept dictated messages for later sending via the “Send Later” feature. It can automatically add addresses from Apple Messages to Contacts based only on voice commands, and enhance photos with commands like “Make this photo pop!” Siri now maintains context between requests, assists users who stumble over words, and types responses for those who prefer text communication. Additionally, a new design features a glowing light wrapping around the screen when Siri is active.

For users seeking help with device functions, Siri can now answer thousands of questions about performing various tasks on iPhone, iPad, and Mac and execute many new actions across Apple and third-party apps.

In summary, Apple introduced an extensive array of new AI-powered features, positioning itself prominently in the competitive landscape of generative AI technology. With over 2.2 billion devices in circulation globally, even modest adoption rates of these features will likely establish Apple as a leading generative AI provider, marking a significant chapter in the evolution of this technology.