Apple is taking a significant leap into generative AI with its new offering, Apple Intelligence, which is designed to benefit not just consumers but developers as well. During its keynote address at WWDC 2024 on Monday, Apple unveiled that developers could seamlessly integrate experiences powered by Apple Intelligence into their apps.
To facilitate this, Apple has updated its software development kits (SDKs) with a range of new APIs and frameworks. This enhancement allows app developers to incorporate features like the Image Playground—enabling AI-generated image creation—with minimal coding effort. For instance, Apple showcased how an app like Craft could enhance user documents by incorporating AI-generated visuals effortlessly.
Additionally, AI-driven writing tools will be readily available in any application utilizing the standard editable text view. Apple illustrated this with Bear Notes, demonstrating how the app would automatically enable users to rewrite, proofread, and summarize their notes.
Moreover, Apple is enhancing the functionality of Siri within apps. Developers already utilizing SiriKit—an SDK for integrating Siri capabilities—will automatically benefit from new features without any extra work. This enhancement applies to various functionalities, including Lists, Notes, Media, Messaging, Payments, Restaurant Reservations, VoIP Calling, and Workouts.
During the Developer keynote, Apple highlighted two exciting new Siri capabilities for developers. First, Siri will be able to activate any menu item within an app, allowing users to say phrases like “show my presenter notes” or a more casual “I need to see my speaker notes.” Secondly, Siri can now access any text displayed on the screen using Apple’s standard text systems, enabling users to take immediate action. For example, if a reminder states “wish grandpa a happy birthday,” users can simply command, “FaceTime him” to act on that note.
In addition, Apple’s App Intents framework will gain access to Apple Intelligence, allowing developers to define new intents across various categories. This initial rollout includes areas such as Books, Browsers, Cameras, Document Readers, File Management, Journals, Mail, Photos, Presentations, Spreadsheets, Whiteboards, and Word Processors. Apple asserts that these intents are designed and tested for ease of adoption by developers.
With these intents, a photo editing application like Darkroom can utilize the "Apply Filter" intent, allowing users to say, “Apply a cinematic preset to the photo I took of Ian yesterday.” The capability will continue to expand to more domains over time.
Initially, users can start development with the Shortcuts app, and as Siri evolves, it will be able to leverage app intents in supported domains. Apple also mentioned that apps compatible with existing SiriKit domains will benefit from improved conversational features, such as responding accurately to user prompts or comprehending references made earlier in the conversation.
Siri will gain an enhanced data search capability from apps using a new Spotlight API, allowing app entities to be included in its index. This includes a semantic index of items like photos, messages, files, and calendar events.
On the same day, Apple introduced its own password manager app, AI-generated Bitmoji, and a Calculator specifically for iPad users.
This post has been updated to include further details from the Developer keynote.
Ranking
