Snap Unveils Real-Time Image Model for Creating Engaging AR Experiences

At the Augmented World Expo on Tuesday, Snap revealed an early version of its innovative real-time, on-device image diffusion model, designed to create immersive augmented reality (AR) experiences. The company also introduced new generative AI tools tailored for AR creators.

Snap co-founder and CTO Bobby Murphy took the stage to highlight that this model is compact enough to function on smartphones and quick enough to re-render frames instantly based on a text prompt.

Murphy emphasized that while the advent of generative AI image diffusion models is exciting, these models must operate at significantly faster speeds to effectively enhance augmented reality. This necessity has driven Snap’s teams to focus on advancing machine learning models.

In the coming months, Snapchat users can expect to see Lenses powered by this generative model, with plans to roll out these features to creators by year-end.

“This and future real-time, on-device generative machine learning models represent an exciting new frontier for augmented reality, enabling us to rethink how we conceptualize rendering and crafting AR experiences,” Murphy stated.

Additionally, Murphy announced that Lens Studio 5.0 is launching today, equipping developers with new generative AI tools that will streamline the creation of AR effects, potentially saving them weeks or even months of effort.

AR creators can now quickly develop selfie Lenses using highly realistic machine learning face effects. They can also apply custom stylization effects that transform the user’s face, body, and environment in real time. Furthermore, creators will be able to generate 3D assets in mere minutes for incorporation into their Lenses.

Moreover, using Snap’s Face Mesh technology, AR creators can create characters like aliens or wizards based on text or image prompts and generate face masks, textures, and materials in just minutes.

The latest iteration of Lens Studio also introduces an AI assistant, ready to address any questions AR creators may have.