Exclusive: Stability AI Unveils Advanced 3D and Image Fine-Tuning Features for Stable Diffusion

Stability AI today unveiled several exciting enhancements to its Stable Diffusion platform, significantly expanding its capabilities in text-to-image and entering the realm of 3D content creation.

The standout feature of this update is the introduction of the Stable 3D model. Previously focused on two-dimensional (2D) image generation, Stable Diffusion now supports a wide range of 3D content creation applications, including graphic design and video game development.

In addition, Stability AI has launched the Sky Replacer tool, specifically designed to replace the sky in 2D images seamlessly. The Stable Diffusion platform also introduces Stable Fine-Tuning, aimed at helping enterprises accelerate image fine-tuning for specific use cases. To enhance content authentication, images generated via the Stability AI API will now include an invisible watermark, ensuring greater integrity in creative workflows as generative AI becomes increasingly integral to various industries.

“It’s about equipping creative storytellers with tools for enhanced control over visuals,” said Emad Mostaque, CEO of Stability AI, in an exclusive interview.

Navigating a Competitive Generative AI Landscape

Stability AI's advancements arrive amid fierce competition in the text-to-image generation sector. Adobe's Firefly tools, integrated with its design software, and OpenAI's recently launched DALL-E 3 models both represent significant steps in the market. Midjourney is also ramping up its feature offerings to assist designers in image generation.

Mostaque recognizes the competitive landscape and aims to differentiate Stability AI by focusing on enabling comprehensive creative pipelines rather than solely providing models. The new Sky Replacer and Fine-Tuning features represent significant advancements beyond basic image generation capabilities.

Sky Replacer: A Practical Business Tool

Replacing image backgrounds is not a new concept; however, Stability AI is automating this process to enhance efficiency for business users. Mostaque highlights that the Sky Replacer tool has specific applications, particularly in industries like real estate.

Users desire various backgrounds and lighting effects, and Stability AI is focused on creating optimized workflows that offer organizations greater control tailored to their specific needs. “Sky Replacer is the first of many tools we’ll introduce that cater to industry-specific requirements, reflecting our experiences over the past six to twelve months,” he noted.

Stable 3D: New Possibilities for Creators

The new Stable 3D model enhances the diffusion model used in Stable Diffusion by incorporating additional 3D datasets and vectorization techniques. Mostaque expressed his excitement about the potential for creating immersive 3D worlds, built upon the foundation of Stable Diffusion and the Objaverse-XL dataset, one of the largest 3D databases globally.

While traditional 3D image creation methods can be resource-intensive, Mostaque believes that Stable 3D will streamline this process, making it more efficient. Initially available as a private preview, Stable 3D is expected to evolve and improve over time.

“This approach is incredibly efficient compared to classical 3D model creation,” he stated. “What once took considerable time to produce can now be done more quickly.”

Integrating Watermarks for Content Authenticity

In light of the Biden Administration's recent Executive Order on AI, which calls for the incorporation of watermarks in generated content, Stability AI is implementing invisible watermarks and Content Credentials within its API. This initiative, part of a broader industry effort led by Adobe and others, aims to ensure clear authorship of generated content. Mostaque emphasized that these measures are part of Stability AI's commitment to fostering authenticity in digital creation.

“We are pioneering initiatives to determine the authenticity of generated content, and our watermark integration is a step toward achieving that goal,” he affirmed. This development also supports future attributions and mechanisms aimed at enhancing content verification.