ElevenLabs Develops a Universal AI Dubbing Machine for Seamless Multilingual Content Creation

After Disney releases a new film in English, the company undertakes the considerable task of localizing it into as many as 46 global languages to reach a broader audience. This effort is managed by Disney Character Voices International Inc, dedicated to making movies accessible worldwide. Unlike traditional dubbing, where local voice actors bring the characters to life, a new "AI dubbing" system from ElevenLabs promises to closely replicate the original actors' voices, regardless of the language spoken on-screen.

ElevenLabs, an innovative AI startup, provides a voice cloning service that enables subscribers to create nearly identical vocalizations using just a few minutes of audio samples. Following the feature's beta release, it quickly became a tool for impersonating celebrities, sometimes without their knowledge. However, the new AI dubbing capability is designed for legitimate use, permitting over 20 languages, including Hindi, Portuguese, Spanish, Japanese, Ukrainian, Polish, and Arabic.

This technology enables media companies, educators, and content creators without vast budgets to adapt their work for global audiences. ElevenLabs claims that the system can translate spoken content into another language within minutes and even generate new dialogue in the target language while mimicking the original actor’s voice. Importantly, it can maintain the "emotion and intonation" of the original dialogue, enhancing the viewer's experience.

"It will help audiences enjoy any content they want, regardless of the language they speak," said ElevenLabs CEO Mati Staniszewski. "And it will mean content creators can seamlessly and authentically reach a far larger audience worldwide."