D-ID Introduces Advanced AI Video Translation Tool Featuring Voice Cloning and Lip Sync Technology

AI video creation platform D-ID has launched an innovative tool that leverages AI technology for translating videos into multiple languages. This platform not only translates the dialogue but also replicates the speaker’s voice and syncs their lip movements to match the translated script.

D-ID's new technology builds on their previous success, notably seen during the viral trend when users animated old family photos that eventually could speak. Following this success, D-ID secured $25 million in Series B funding in 2022, focusing on expanding its offerings for enterprise clients in the U.S. who utilize AI-driven video content.

With the introduction of AI Video Translate, currently free for D-ID subscribers, creators can now easily adapt their videos into 30 different languages, including Arabic, Mandarin, Japanese, Hindi, Spanish, and French. A D-ID subscription starts at $56 per year for basic access and limited credits, scaling up to $1,293 annually for more extensive features before transitioning to enterprise-level pricing.

D-ID's AI video technology aims to reduce localization costs for businesses aiming to reach global audiences in sectors such as marketing, entertainment, and social media. This technology stands to compete with other dubbing and AI video solutions.

Traditionally, dubbing technologies have empowered viewers to enjoy content in their preferred language, but access has often been limited for smaller creators. That landscape is evolving, as major companies expand accessibility to cutting-edge technologies. YouTube, for instance, recently launched a multi-language audio feature to help creators reach broader audiences; well-known creator MrBeast (Jimmy Donaldson) was among the first to adopt this, translating several of his popular videos into 11 languages.

AI capabilities are enhancing the functions of voice creation, translation, and cloning. Microsoft announced plans to incorporate AI for real-time translation and dubbing of YouTube videos and more. In July, Vimeo introduced tools to translate audio and captions while replicating the speaker's voice through AI technology.

Numerous companies, including Descript, ElevenLabs, and Speechify, offer services in voice cloning or AI translation. Platforms like HeyGen and Deepbrain AI provide solutions that allow users to create videos featuring AI avatars fluent in multiple languages.

AI libraries for dubbing and lip sync, such as Wav2lip, have further simplified the creation of these tools, making them more accessible and potentially cost-effective. D-ID’s proprietary model, Rosetta-1, powers this new Video Translate feature.

The Video Translation technology will be accessible through D-ID Studio and its API, with a one-month trial available for potential users. The platform supports videos ranging from 10 seconds to 5 minutes, with a maximum file size of 2GB. The feature works best with a single individual in the frame, ideally facing the camera with their face clearly visible at all times.