Video Editing Startup Captions Unveils Lipdub: A Multilingual Dubbing App Supporting 28 Languages

Captions, an innovative AI-driven video editing startup, has introduced a new app called Lipdub, designed to translate videos into 28 languages seamlessly.

Available for free on the App Store, Lipdub supports a variety of languages, including French, Hindi, Spanish, Italian, Portuguese, Japanese, and more. Unique features allow users to translate videos into engaging styles, such as Texas slang, Gen Z speak, pirate lingo, and even baby talk. A demonstration video showcases the app's ability to adjust lip movements in sync with the chosen language, although some users have noted occasional audio-lip movement lag.

Users can translate a video featuring a single speaker for up to one minute and easily share it across various social media platforms. Captions reports that over 3 million creators have utilized its original video editing app, which boasts more than 100,000 active daily users. The Captions app is packed with powerful AI-enhanced features, including the ability to eliminate filler words like "ums" and "ahs," reduce background noise, and enhance audio clarity. It also includes the “AI Lipdub” feature for post-production editing, allowing users to modify lip movements based on transcript changes.

Founded in 2021 by Gaurav Misra, the former head of design engineering at Snap, Captions recently secured $25 million in a Series B funding round led by Kleiner Perkins, with contributions from Sequoia Capital, Andreessen Horowitz (a16z), and SV Angel. To date, the company has amassed a total funding of $40 million.

The integration of translation and AI dubbing to broaden audience reach is an emerging trend in the digital landscape. For example, earlier this year, YouTube began testing an AI-powered tool that enables users to automatically dub their videos in multiple languages, with ongoing efforts to enhance lip-syncing accuracy. Last month, the platform announced plans to incorporate AI dubbing directly into YouTube Studio, simplifying the video translation process for creators.

Moreover, AI voice-generation platform ElevenLabs recently launched its dubbing tool, capable of supporting 29 languages. According to Rest of World, dubbing service providers are earning millions by translating content for popular YouTubers like MrBeast.

The surge in interest surrounding AI dubbing startups is notable, with companies such as U.K.-based Papercup and Israel-based Deepdub also securing substantial investments.