Pika Launches AI-Powered Lip Sync Feature, Igniting Competition in the Video Industry

OpenAI continues to captivate audiences with its AI video generation model, Sora, impressively showcasing new demo examples. However, access remains limited for the public. In the meantime, competitor Pika has introduced an exciting feature for its subscribers: Lip Sync.

This innovative add-on allows users to add spoken dialogue to their videos using AI-generated voices from ElevenLabs, a separate generative audio startup. It also synchronizes mouth movements with the dialogue, enhancing the realism of the speaking characters.

The Pika Lip Sync feature offers both text-to-audio conversion and the option to upload audio tracks. Users can type or record dialogue for their Pika AI-generated characters, allowing them to customize voice styles.

Currently, Lip Sync is in "early access," available exclusively to Pika Pro users ($58 per month, billed annually at $696) and members of Pika’s invitation-only "Super Collaborators" program via Discord.

Breaking Barriers for AI in Filmmaking

While Pika’s AI-generated videos may not achieve the same realism as OpenAI’s Sora or rival Runway, the Lip Sync feature marks a significant advancement. It provides a solution for one of the remaining challenges in using AI for long narrative films, as most leading AI video generators lack a similar native feature.

Traditionally, users have relied on cumbersome third-party tools and labor-intensive post-production methods to add dialogue and synchronize lip movements, resulting in a low-budget appearance.

In related news, Runway has updated its Multi Motion Brush feature, originally launched last month. This tool allows users to apply up to five independent motion directions to different objects in videos, such as a dog jumping for a frisbee. The update includes region detection, enabling automatic selection of objects for motion application, while manual “painting” options remain available.

Pika also enables users to edit video components and expand the canvas, though it currently lacks a brush tool, resulting in less precise motion controls compared to Runway.

Ongoing Concerns Surrounding AI Training Data

Despite the excitement surrounding Pika’s new feature, questions persist regarding the training data used for its video model. Ed Newton-Rex, CEO of the AI certification nonprofit Fairly Trained, raised these concerns on social media, emphasizing the need for consent from creators and data owners during AI training.

As video AI generator companies continue to innovate and enhance their offerings, the competition intensifies, benefitting users but alarming some professionals in the filmmaking industry. Notably, filmmaker Tyler Perry faced backlash for halting an $800 million studio expansion after witnessing Sora-generated videos, expressing concerns over potential job losses due to this technology.