Meta AI Introduces Lip-Synced Translations Featuring Celebrity Voices, Including Judi Dench and John Cena

Meta's AI assistant, Meta AI, is introducing an exciting new voice feature.

During the Meta Connect 2024 developer conference held in Menlo Park on Wednesday morning, Meta revealed that Meta AI can now verbally respond to queries across its platforms: Instagram, Messenger, WhatsApp, and Facebook. Users can select from a variety of voices, including celebrity AI clones like Awkwafina, Dame Judi Dench, John Cena, Keegan-Michael Key, and Kristen Bell.

Unlike OpenAI’s Advanced Voice Mode for ChatGPT, which offers rich emotional expressiveness, Meta AI's new voice feature is more similar to Google’s Gemini Live, which listens to speech, processes it, and then reads the response using a synthetic voice.

Meta is banking on the allure of high-profile talent to make an impact; as reported by The Wall Street Journal, millions were spent to secure the celebrity likenesses. While we're intrigued, we'll withhold our final thoughts until we experience it firsthand.

In addition to the new voice feature, Meta AI can now analyze images, thanks to advancements in the AI models driving the platform. In supported regions, users can share photos—like a blooming flower—and ask Meta AI to identify it. Alternatively, you can upload a picture of a dish and request a recipe. (Just remember, there may be occasional inaccuracies in the responses.)

Furthermore, Meta is testing a translation tool for voice content in Instagram Reels. This tool automatically translates creators' voices, providing dubbed audio while aligning lip movements in another language.

Meta is currently running "small tests" for this translation feature within select creators’ videos from Latin America, offering English and Spanish content in the U.S.