Meta Launches Testing of GPT-4V Competitor: A New Multimodal AI in Ray-Ban Smart Glasses

Meta Platforms, the parent company of Facebook, Instagram, WhatsApp, and Oculus VR, is making waves this week with the introduction of a new multimodal AI for its Ray-Ban Meta smart glasses. Following the release of its voice cloning AI, Audiobox, Meta has begun a beta trial in the U.S. to test this innovative AI technology.

According to Andrew Bosworth, Meta's Chief Technology Officer, the multimodal AI is set for a public launch in 2024. In an Instagram video, he shared, "Next year, we’re launching an AI assistant that utilizes the glasses' camera to provide information not only based on user queries but also about the surrounding environment." This beta testing program is currently underway, although details on how to participate have not been disclosed.

The Ray-Ban Meta smart glasses, priced at $299, were showcased at Meta's annual Connect conference in September. The latest models already include a basic AI assistant that responds to voice commands, similar to Amazon’s Alexa or Apple's Siri, but it lacks the capability to intelligently analyze live video or photos.

In his post, Boz illustrated the AI's potential by wearing the glasses while observing a lighted piece of wall art. He queried the AI, which accurately identified the artwork as a “wooden sculpture” and described it as “beautiful.” This suggests that the AI may also rely on a connected smartphone for enhanced performance and interaction.

Meta CEO Mark Zuckerberg also demonstrated the AI's capabilities, showcasing its ability to recommend matching clothing items, generate witty captions, recognize a fruit he was holding, and translate text from Spanish to English.

The move highlights Meta's commitment to integrating AI into its products and promoting open-source AI through its Llama 2 model. In a landscape where generative AI has slowly transitioned towards hardware, Meta's initiative stands out, especially when compared to previous attempts, like Google Glass, which faced challenges with usability and public perception.

Will Meta's multimodal AI for Ray-Ban smart glasses overcome the pitfalls experienced by earlier smart glasses? As technology evolves and societal attitudes shift, only time will tell if this innovation finds its place in the market.