Google Experiments with Video Search Capabilities Powered by Gemini AI

Google has acknowledged that video platforms such as TikTok and Instagram are significantly impacting its core Search product, especially among the younger Gen Z demographic. In response, the company is enhancing the integration of video searches within Google Search, leveraging the capabilities of Gemini AI. At the Google I/O 2024 developer conference on Tuesday, Google revealed an innovative feature allowing users to upload a video along with a text query, providing an AI-generated overview of the relevant answers.

Initially, this feature will be available as an experiment in Search Labs for English-speaking users in the U.S.

This new multimodal capability enhances an existing feature that permits users to combine text with visual searches. Launched in 2021, this functionality has been particularly useful for queries that involve visual elements that are difficult to articulate or when items can be described in various ways. For instance, if you have a photo of a shirt you like, you could use Google Lens to find similar patterns on associated products, such as socks.

With the introduction of video search, Google is responding to the preferences of younger users, who increasingly communicate and express themselves through video on their smartphones. It stands to reason that they would also prefer utilizing video for searches.

The new feature enables users to upload a video and pose a question to generate a search query. During a demonstration, Google presented a scenario involving a malfunctioning record player, focusing on the arm that wouldn't stay positioned on the record. The query featured a video of the issue along with the question, “Why will this not stay in place?” Google’s Gemini AI reviews the video frame by frame to identify the problem and then provides an overview of potential solutions.

For those seeking further information, Google also offers links to discussion forums and instructional videos on topics such as how to rebalance the arm on your turntable.

While Google showcased its capability to interpret video content in conjunction with search queries, this advancement has broader implications, including video analysis on users' devices, those stored in private cloud services like Google Photos, and videos shared on platforms like YouTube.

The company has not specified the duration of testing for the new feature in the U.S. or when it might expand to additional markets.

Stay tuned for the launch of our AI newsletter! Sign up here to receive the latest updates directly in your inbox.