Google Assistant Now Enhanced by Gemini — Here’s What You Need to Know

Google Assistant, as we once knew it, is undergoing a significant transformation—at least in terms of its core functionality.

Last year, Google enhanced Google Assistant by integrating AI features from its Bard chatbot, enabling the virtual assistant to interpret images and access information from emails, documents, and more. Now, the tech giant is giving users the opportunity to swap out the AI models that power Google Assistant’s conversational abilities for Gemini, its latest generation of AI technology.

Soon, users will experience a reimagined Google Assistant powered by Gemini through a new app for Android devices, which will replace the current Google Assistant app. An update for the Google app on iOS will also roll out in the coming weeks. It’s important to note, however, that this new experience will be mobile-only, with no immediate changes to Google Assistant devices like Nest or Home speakers.

The revamped experience will feature an overlay that serves as a user interface for the Gemini models, providing contextual recommendations and personalized suggestions. Sissie Hsiao, Google Assistant VP, explained in a recent briefing, "This overlay allows you to engage in dynamic interactions with Gemini. For instance, you can generate captions for photos you've taken, or delve deeper into an article and ask questions about its content."

The Gemini-powered Assistant—accessible via a corner swipe, power button, or “Hey Google” command—can handle both images and text commands. For example, if you upload a picture of a newly purchased plant and ask for care tips and related YouTube videos, the Gemini-powered Assistant will strive to meet your request.

If you've engaged with the Gemini model via Bard on the web, you likely recognize its wide array of AI capabilities—which remain intact with this new integration. In the onboarding process, Google provides example prompts such as, “Help me draft a text for a friend who’s stressed at work,” “Suggest surprise ideas for my concert-loving friend’s birthday,” and “Assist me in adding more plant-based meals to my diet.” Like its web counterpart, the Assistant can also create images, though the model used remains unspecified; Bard originally utilized Imagen 2 for this purpose.

As you might expect, conversations with the Gemini-powered Assistant on mobile will seamlessly sync with web experiences—provided you are logged into the same Google Account.

If the new Gemini-powered experience doesn't meet your expectations, users can easily switch back to the original models at any time through the settings in the Assistant app on Android or the Google app on iOS.

It's worth noting that the transition to Gemini models isn’t entirely a wholesale replacement of the previous models. Features such as quick phrases, which allow users to skip saying “Hey Google” for certain tasks, will continue to rely on the older models. Basic functions like setting timers, making calls, and controlling smart home devices will also remain unchanged. Additionally, Google indicates that even if you choose Gemini as your default model in Assistant, apps like Google Maps that are integrated with Assistant may still utilize the older models.

In a move that might frustrate dedicated Google Assistant users, the most advanced Gemini model, Gemini Ultra, will not be available for free. This model, which offers enhanced reasoning, coding, and instruction-following capabilities, is part of a new subscription service, the Google One AI Premium Plan, priced at $20 per month. The standard Gemini-powered Assistant will rely on the less advanced Gemini Pro model, which received a lukewarm response when it debuted with Bard in December.

Google is promoting the Google One AI Premium Plan—an upgrade of its existing Google One program, currently featuring a two-month free trial—as the exclusive pathway to access its leading GenAI features through Assistant. To gain future updates for Gemini Ultra, which will include expanded multimodal capabilities and improved interactive features, users must maintain their subscription.

To enhance the value proposition, the AI Premium Plan includes 2TB of storage and, in the near future, will introduce Gemini-powered functionalities across Gmail, Google Docs, Google Slides, Google Sheets, and more. However, it remains to be seen whether this will appease users who feel that these costly new AI features should have remained free.

Starting today, the new Gemini-powered Assistant is available in English in the U.S., with plans for expansion to other locations and additional languages—Japanese and Korean—beginning next week.

Most people like

Find AI tools in YBX