If you tuned into the Super Bowl last night and sat through the commercials, you likely encountered Microsoft's advertisement for its new Copilot AI app.
Billed as "your everyday AI companion," Copilot is more than just a search tool; it enables users to create images and game code using advanced generative AI large language models (LLMs). This Super Bowl ad has seemingly succeeded, as Copilot became the second most downloaded app in the Apple App Store and ranked number 12 in the Google Play Store for Android devices at the time of writing.
However, Microsoft’s AI journey isn't without challenges. Some users on X (formerly Twitter) reported that both Copilot and Google's rival, Gemini, provided outdated and incorrect information regarding the Super Bowl and its participating teams, the San Francisco 49ers and the Kansas City Chiefs. In contrast, a standard Google Search accurately delivered the teams' details.
For instance, Copilot mistakenly listed Tyreek Hill, a player for the Miami Dolphins, as still being part of the Kansas City Chiefs, highlighting a potential knowledge cutoff issue.
Although I found Copilot yielding accurate responses just moments ago, this inconsistency is a common flaw among LLMs, where similar prompts can generate vastly different answers for different users.
While the surge in interest surrounding Microsoft’s Copilot app is promising for its AI goals, the app's accuracy and reliability will ultimately determine user retention. Microsoft now has a crucial opportunity to harness the public curiosity sparked by its Super Bowl ad—how it navigates this moment will significantly impact Copilot’s future success.