OpenAI SearchGPT Official Demo Exposes Vulnerability: Uncovering the Secrets Behind the Source Code and Search Mechanism

Title: Challenges Facing SearchGPT: The Battle Between Illusion and Reality

Just two days after the launch of SearchGPT, a demonstration created by user Kesku sparked considerable buzz online, mainly due to its astonishingly fast output results. However, an official demonstration released by OpenAI was scrutinized in a report by The Atlantic, revealing some significant inaccuracies. When asked about "the music festival in Boone, North Carolina, in August," SearchGPT surprisingly provided the wrong date, raising concerns about its reliability.

OpenAI spokesperson Kayla Wood confirmed the error to The Atlantic, stating that this was an initial prototype and improvements are underway. This incident draws parallels to a major blunder made by Google's Bard, which also faced criticism for inaccuracies upon its launch. In February 2023, Bard mistakenly claimed that the James Webb Space Telescope had captured the first image of an exoplanet, a feat actually achieved by the European Southern Observatory's VLT. This misstep resulted in a 9% plunge in Alphabet's stock price, wiping out $100 billion in market value.

In contrast, OpenAI has opted for a more cautious approach by limiting access to internal testing, learning from Google's misfortune. Given the context of widespread access, even if OpenAI manages to minimize the occurrence of hallucinations in SearchGPT, a mere 1% error rate could lead to millions of inaccurate responses daily. Moreover, there currently are no reliable methods to entirely eliminate hallucinations and errors in large language models (LLMs).

Andrej Karpathy has pointed out on Twitter that hallucinations are not mere bugs, but rather a distinct characteristic of LLMs. He likens LLMs to “dream machines,” which, when prompted, create content that is often helpful but can inadvertently drift into factual errors, resulting in "hallucinations." This mechanism is fundamentally different from traditional search engines, which return the most relevant documents from their databases without creating entirely new responses.

Karpathy believes that the current AI search models based on LLMs cannot guarantee 100% accurate results. This raises an intriguing question: In the transformation of search engines, will the creativity of LLMs coexist with the reliability of traditional search methods, or will one ultimately replace the other? This question warrants careful consideration.