Searching on Google, the world’s largest search engine with more than 90% of the global market, looks a little different now. Instead of a page of links to other sites, Google automatically generates a summary at the top of the page, citing a handful of select websites.
The changes were announced last week, with Google rolling out the AI summaries to its 250 million-plus monthly U.S. users. But now, some users are reporting the new tech is getting important information wrong.
Though Google said that early testers were more satisfied with AI overview search results, a wider audience has already spotted inaccuracies, or hallucinations, with the feature.
When an X user typed in a search query “cheese not sticking to pizza,” Google’s AI Overview answered, “You can also add about ⅛ cup of non-toxic glue to the sauce to give it more tackiness.”
https://t.co/W09ssjvOkJ pic.twitter.com/6ALCbz6EjK
— SG-r01 (@heavenrend) May 22, 2024
Google’s source for the answer appears to be a Reddit post written 11 years ago.
“This example is funny, but there can be dangerous ones that aren’t as obvious,” wrote the X user who flagged the issue.
Google inked a deal with Reddit in February to train its AI. OpenAI signed a similar agreement with Reddit earlier this month.
Related: OpenAI’s New Deal Sees the ChatGPT Trailblazer Following a Competitor’s Lead
Seattle-based AI researcher Dr. Margaret Mitchell also pointed out two AI hallucinations on X on Wednesday.
The first showed that Elon Musk went to UC Berkeley. At the time of writing, the error appears to be corrected.
The second showed that President Andrew Jackson, who died in 1845, graduated from the University of Wisconsin-Madison in 2005.
President Andrew Johnson, who never went to school and taught himself how to read, earned 14 degrees from UW-Madison, according to the AI overview.
Meanwhile, over in Google Search.
Andrew Johnson has been killin it, I never knew. pic.twitter.com/IV2zCmI6Zv— MMitchell (@mmitchell_ai) May 22, 2024
Entrepreneur verified that the second error was still present at the time of writing.
Inaccurate on multiple counts. Credit: Entrepreneur
Google’s AI appears to draw from a 2016 UW-Madison alumni association article showing the graduation years of alumni with presidential names.
Related: Oh Great, AI Can Detect Sarcasm Now
The errors illustrate how AI could fail to pick up on nuance in training data and potentially amplify misinformation, especially when integrated into a widely used platform like Google.
Anastasia Kotsiubynska, Head of SEO at SE Ranking, foreshadowed the hallucinations in a remark shared with Entrepreneur last week.
“Most likely, there will still be misleading information in search results and hallucinations, and many users will probably use this information without double-checking,” Kotsiubynska said at the time.
Related: Site Traffic Down? Here Are the Big AI Changes Google Made to Its Search Tool
Google could start testing ads within AI overviews “soon,” the company announced Tuesday, adding another layer to its new AI search experience.
