Google AI Overviews gives wrong and misleading answers

Google’s “AI Overviews” feature was criticised for providing factually inaccurate, misleading and nonsensical answers. 

The AI-generated summaries, which appear at the top of Google’s search results, were found to contain basic factual errors. These include treating jokes as facts, such as suggesting using “1/8 cup of non-toxic glue” to stop cheese from sliding off pizza, which was traced back to a troll post. 

Another example is the system incorrectly stating that “astronauts have met cats on the moon” and that "Neil Armstrong said, ‘One small step for man’ because it was a cat’s step".

Google defended the system, saying it works as intended for most queries and that many of the problematic examples are uncommon or had been doctored. However, experts expressed concerns about the potential for the system to perpetuate bias and misinformation.

These errors are seen as particularly problematic given the prominence of Google’s search results page.

News, commentary, analysis 🗞️

• https://www.theverge.com/2024/5/23/24162896/google-ai-overview-hallucinations-glue-in-pizza

• https://edition.cnn.com/2024/05/24/tech/google-search-ai-results-incorrect-fix/index.html

• https://arstechnica.com/information-technology/2024/05/googles-ai-overview-can-give-false-misleading-and-dangerous-answers/

• https://www.nytimes.com/2024/05/24/technology/google-ai-overview-search.html

• https://searchengineland.com/google-ai-overview-fails-442575

• https://www.euronews.com/next/2024/05/27/googles-new-ai-summaries-tool-causes-concern-after-producing-misleading-responses

• https://www.technologyreview.com/2024/05/31/1093019/why-are-googles-ai-overviews-results-so-bad/