As denizens of the Internet, we have all often seen a news item so ridiculous it caused us to think, “This seems like an Onion headline.” But as real human beings, most of us have the ability to discern between reality and satire. Unfortunately, Google’s newly launched “AI Overview” lacks that crucial ability. The feature, which launched less than two weeks ago (with no way for users to opt-out), provides answers to certain queries at the top of the page above any other online resources. The artificial intelligence creates its answers from knowledge it has synthesized from around the web, which would be great, except not everything on the Internet is true or accurate. Obviously.
Ben Collins, one of the new owners of our former sister site, pointed out some of AI Overview’s most egregious errors on his social media. Asked “how many rocks should I eat each day,” Overview said that geologists recommend eating “at least one small rock a day.” That language was of course pulled almost word-for-word from a 2021 Onion headline. Another search, “what color highlighters do the CIA use,” prompted Overview to answer “black,” which was an Onion joke from 2005.
Writer Parker Molloy conducted further tests that found Google’s AI simply couldn’t differentiate between news and satire. But other examples suggest the tool doesn’t know how to sift out bad information, period. One viral search on how to keep cheese from sliding off a pizza got a response to try using non-toxic glue, apparently culled from a decade-old Reddit comment. (Katie Notopoulos of Business Insider actually made and ate the glue pizza—you can follow her journey here.) Some answers are predictably more sinister. For instance, one user searched “How many Muslim presidents has the US had?” and got result, “The United States has had one Muslim president, Barack Hussein Obama.”
In a statement to CNBC, a Google spokesperson defended the product. “The vast majority of AI Overviews provide high quality information, with links to dig deeper on the web,” they said. “Many of the examples we’ve seen have been uncommon queries, and we’ve also seen examples that were doctored or that we couldn’t reproduce.” The company asserted that AI Overview has undergone extensive testing and that Google will take “swift action where appropriate under our content policies.”
In other words, the goofy viral searches you’re seeing all over social media have probably already been wiped. As the “experiment” continues, however, we won’t be surprised to see more disturbingly inaccurate search results cropping up. Sure, AI is supposed to learn and improve the more you use it, but if the Internet is increasingly filled with junk and nonsense, what is AI Overview going to learn? The whole situation feels ripped from the headlines—The Onion headlines, that is. Sometimes reality is stupider than satire.