AI Overviews Odd Answers: What Google Changed and What It Means for You

If you have seen the screenshots of AI Overviews telling people to eat rocks or put glue on pizza, you are probably asking two things. Can I trust these answers, and what does this mean for my site? On 30 May 2024, Google explained where the AI Overviews odd answers came from and listed the changes it has made. The short version: Google has tightened when AI Overviews show and which sources they lean on, but the feature is staying.

In this article I cover what went wrong, what Google says caused it, the exact changes it made, and what I think site owners and marketers should do now.

What Happened in the Two Weeks After Launch

Google rolled out AI Overviews to everyone in the US after announcing them at Google I/O on 14 May 2024. I covered the launch itself in my article on what AI Overviews mean for clicks. Within days, people started sharing strange answers on social media.

The examples that spread most widely included:

• A suggestion to add glue to pizza sauce so the cheese sticks better.
• An answer to “How many rocks should I eat?” that treated the question as serious.
• Other clearly wrong claims, such as getting simple dates and facts wrong.

Some of the screenshots were real. Others were not. Google says a number of the most shocking ones, about leaving dogs in hot cars, smoking while pregnant and depression, were faked and never appeared in AI Overviews.

Google said it was removing specific bad answers under its content policies as it found them. Then, on 30 May, Liz Reid, who leads Google Search, published a longer post explaining the problem and the fixes.

Why AI Overviews Gave Odd Answers

The most useful part of Google’s explanation is how AI Overviews work. Google says they are not a chatbot that makes things up from training data. They run searches, find top-ranking pages and then summarise what those pages say.

That matters because it means most odd answers came from the web itself, not from the model inventing facts. Google pointed to three main causes.

Data Voids and Joke Questions

A data void (a topic where very little good content exists) gives the system almost nothing reliable to work with. Hardly anyone writes a serious page about how many rocks to eat. So when someone asked, the top result was satire, and the AI Overview repeated it as if it were real advice.

Google also says many people were typing odd questions on purpose to trigger errors. Before the launch, almost nobody searched those phrases, so the system had not been tested against them at scale.

Forum Posts Taken at Face Value

Forums can be a great source of first-hand experience. But they also contain sarcasm and trolling. Google admitted that AI Overviews sometimes picked up a joke forum comment and presented it as helpful advice. The glue-on-pizza answer is the clearest example of this.

Misreading the Query or the Page

In some cases, the system misunderstood what the searcher meant, or misread the language on a page. Google describes these as a small number of cases, but they are the hardest to spot as a reader because the answer looks confident.

What Google Changed

Google says it has made more than a dozen technical improvements. Rather than only removing bad answers one by one, it says it is fixing the patterns that caused them. Here are the changes it named:

1. Better detection of nonsense queries. Questions that should not get an AI Overview, like joke questions, are now less likely to trigger one.
2. Less satire and humour content. Google has limited how much satirical and humorous pages can feed into answers.
3. Less user-generated content where it could mislead. Forum and community posts are used less in answers that could give bad advice.
4. Trigger limits where AI Overviews were not helpful. Google added restrictions for types of queries where the summaries were not proving useful.
5. Tighter rules for news and health. Google says it aims not to show AI Overviews for hard news topics, where freshness and accuracy matter most. For health, it added further triggering refinements.

Google also shared two numbers worth knowing. It says it found a content policy violation on less than one in every 7 million unique queries where AI Overviews appeared. And it says the accuracy rate of AI Overviews is about the same as featured snippets.

It is fair to read those figures with some care. They are Google’s own measures, and “content policy violation” is a narrower test than “wrong” or “unhelpful”. A silly answer can break no policy and still be wrong.

What AI Overviews Odd Answers Mean for Your Site

For most site owners, the biggest effect of this episode is not the headlines. It is the quiet change in when AI Overviews show and which sources they use. Here is how I see it.

Fewer AI Overviews on Some Queries

The new trigger limits mean some searches that showed an AI Overview in mid-May may no longer show one. That is especially likely for odd phrasing, very niche questions, hard news and some health topics.

If you run a news or health site, this could mean your normal listings sit higher on the page again for some queries. I would not assume that across the board, though. Check your own key queries by hand before drawing a conclusion.

Forum Content May Be Cited Less in Some Answers

Forums and community sites have been very visible in Google results over the past year. Google’s change does not remove them from search. It limits user-generated content in AI answers where that content could give misleading advice.

If your own site has a forum, comments or Q&A, this is a reminder that moderation matters. Clear, answered threads with accurate replies are far more useful to Google than joke replies that rank by accident.

Clear, Reliable Pages Have a Better Shot

When the system steps away from satire and sarcasm, it needs clear, trustworthy pages to summarise. That plays to the strengths of sites that already publish careful, well-sourced content.

This lines up with what Google has asked for since it added “Experience” to E-A-T. If you want a refresher, I explained that change in my article on E-E-A-T and first-hand experience. In my experience, pages that show who wrote them, why they know the topic, and where their facts come from age well through every change like this.

Your Brand Can Show Up Next to a Bad Answer

The less talked-about risk is reputation. If an AI Overview cites your page while getting something wrong, your link sits right next to the mistake. Readers may blame you, not Google.

This is why I now check AI Overviews for brand and product queries, not just for top traffic terms. I want to see what Google says about the business and which pages it uses as support.

How to Check Your Pages in AI Overviews

There is still no AI Overviews filter in Search Console. Clicks and impressions from AI Overviews are counted inside the normal web search data. So you have to combine a manual check with your usual reports. This is the process I use.

1. Pick your priority queries. Choose 20 to 50 searches that matter to your business: brand terms, product names, and your top question-style queries.
2. Search them from the US, signed out. AI Overviews are only live in the US for now. Use a clean browser window so personal history does not change what you see.
3. Record what shows. For each query, note whether an AI Overview appears, whether your page is cited, and whether the answer is correct.
4. Flag wrong or risky answers. Pay most attention to answers that are wrong about your brand, prices, opening hours, safety or health.
5. Use the feedback button. Each AI Overview has a feedback option. If an answer is wrong, report it and say what is wrong in plain words.
6. Fix your own page if it is the source. If the AI Overview misread your page, look at how the information is written. Make the key fact clear, near the top, and hard to misunderstand.
7. Repeat every two to four weeks. Google is still adjusting, so one check is not enough. Keep a simple sheet so you can see changes over time.

I’d start with brand queries before anything else. A wrong answer about your own business is the one most likely to cost you a customer.

How to Make Your Content Harder to Misread

You cannot control what Google’s model writes. You can control how clear your page is. A page that is easy for people to understand is usually easy for an AI system to summarise correctly.

Here is what I focus on:

• Answer the question early. Put the direct answer in the first few lines, then explain. Don’t bury it under a long intro.
• Keep jokes and sarcasm clearly marked. If you publish humour, make sure the page clearly signals it. Satire sites were part of the problem here.
• Use specific, plain wording. “Open Monday to Friday, 9am to 5pm” is safer than “We’re around most weekdays”.
• Add context to risky advice. For health, safety or money topics, say who the advice is for and when to get expert help.
• Show who wrote it. A named author with real knowledge of the topic is a clear trust signal for readers and for Google.
• Keep facts up to date. Old prices, dates and policies are an easy way to end up quoted wrongly.

Google’s own guidance on creating helpful, reliable, people-first content is still the best checklist for this. Nothing in the 30 May update changes it. If anything, the update makes it more important.

Can You Keep Your Content Out of AI Overviews?

Some site owners are asking whether they should block AI Overviews after seeing these errors. The honest answer is that there is no clean opt-out.

• Google-Extended does not help. It controls whether your content is used to train Google’s Gemini models. It does not affect Google Search or AI Overviews.
• Snippet controls are a blunt tool. The nosnippet and max-snippet rules limit what Google can show from your page. But they also shrink or remove your normal text snippet in search results. Google explains these in its documentation on controlling snippets.
• noindex removes you from Search. That keeps you out of AI Overviews, but also out of every other result.

In my experience, most sites should not touch these settings because of this news. Blocking snippets to avoid a rare bad summary usually costs more traffic than it saves. I would only consider it for a page where a misread summary could cause real harm, and even then I’d try rewriting the page first.

What This Means for Searchers and Clients

Many business owners I talk to are also searchers. They want to know if they can trust what they see. My advice is simple: treat an AI Overview like a quick summary, not a final answer.

Click through to the sources for anything that matters, especially health, money, legal and safety questions. If you only want the classic list of links, Google added a “Web” filter in mid-May that shows plain web results without AI Overviews.

If you report to clients, this is a good time to explain that AI Overviews are still changing week by week. Big swings in click-through rate on question queries over the next month may come from Google adjusting triggers, not from anything on the site.

Is Google Pulling Back From AI Overviews?

No. Google’s post makes it clear that AI Overviews are staying. Google says people who use them are more satisfied with their results and that visitors are more likely to stay on pages they reach from an AI Overview. Google has not shared figures behind those claims.

What has changed is the tone. Google now openly says the feature makes mistakes, explains why, and describes limits on where it shows. I expect more quiet adjustments like these rather than a full rollback.

It is also worth keeping this separate from other news this week. The documents leak that people are discussing is a different story about ranking signals. If you want my take on that, read what the Google Search API leak does and doesn’t tell us.

My Take on AI Overviews Odd Answers

I think this episode is less about AI being “broken” and more about what happens when a summary tool meets the open web at full scale. The model mostly repeated what it found. When it found satire, trolling or nothing at all, it still tried to answer.

For site owners, that means quality content matters more, not less. The system needs reliable pages to lean on. If your page is the clear, careful answer on a topic, you are in a better position after this update than before it.

Conclusion

AI Overviews odd answers came mostly from data voids, satire and joke forum posts, plus some misread queries and a number of faked screenshots. Google’s fixes limit when AI Overviews appear, reduce satire and risky user-generated content, and tighten rules for news and health. The feature itself is here to stay.

Your next step: pick your 20 most important queries, including brand searches, and check what AI Overviews say about them this week. Fix any page that is being misread, report wrong answers, and keep a record. If you have not set a baseline for clicks yet, the steps in my AI Overviews launch article walk you through it.