Skip to main content
USA-Based Digital Agency

Does Blocking Google-Extended Remove You From AI Overviews?

A lot of robots.txt files block Google-Extended in the belief that it keeps a site out of AI Overviews, or allow it in the belief that it gets them in. Google’s own documentation says it does neither.

By George Shvaya · · 6 min read

No. Blocking Google-Extended does not remove a site from AI Overviews or AI Mode. Google documents it as a product token that controls Gemini training and grounding only, with no user agent of its own and no effect on Search. AI Overviews and AI Mode are governed by Googlebot and the ordinary snippet controls.

What Google-Extended actually controls

Google’s crawler reference describes Google-Extended as “a standalone product token that web publishers can use to manage” whether content Google has crawled may be used to train future Gemini models, the ones behind Gemini Apps and the Vertex AI API for Gemini. It also covers grounding, where Search index content is supplied to the model at prompt time in Gemini Apps and in Grounding with Google Search on Vertex AI.

That is the whole scope. The same page is explicit about what it does not do: “Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.”

It is not a crawler, so there is nothing to see in your logs

The confusion starts with the robots.txt syntax. User-agent: Google-Extended looks like every other crawler rule, so people expect a bot with that name to appear in their access logs. It never will. Google states that “Google-Extended doesn’t have a separate HTTP request user agent string.” Crawling is done by the existing Google user agents; the token exists so that publishers have something to address in robots.txt.

If you are auditing which AI crawlers reach your site, do not spend time looking for Google-Extended in the logs. Look for Googlebot, and read your robots.txt to see how the token is set.

What actually governs AI Overviews and AI Mode

Google’s AI features documentation treats AI Overviews and AI Mode as Search features and says so in plain terms. To be eligible, a page has to be indexed and able to show with a snippet. Beyond that, “there are no additional technical requirements.” Googlebot’s robots.txt rules control crawling for Search, and because AI is built into Search, they control the AI features too.

To limit what appears, Google points to the preview controls that have existed for years:

  • nosnippet to suppress any text preview for the page;
  • data-nosnippet to exclude a specific section;
  • max-snippet to cap the preview length;
  • noindex to keep the page out of results altogether.

Google adds that these are the controls for AI features, and that Google-Extended “is not the control for these features.” If content still appears after you add a control, Google says to confirm the control is visible to Googlebot in the URL Inspection tool and wait for a recrawl, which “can take anywhere from several days to several months.”

Goal to control

Which robots or snippet control matches each publisher goal
GoalCorrect controlWhat happens
Stop my content training future Gemini modelsUser-agent: Google-Extended / Disallow: /Opts out of Gemini training and grounding. No effect on Search, AI Overviews or AI Mode.
Keep a page out of Google entirelynoindex (meta or header)Removes the page from Search, which removes it from AI Overviews and AI Mode too.
Stay in Search but show no snippet anywherenosnippetPage can still rank; no text preview is shown, in classic results or AI features.
Hide one section of a page from previewsdata-nosnippet attribute on the elementThat section is excluded from snippets and AI features; the rest of the page is unaffected.
Limit how much text Google can quotemax-snippet:[number]Caps the preview length used in classic results and AI features.
Appear in AI Overviews and AI ModeAllow Googlebot; be indexed and snippet-eligibleNothing further is required. No special markup, no AI text file.

Is your robots.txt saying what you think it says?

Our free audit reads your robots.txt, meta robots and snippet controls and reports which AI surfaces each page is actually eligible for.

Get a Free AI Visibility Audit

Prefer to talk it through? Book a Free Call

Getting in requires nothing special either

The flip side of the myth is the belief that allowing Google-Extended, adding a particular schema type or publishing an AI-specific text file earns a place in AI Overviews. Google’s guidance closes each of those doors. “There’s also no special schema.org structured data that you need to add,” no machine-readable or AI-specific text files are required, and structured data should simply match the visible text on the page. Google also notes that indexing and serving are never guaranteed, and that AI Overviews appear only when its systems decide they add value beyond classic results.

What does help is the unglamorous list Google repeats everywhere: allow crawling, link internally so pages are discoverable, deliver a good page experience, and put the substance in text. In our experience the first item fails more often than people expect, which is why index coverage sits ahead of any generative engine optimization work in our sequence. The glossary entries on AI Overviews and AI Mode cover what each surface is.

Where the traffic shows up

Google reports clicks and impressions from AI Overviews and AI Mode inside the ordinary Search Console Performance report under the “Web” search type, mixed in with classic results. There is no separate filter in the documentation we reviewed. Google’s stated position is that clicks from AI Overviews are higher quality because users are more likely to spend time on the site, and it recommends tracking engagement and conversions in an analytics tool rather than looking for a dedicated report. The measurement section of our AI Search Playbook works through what you can and cannot see.

What to do with your robots.txt

  1. Decide the Gemini training question on its own merits and set Google-Extended accordingly. It will not change your Search or AI Overviews footprint either way.
  2. If a page should stay out of AI features but remain in Search, use nosnippet, data-nosnippet or max-snippet, then confirm with URL Inspection.
  3. If you want to be in AI Overviews, make sure Googlebot is allowed, the page is indexed, and the snippet controls are not accidentally suppressing it. That is the whole checklist.
  4. Write a comment in robots.txt above the Google-Extended line saying what it controls, so the next person does not re-learn this the hard way.

Frequently asked questions

Does Google-Extended affect AI Overviews or AI Mode?

No. Google documents Google-Extended as a product token that controls whether crawled content is used to train future Gemini models and for grounding in Gemini Apps and Vertex AI. Google states it does not impact a site’s inclusion in Google Search and is not a ranking signal. AI Overviews and AI Mode are Search features governed by Googlebot and the snippet controls.

How do I keep a page out of AI Overviews without leaving Google Search?

Use the preview controls Google already honors: nosnippet to suppress any snippet, data-nosnippet to exclude a section of the page, or max-snippet to cap the length. Google lists these as the way to limit what appears from a page in AI features. Changes take effect after Googlebot recrawls, which Google says can take from several days to several months.

Is there a user agent string for Google-Extended?

No. Google says Google-Extended doesn’t have a separate HTTP request user agent string. Crawling happens with the existing Google user agents, and the Google-Extended token exists only so you can address it in robots.txt.

Do I need special schema or an AI text file to appear in AI Overviews?

No. Google says there are no additional technical requirements beyond being indexed and eligible to show with a snippet, that there is no special schema.org structured data you need to add, and that no AI-specific text files are required. Structured data should match the visible text on the page.

Where does AI Overviews traffic show up in Search Console?

In the Performance report under the Web search type, mixed in with ordinary Search traffic. Google does not break it out as its own search type in the documentation we reviewed.

About the work

Webvello is an AI search optimization and index engineering company founded in 2024, based in Roseville, California. We’ve delivered 50+ projects across 37+ cities, with average client ROI in the 3–5x range on properties where the technical foundation was the binding constraint. Founder George Shvaya came to search from cybersecurity and systems engineering, which is why this post quotes Google’s documentation instead of guessing at it.

Sources: Google Search Central, “AI features and your website” and “Overview of Google crawlers and fetchers,” both re-read on October 15, 2026. Google updates these pages; check them before relying on a quotation here.

Want someone to read your robots.txt, meta robots and snippet controls page by page?

Book a Free Call