What is Google-Extended and should you block it?
Google-Extended is a robots.txt token from Google. It controls what Google may do with pages that Googlebot fetched; it does not fetch anything itself.
Want your site counted in the October census re-run?
Leave your domain and address to be counted for the October census re-run. Results are posted on /data; nothing is emailed.
What Google-Extended does
Google says Google-Extended controls whether content "may be used for training future generations of Gemini models" and for grounding in Gemini Apps and Vertex AI. It is a robots.txt token only: "Google-Extended doesn't have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings."
On robots.txt: Google-Extended is honoured as a robots.txt control token; the fetching itself is done by Googlebot and Google's other documented agents.
Google's wording on the part people worry about: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." AI Overviews and AI Mode are Search features served from the same index, so blocking this token does not take you out of them. Google-Extended cannot remove you from them. Since August 31, 2026 there is a separate control for that: a Search Console setting that takes a site out of AI Overviews, AI Mode and AI Overviews in Discover and, in Google's words, is not used as a ranking signal for results outside those features.
What blocking it costs
It costs nothing in search. Disallowing Google-Extended tells Google not to use your pages for the purposes the token covers. The pages are still fetched by Googlebot and still appear wherever Googlebot normally shows them.
How many of the top 5,000 sites block it
In my September 2026 census of the Tranco top 5,000, 2,771 sites returned a robots.txt. 459 of them block Google-Extended (16.6%), which makes it the 6th most-blocked of the 20 crawlers I checked. 514 files mention it by name; the rest of the mentions either allow it explicitly or restrict only part of the site.
For comparison, 699 sites (25.2%) block at least one training crawler, against 369 (13.3%) that block at least one AI search crawler. Most sites that opt out of training are doing it deliberately. Blocking AI search is more mixed, and skews more deliberate at large, recognizable sites than the rank alone would suggest: the breakdown is here. The full census is here.
The robots.txt lines
To opt out of the uses the token covers, with no effect on search:
User-agent: Google-Extended Disallow: /
How to verify a request really came from Google-Extended
Anyone can put a name in a user-agent string. Since Google-Extended is a token and Googlebot does the fetching, verify the fetcher: Google publishes the address ranges at developers.google.com/static/crawling/ipranges/common-crawlers.json. A request from outside them is not Googlebot.
Source: Google's crawler documentation. Census: the September 2026 robots.txt census, raw data at /data/census.json.
Other Google agents: Googlebot. All 20 crawlers. Check your own robots.txt against all of them with the checker.