Google-Extended
Google-Extended is a robots.txt token that lets a site owner control whether Google uses their content for Gemini model training and grounding, without affecting how Googlebot crawls the site for Search or whether pages appear in AI Overviews.
Google-Extended is not a crawler. It's a token you can name in robots.txt to tell Google whether the content it already crawls with Googlebot may be used to train and ground Gemini models. Google's documentation is explicit that there's no separate Google-Extended bot fetching your pages; Googlebot does the crawling, and the Google-Extended rule decides what that crawled content can be used for on the AI side.
The most important thing Google says about it is what it does not do. Per Google's documentation, a Google-Extended rule doesn't affect a site's inclusion or ranking in Google Search, and AI Overviews and AI Mode use the normal Googlebot crawl, not the Google-Extended token. So blocking Google-Extended opts you out of Gemini training and grounding but leaves your Search presence and your AI Overviews eligibility alone.
That makes it a cleaner decision than the OpenAI equivalent. With OpenAI, blocking the wrong bot (OAI-SearchBot instead of GPTBot) removes you from ChatGPT's search answers. With Google, blocking Google-Extended affects the Gemini app's use of your content but shouldn't touch what shows up when someone Googles a plumber in your city, including the AI Overview at the top.
For a small business, the question is whether you want Gemini to be able to draw on your pages when someone asks it a question. Gemini accounts for about 11.56% of AI referral traffic according to Similarweb-family data, second to ChatGPT. Blocking Google-Extended means giving up whatever share of that Gemini might send you. Most local businesses have no reason to block it; the content is public and the goal is to be found.
One related note. Google announced Google Business Profile integration in the Gemini app on June 10, 2026. That integration helps business owners manage their profile and get operational suggestions inside Gemini. It's not a documented change to how Gemini cites or recommends businesses to end users, so don't read it as Gemini now recommending from GBP data.
Keep exploring the glossary
- AI CrawlerAn AI crawler is an automated bot such as GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, or Google-Extended that fetches web pages so an AI company can train its models or retrieve current information to answer user questions.
- robots.txt for AI Crawlersrobots.txt for AI crawlers means using the robots.txt file at your site's root to allow or block named AI bots such as GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot, deciding which AI systems may read your pages for training or live answers.
- AI OverviewsAI Overviews is Google's AI-generated summary that appears above the organic results for many searches, answering the query directly and citing a handful of sources instead of showing ten ranked links.
- GPTBotGPTBot is OpenAI's web crawler for collecting publicly available content to train its AI models, identified by the user-agent string GPTBot and controllable through robots.txt, and it is separate from OAI-SearchBot, which powers ChatGPT's search answers.
Common questions about Google-Extended
- Does blocking Google-Extended hurt my Google rankings?
- No, according to Google's own documentation. Google-Extended only controls whether your content is used for Gemini training and grounding. Search crawling, ranking, AI Overviews, and AI Mode all run on the regular Googlebot crawl and aren't affected by a Google-Extended rule in your robots.txt.
- Is Google-Extended a separate crawler I'll see in my logs?
- No. Google states there's no distinct Google-Extended user-agent fetching pages, so you won't find it in server logs. Googlebot does the crawling, and the Google-Extended token in robots.txt controls what that already-crawled content may be used for on the Gemini side.
- Should a local business block Google-Extended?
- Usually not. Blocking it removes your content from Gemini's grounding, and Gemini is the second-largest AI referral source at about 11.56% per Similarweb-family data. The content is already public and crawled by Googlebot either way. Block it only if you specifically object to Gemini using your pages.