This guide is for bloggers, marketers, agency owners and small business site owners who want to decide whether AI models may be trained on their content. On 30 October 2026, Google’s Search Central blog announced that it plans to support a shared AI training control in robots.txt. Below is what the post says, what it leaves out, and what you can do today.
What Google announced
The Search Central post, “Supporting a shared AI training control for websites,” says Google plans to support a shared control built on work from the IETF’s AI Preferences (AIPREF) working group. The idea is simple. Today, a site owner who wants to say “do not train on my content” has to look up a separate instruction for each AI company. A shared control would let you set one broad preference that participating AI providers can read.
Google says it aims to support the control, along with other AI providers, in 2027, and that it will post updates on its progress. That is a target, not a launch date. Nothing in the post suggests you can use the shared control in Google’s systems today.
What the shared preference looks like (draft only)
The Search Central post describes the mechanism but does not show token syntax. The syntax below comes from an IETF Internet-Draft, draft-kelley-aipref-training-preference-signal, version 00, last updated 30 October 2026. Treat it as a draft that may change. The draft says it is not endorsed by the IETF and has no formal standing, it expires on 3 May 2027, and it lists a single author, Kevin Kelley of Anthropic. Google’s crawler documentation names the same __usage_ai-training user-agent token and says Google intends to support it in early 2027.
According to the draft, the preference is written as an ordinary robots.txt group. It adds no new directives. Instead, it uses a policy token in the User-Agent line. The draft’s example reads:
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/The draft describes this as a preference against AI training for every path except /press-kit/, where it expresses a preference in favor. Other points from the draft worth knowing:
- The token is matched exactly, ignoring case. A wildcard group (
User-Agent: *) does not count as matching it. - If your robots.txt has no group for the token, no preference is expressed.
- Content-usage preferences are separate from crawler rules and do not direct crawling behavior. To limit crawling, the draft says to use crawler product tokens instead.
- The draft says specific expressions should take precedence over general ones, and that private agreements may override preferences.
Because this is version 00 of an individual draft, do not copy it into production robots.txt files and assume it will work. Wait until Google and other providers publish final documentation.
How it interacts with Google-Extended
Google-Extended is the control Google already offers. Google’s crawler documentation describes it as a standalone product token that publishers can use to manage whether content Google crawls may be used to train future Gemini models. The same page says it also covers grounding, where content from the Google Search index supplies Gemini Apps and Vertex AI with information at prompt time.
According to the Search Central post, the two controls are designed to work together:
- If you use the shared policy token to opt out of training and you have not explicitly specified Google-Extended, Google plans to treat your site as if Google-Extended were disallowed.
- Company-specific controls keep working alongside the shared signal.
- The post says websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps.
In practice, an explicit Google-Extended rule appears to be the more precise tool, and the shared token a broad default. The post does not spell out every edge case, such as what happens if the two conflict across different paths, so check Google’s documentation again when the feature ships.
What owners can do today
Google’s documentation shows the Google-Extended token in robots.txt, with this example:
user-agent: Google-Extended
allow: /archive/1Q84
disallow: /archive/That example blocks the /archive/ folder except one page. Follow these steps to apply the same idea to your site:
- Decide what you actually want. Some owners want no AI training on anything; others want to protect paid or original material but allow the rest.
- Open your robots.txt file. On WordPress, this may be a file in the site root or something managed by an SEO plugin or your host, so check which one controls yours before editing.
- Add a Google-Extended group with the paths you want to disallow, following the format in Google’s example.
- Save, then load
yourdomain.com/robots.txtin a browser to confirm the live file shows your change. - Keep a note of the date and what you changed, so you can revisit it when the shared control arrives.
Google-Extended does not cover other companies. Each AI provider publishes its own rules, so an opt-out for Google does not opt you out elsewhere. That gap is exactly what a shared control is meant to close.
What the post does not say
Much of the early commentary on announcements like this goes beyond what the source supports. Here is what is and is not established.
| Question | What the sources say |
|---|---|
| Is the shared control live now? | No. Google says it aims to support it in 2027. |
Will the final token be __usage_ai-training? | Not confirmed. That is the token in the IETF draft and in Google’s crawler documentation, but the draft is version 00 and may change. |
| Does Google-Extended affect Search inclusion? | Google’s documentation says it does not impact a site’s inclusion in Google Search and is not a ranking signal. |
| Does the new shared control affect Search crawling, indexing or ranking? | The post does not say. It covers AI training preferences only. |
| Does it change AI Overviews? | The post does not address AI Overviews. |
| Do other AI companies support it? | Google says it aims to support it “along with other AI providers” in 2027. The post does not name them in the material reviewed here. |
If your concern is how your pages appear in AI Overviews and AI Mode, that is a separate set of controls. Our guide to Google AI Overviews and AI Mode: What You Can Control covers it. And if you worry that a robots.txt edit could hurt your visibility, the Google-Extended documentation is reassuring on Search inclusion, but any traffic change deserves a proper diagnosis, as in Why Did My Google Rankings Drop? A Guide to Recovery.
Common mistakes to avoid
- Blocking the wrong token. Google-Extended is the AI-specific token. Google’s documentation says it has no separate HTTP user agent string, so do not confuse it with the crawler user agents you may see in server logs.
- Treating the draft as final. Version 00 of an individual draft can change before any provider ships support.
- Assuming robots.txt is a legal contract. The draft itself says preferences are not crawler access controls and that private agreements may override them.
- Forgetting other providers. Until a shared control exists, each AI company needs its own rule, so review their documentation separately.
- Editing the wrong file. On WordPress, an SEO plugin or your host may manage robots.txt rather than a file you upload. Always check the live URL after saving to confirm your rule appears.
Key takeaways
- Google plans to support a shared AI training preference in robots.txt, based on IETF AIPREF work, and aims to do so in 2027.
- The syntax in the IETF draft uses a
__usage_ai-trainingtoken in an ordinary robots.txt group, but it is a version 00 draft and may change. - Google says an opt-out through the shared token, with Google-Extended unspecified, will be treated as Google-Extended disallowed.
- Today, the Google-Extended token in robots.txt is the documented control, and Google says it does not impact inclusion in Google Search.
- The post covers training preferences only and says nothing about ranking or AI Overviews.
FAQ
Can I use the shared AI training control now?
Not with Google. The company says it aims to support it in 2027. Until then, use Google-Extended for Google and each provider’s own documented control for others.
Will blocking Google-Extended hurt my Search rankings?
Google’s documentation says Google-Extended does not impact a site’s inclusion in Google Search and is not used as a ranking signal.
Do I need to change my robots.txt now to prepare?
No preparation is required for the shared control. If you want to limit Gemini training use today, add a Google-Extended rule. Revisit the file once final syntax is documented.
Is the IETF draft an official standard?
No. It is an individual Internet-Draft that is not endorsed by the IETF and has no formal standing. Its stated intended status is Standards Track, but that is a long way from adoption.