Google, Microsoft, OpenAI and Anthropic Back a Shared AI Training Opt-Out Signal
Four major AI providers are aligning around a machine-readable robots.txt policy token for AI training preferences. The proposal separates model-training choices from AI answer grounding, creating a more consistent control for publishers.
Google, Microsoft, OpenAI and Anthropic have announced support for work toward a shared way for website owners to express AI training preferences in robots.txt. The initiative could give publishers one interoperable signal instead of requiring separate crawler directives for every AI provider.
The work is tied to the Internet Engineering Task Force's AI Preferences initiative, or AIPREF, and its draft Machine-Readable Training Preference Signal. The draft introduces a policy token called __usage_ai-training, which can be applied to an entire site or to specific paths while retaining the existing robots.txt framework.
The important distinction is that the proposal concerns training preferences, not necessarily whether a provider can use a page as a source for an AI-generated answer. Publishers will still need to understand and manage training and grounding as separate decisions where providers offer separate controls.
What the shared signal is designed to do
The draft aims to make a publisher's preference about AI model training machine-readable and interoperable. Rather than having each crawler or company interpret a different non-standard instruction, participating providers would recognize the same policy token.
Google says the shared signal will enable publishers to express AI training preferences in a single step. In its announcement on AI training controls, Google says it plans to roll out support in 2027 as the IETF standard develops.
For Google specifically, the company says that when a site uses the policy token to opt out of training and does not specify Google-Extended, it will interpret the preference as a disallow for Google-Extended. Google-Extended is Google's existing control for whether content may be used to improve Gemini models and certain generative AI systems.
The IETF's Machine-Readable Training Preference Signal draft defines the proposed token and its intended scope. It remains a draft, so publishers should treat its technical form and provider implementations as subject to change before broad deployment.
Training and grounding are different choices
The most consequential part of this change is not simply the prospect of one opt-out. It is the clearer separation between content used to train models and content used to ground, cite or inform a generated answer.
Microsoft's Bing announcement describes independent preferences for AI Training and AI Grounding. In practical terms, that means a publisher may be able to decline use of content for model training while still allowing it to be retrieved or referenced in AI-generated responses, depending on the provider's controls and implementation.
That distinction matters for search visibility:
- A training opt-out addresses whether content contributes to future model development.
- A grounding preference addresses whether content can be used to support an AI answer at the time of retrieval.
- Traditional search crawling and indexing can be governed by separate directives and should not be assumed to follow the same rules.
An organization that blocks all AI-related access without considering these separate functions could reduce opportunities for its content to appear in answer experiences. Conversely, allowing grounding is not the same as granting permission for training. The right choice will depend on a site's editorial policy, commercial model, rights strategy and visibility goals.
When support is expected
The providers are aligned on the direction, but their timelines are not identical. Google has set a 2027 target rather than a specific launch date. Microsoft says it will begin honoring AI training opt-outs on February 1, 2027, and has set March 1, 2027 as the transition deadline for legacy NoArchive and NoCache signals in this area.
OpenAI has also said it intends to honor a shared robots.txt signal, while Anthropic has described active work to support the draft in its crawlers. OpenAI's statement on website training preferences confirms its commitment to the shared approach.
These announcements establish intent and a path toward interoperability. They do not mean every provider is already honoring the final standard, nor do they guarantee that the draft's exact syntax will remain unchanged. Implementation details, crawler documentation and support dates should be checked before making production changes.
| Provider | Position described in the announcements | Timeline stated |
|---|---|---|
| Plans to support the shared training-preference signal; connects an opt-out to Google-Extended when Google-Extended is not separately specified. | 2027 | |
| Microsoft | Describes separate AI Training and AI Grounding preferences. | Training opt-outs from February 1, 2027 |
| OpenAI | Committed to honoring a shared robots.txt signal for AI training preferences. | Aligned with the developing standard |
| Anthropic | Reported active development of support in its crawlers. | Aligned with IETF discussions |
What publishers should do now
Publishers do not need to rush into copying unverified configuration examples. The safer preparation is to decide what policy the organization wants before provider support becomes broadly available.
First, inventory existing crawler directives in robots.txt. Note any rules for Google-Extended, OpenAI, Microsoft or Anthropic crawlers, and identify whether they reflect a deliberate current policy or a historical default.
Second, establish separate internal positions for model training, AI grounding and conventional search crawling. These decisions may involve legal, editorial, product, advertising and SEO teams. A publisher that values discovery through AI answers may reasonably reach a different grounding decision from its training decision.
Third, monitor provider documentation for the final syntax and the dates on which each provider begins honoring it. The draft is designed to work at both site and path level, which could be useful for sites with different licensing or commercial arrangements across sections. However, the final policy should be deployed only after the site confirms the exact implementation requirements.
Finally, document changes and watch visibility data after implementation. robots.txt controls express crawler preferences, but they do not by themselves measure search traffic, AI citations or commercial outcomes. Any visibility effect will depend on how each provider implements the signal and how often its AI products retrieve and reference the relevant content.
For teams deciding how to balance AI training restrictions with answer visibility, GEO Search Leads can help frame the website-side questions around content accessibility, technical signals and AI discovery opportunities.
Create a GEO Search Leads account to begin assessing your site's GEO readiness and identify areas that may need a clearer optimization plan.
Frequently Asked Questions
What is the proposed shared AI training signal?
It is an IETF draft mechanism that uses the __usage_ai-training policy token in the existing robots.txt framework to express whether content may be used for AI model training. Google, Microsoft, OpenAI and Anthropic have indicated support for the direction of this work.
Does opting out of AI training also block content from AI-generated answers?
Not necessarily. The announced approach distinguishes model training from AI grounding, which is the use of content to support an AI-generated answer. Providers may offer separate controls for those choices.
When will Google support the shared signal?
Google says it plans to roll out support in 2027. Its announcement does not provide a more specific date.
Should publishers remove existing AI crawler directives now?
No provider announcement requires publishers to remove existing directives immediately. Existing rules should be reviewed against the site's current policy, while publishers wait for final implementation documentation and support dates.
Will the shared signal affect ordinary Google Search crawling and rankings?
The announced signal is about AI training preferences. It should not be assumed to replace conventional search crawling or indexing controls, or to directly determine Google Search rankings.
Conclusion
A shared AI training preference signal could replace a fragmented set of provider-specific choices with a more consistent publisher control. Its value will depend on adoption and implementation, but the separation of training from grounding gives site owners a more precise framework for making AI access decisions without conflating them with traditional search visibility.