Google, OpenAI and Microsoft back one AI training opt-out

Website owners will soon be able to refuse AI training with one line in robots.txt instead of one rule per AI company. On October 30, 2026, OpenAI, Google and Microsoft each said they will honor a shared token called __usage_ai-training, PPC Land reported. The author of the draft standard, who works at Anthropic, said Anthropic has started building support too.

Robots.txt is the plain text file at the root of a website that tells crawlers what they may fetch. Until now, a site that wanted to stay out of AI training had to name each company's crawler or control separately, such as GPTBot or Google-Extended.

How the new token works

The token goes where a crawler's name normally goes, on a User-Agent line. The rules under it use the usual Disallow and Allow lines. The draft's own example refuses training on the whole site except a press kit folder:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

Two details matter for anyone editing the file. A wildcard group, User-Agent: *, does not count as a training preference, according to the draft. And a site with no __usage_ai-training group has expressed no preference at all, which is true of every robots.txt published today, Relevant Audience notes.

The standard is an IETF Internet-Draft, draft-kelley-aipref-training-preference-signal-00, written by Kevin Kelley. It is an individual submission, not yet endorsed by the IETF, and it expires on May 3, 2027 unless it is updated.

Who committed, and when

Company	What it said	When
Microsoft	Will honor training and grounding preferences at crawl time	February 1, 2027
Google	Aims to support the control	2027 ("early 2027" in its crawler docs)
OpenAI	Will honor the signal for "foundational generative AI training"	"In the coming months"
Anthropic	Draft author says support has started	No date given

Google's John Mueller wrote that "We aim to support this new control along with other AI providers in 2027." If a site refuses training with the shared token and has no Google-Extended rule, Google plans to read that as a Google-Extended disallow. An explicit Google-Extended rule still wins.

OpenAI said the new signal "will not replace the controls we already provide through robots.txt," according to PPC Land. Anthropic has not issued a company statement. Kelley wrote on the IETF mailing list that "Anthropic has already started implementing support for this draft," adding that it will take time to reach all of its services.

Microsoft goes further

Microsoft's plan, described by Bing's Krishna Madhavan, adds page-level controls. A page can override robots.txt with a meta tag, <meta name="Bingbot" content="ai-training:disallow">, or the header X-Robots-Tag: Bingbot: ai-training:disallow.

Bing also adds a separate grounding preference. Grounding means using a page to inform an AI answer, as opposed to training a model on it. It works per page only, with the values Allow, Disallow and DisallowExtended.

Bing's older NoArchive and NoCache tags stop counting as AI controls after March 1, 2027, PPC Land reports.

What the token does not do

It is a preference, not a lock. The draft says the token "is not a directive for crawling behavior," so crawlers may still fetch pages. A crawler that does not cooperate can ignore it, TokenPost reports.

Changing the file also does not remove content already collected for existing models. Google says Google-Extended does not affect Search inclusion or ranking, and Microsoft says Bing search eligibility is unchanged. PPC Land cites TollBit data from August 2026 showing about 15% of AI page fetchers in Europe reached URLs that sites had disallowed.

What this means for developers

• Add the group now if you want out. A __usage_ai-training group costs nothing today, and Microsoft starts reading it on February 1, 2027. Put it in its own group, never under User-Agent: *.
• Keep your existing rules. Company-specific groups like GPTBot and Google-Extended still apply, and the more specific rule wins. Do not delete them on the strength of a draft.
• Separate training from search. Refusing training does not remove you from Google or Bing search, so there is no ranking reason to hold back.
• Plan for Bing's grounding switch. If you used NoArchive or NoCache to keep content out of AI answers, move to the new page-level grounding preference before March 1, 2027.
• Block crawlers by name if you need a hard stop. The token states a preference about use. Only a per-crawler Disallow, or a block at your server, stops the fetch itself.