Google-Extended
-
Google, OpenAI and Microsoft back a shared robots.txt signal for AI training
OpenAI and Microsoft say they will honor a shared robots.txt signal for AI training preferences, and Google plans to. It covers training only and rests on an IETF draft with no formal standing.
-
Google says robots.txt governs crawling, not use, and lists its AI Search controls
A Google video and a documentation update on Oct. 21 separate controlling crawling from controlling use in AI Overviews and AI Mode. A standardized robots.txt rule for AI-use preferences is still only an IETF draft.