Have it both ways: stay discoverable in search while disallowing AI training
- ID
- 24649
- Status
- summarized
- Published
- 15 Sep 2026, 9:00 PM
- Fetched
- 15 Sep 2026, 10:29 PM
- Provider
- Cloudflare Blog
- Category
- infrastructure
- Original URL
- https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/
- Source URL
- https://blog.cloudflare.com/rss/
Summary
- Score
- 7.0
- Created
- 15 Sep 2026, 10:30 PM
- Tags
- Audience
- developerssaas_founders
What happened
Cloudflare announced a 'Disallow AI Training' setting that lets site owners stay indexed for search while blocking mixed-use crawlers (like those from Apple, Google, and Microsoft) from training on their content. Cloudflare reports <1% of its sites block search bots while 17% block training, motivating the granular control. The company is also working with crawler operators on an 'Accountable' designation and plans to let sites control how much content appears in AI summaries by early next year.
Why it matters
If you run a site behind Cloudflare, you can now flip one setting to keep search discoverability without surrendering content to AI training—useful for content businesses, SaaS docs, or marketing sites that depend on search traffic but don't want to feed competitors' models. The upcoming AI-summary controls mean you should watch for a Cloudflare dashboard option early next year to limit how much of your content gets quoted in AI overviews.
Discussion angle
Does this actually shift leverage back to site owners, or will non-Accountable crawlers and AI-summary scrapers just ignore it—and should Malaysian content/SaaS sites adopt this now or wait to see if Google's search ranking is affected?