Patreon hard-blocks AI scrapers via Cloudflare
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Patreon is using Cloudflare's AI Crawl Control to actively block AI training bots, moving beyond the robots.txt instructions scrapers had been ignoring since at least 2023.
- In testing, weekly AI training crawler attempts on Patreon dropped from "thousands" to "zero," according to a company blog post.
- Cloudflare now offers a suite of AI-restriction tools Patreon is tapping into, including a "Pay Per Crawl" marketplace letting sites charge bots for access and a default block on "mixed-use" crawlers that both index and train on ad-hosting pages.
- The timing is driven by Patreon's newer discovery features — a redesigned Home Feed and tweet-like Quips — that the company said could expose more creator content to crawlers beyond its paywall.
- Patreon will continue permitting bots that index pages and organize information that can funnel users back to the platform.
- Patreon's product chief Drew Rowny framed the move as creator agency: "Consent shouldn't depend on whether a scraper chooses to behave."
Why it matters: Robots.txt is a voluntary honor-system file — Patreon's move from requesting compliance to enforcing a block, with attempts verified falling from thousands to zero per week, shows that ask-nicely opt-outs are no longer a viable defense for content platforms. Cloudflare's emerging AI Crawl Control and Pay Per Crawl tools are turning bot blocking into a productized infrastructure layer every publisher can buy into.
