Patreon Shifts Tactics: From Requesting AI Bots Not to Scrape to Implementing Blocks
Patreon, a platform designed for creators to build memberships, is intensifying its efforts against AI systems that scrape content for training purposes. The company announced on Thursday its collaboration with Cloudflare, an internet infrastructure provider, to directly prevent AI bots from accessing creators’ works without authorization.
These enhanced security measures are necessary due to the evolving sophistication of AI scraping techniques since Patreon’s initial efforts to limit AI crawlers in 2023. Historically, the platform’s paywall has protected most of the content from being accessed by crawlers. Recently, however, Patreon implemented new discovery features, such as a revamped Home Feed and tweet-like updates called Quips, which could inadvertently expose more content to scraping bots.
These developments come as various online publishers and creators are recognizing the ways in which AI utilizes their content to enhance its models. In response, Cloudflare has introduced tools enabling publishers to restrict AI bot access, including a new marketplace allowing websites to charge AI crawlers for scraping, named Pay Per Crawl. Recently, Cloudflare updated its policies, enforcing default blocks on “mixed-use” crawlers that both index content and train on pages featuring advertisements.
Patreon is expanding its collaboration with Cloudflare to leverage the company’s AI Crawl Control technology to refine its policies and enforcement measures. Unlike previous practices that relied on the robots.txt files to instruct crawlers, Patreon is now actively obstructing AI training bots from accessing its content.
“Consent shouldn’t depend on whether a scraper chooses to behave,” a blog post from Patreon states, referring to its new stringent strategies.
Testing showed that individual AI training crawlers made thousands of attempts to access Patreon weekly, which plummeted to zero following the new measures. This suggests that AI scrapers had disregarded the robots.txt file and scraped content anyway, despite Patreon’s requests.
However, the platform clarifies that it will still permit bots that index pages and organize information to redirect users back to Patreon.
“As AI technologies become more advanced and prevalent, creators should have substantial control over how their work is utilized by AI enterprises,” stated Drew Rowny, Patreon’s product chief, in the announcement. “On much of the Internet, creators face the challenge of allowing AI training with their content just to expand their audience. Patreon embodies a different approach: empowering creators to grow their audience while maintaining control over their work’s usage.”
