New model empowers content creators to protect their work and opens path to ethical AI innovation
Cloudflare has taken a bold stance to reshape the rules of engagement between AI companies and online content creators. As of July 1, 2025, it has become the first Internet infrastructure provider to block AI crawlers by default, requiring explicit permission before any scraping of website content can occur.
For years, AI models have scraped the open web—articles, blogs, images—without consent, attribution, or compensation. This not only disrupts publishers’ revenue streams but also undermines the incentives for producing original, high-quality content. Cloudflare’s new permission-first framework aims to reverse this imbalance.
“If the Internet is going to survive the age of AI, we need to give publishers the control they deserve.”
— Matthew Prince, Co-founder and CEO, Cloudflare
“Original content is what makes the Internet one of the greatest inventions of the last century,” said Matthew Prince, CEO of Cloudflare. “It’s essential that creators continue making it — and that means giving them control and a fair value exchange.”
Now, when a new domain is registered with Cloudflare, owners will be prompted to explicitly allow or block AI crawlers. They can also control usage based on purpose—training, inference, or search—creating a more transparent and fair digital ecosystem.
The announcement has won strong support from global publishers and platforms. CEOs from Condé Nast, Dotdash Meredith, TIME, Reddit, Gannett, and BuzzFeed praised the move as a long-overdue step toward content sovereignty.
“Cloudflare’s initiative is a game-changer for publishers and sets a new standard for how content is respected online,” said Roger Lynch, CEO of Condé Nast. “This is a critical step toward a fair value exchange on the Internet.”
Alongside this default blocking, Cloudflare is rolling out tools such as verified bot signatures, AI crawler authentication protocols, and a “Pay Per Crawl” beta, which could enable monetization for content creators.
This move could trigger a larger shift across the digital landscape — where AI companies are incentivized to license content, and creators are fairly compensated for their work.
With over 20% of global web traffic flowing through its network, Cloudflare’s decision is more than symbolic — it’s a blueprint for a more ethical, sustainable Internet where AI and original content can coexist with mutual respect.