Cloudflare has partnered with newsletter publishing platform beehiiv to introduce AI Crawl Control, a new capability that gives independent publishers greater visibility and control over how artificial intelligence models access and use their content.
The integration embeds Cloudflare’s AI Crawl Control technology directly into the beehiiv platform, enabling newsletter operators to decide whether AI search engines and agents can freely crawl their content for greater discoverability or restrict AI scraping to protect archives for future monetization and licensing opportunities.
As AI-powered search and autonomous agents increasingly reshape content discovery, publishers are seeking more flexible ways to manage their digital assets without relying on complex technical configurations. The new integration simplifies this process by allowing creators to manage AI access through a straightforward dashboard.
“As the Internet evolves, ensuring creators have the tools to thrive means giving them transparency and control over how AI interacts with their content.” – Matthew Prince, Co-founder & CEO, Cloudflare
“As the Internet evolves, ensuring creators have the tools to thrive means giving them transparency and control over how AI interacts with their content,” said Matthew Prince, Co-founder and CEO of Cloudflare. “Whether publishers want to maximize discoverability or preserve their content for future opportunities, they should be able to make that choice on their own terms.”
The solution removes the need for manual robots.txt updates, firewall configurations, or other technical interventions that publishers previously relied upon to manage AI crawlers.
beehiiv users will gain access to personalized analytics that reveal which AI crawlers are attempting to access their content, which bots are being blocked, and how much referral traffic AI platforms generate. Publishers can also enable or disable access for specific AI models with a single click, while automatic updates ensure new AI crawlers are recognized without requiring code changes.
Tyler Denk, Co-founder and CEO of beehiiv, said the partnership reinforces the platform’s commitment to creator independence as AI transforms digital publishing.
“Publishers need real leverage as AI changes how audiences discover content. Together with Cloudflare, we’re giving creators the insights and controls they need to either expand discovery or protect the value of their work,” he said.
AI Crawl Control will be available in beta for all beehiiv users, providing visibility into AI traffic and referral analytics. beehiiv Max subscribers will additionally be able to actively block AI crawlers and define how their content can be used across the AI ecosystem.
The partnership reflects the growing industry focus on balancing AI-driven content discovery with creator rights, enabling publishers to make informed decisions about how their content is accessed while maintaining ownership and control over their digital assets.
