CloudflareCloudflare's Pay Per Crawl Turns AI Bot Traffic Into a Revenue Line
Cloudflare now blocks AI crawlers by default for new domains and, by its own reporting, handles over a billion HTTP 402 'payment required' responses a day, with its Pay Per Crawl feature (still in private beta) giving publishers a way to allow, charge, or block AI bots (Pay Per Crawl is still in private beta). Here is what the emerging crawl-to-refer economics mean for content owners.
Read 
