pay per crawl articles.

Cloudflare's Pay Per Crawl Turns AI Bot Traffic Into a Revenue Line

Cloudflare now blocks AI crawlers by default for new domains and, by its own reporting, handles over a billion HTTP 402 'payment required' responses a day, with its Pay Per Crawl feature (still in private beta) giving publishers a way to allow, charge, or block AI bots. Here is what the emerging crawl-to-refer economics mean for content owners.