Bots outnumber humans: publishers rethink web monetization

The shift became impossible to ignore when measurement systems themselves showed machines overtaking people. In June 2026, Cloudflare CEO Matthew Prince posted on X that, for the first time, bot traffic surpassed human traffic, and by October Cloudflare said bots accounted for 57.2% of requests to HTML pages, meaning the web’s basic audience denominator had changed: publishers were no longer measuring mostly human readers, but a mixed stream in which software agents had become a primary consumer of content, distorting what traffic, reach, and audience size actually meant.

That mattered because ad pricing and attribution were built on the assumption that a visit implied a person who could see, remember, and act on advertising. According to a 2025 report by HUMAN Security, traffic from AI agents and agentic browsers grew 7,851% year over year, while Prince argued bots “don’t click on ads” and “don’t respond” to brands; if agents consume content without seeing surrounding ads or making decisions through ad persuasion, CPMs lose their logic and publishers are pushed toward machine-specific pricing such as micropayments and pay-per-crawl access.

What publishers are losing first is the old bargain of the open web: let machines read your pages, and they send readers back. Analysis found that Google still crawls about five pages per referred visitor, but OpenAI crawls over 1,000, and Anthropic crawls over 38,000 pages for every single visitor it refers, a ratio that shows how AI systems now extract content while starving publishers of the human visits that would have become pageviews, recirculation clicks, and ad impressions.

The same substitution is now visible in search and news referrals. Wikimedia Foundation showed the squeeze in April 2025, when bots account for about 35% of its pageviews but at least 65% of its most expensive traffic, and six months later human pageviews fell roughly 8% year on year; ProMarket reported that the appearance of an AIO reduced users’ organic clicks to third-party sites by 39.8 percent and increased searches where the user clicked on no links at all by 34.5 percent, while Axios/Chartbeat said chatbot referrals sono passati da 10 mila a 160 mila in tre anni: 16 volte tanto, yet il 40% di chi apre un sito di news da desktop non scrolla nemmeno la pagina.

Publisher ad economics are failing because the metrics that set prices and prove return were designed to capture human intent, not machine activity that mimics it. Research & Data reported that Lunio surveyed 131 senior marketing leaders in May 2026 and found “More than three quarters (75.6 per cent) of surveyed marketers estimate that they lose more than 5 per cent of their monthly performance budget to invalid traffic,” while 24.4 per cent said wasted budgets from invalid clicks were the most damaging outcome, showing how bot activity can corrupt the very signals advertisers use to value inventory.

The contrast with platform advertising makes the break clearer: Meta reported “Ad impressions across the Family of Apps rose 14 percent year over year” and “the average price per ad climbed 12 percent,” while it also disclosed that its new ranking system, Meta Generative Recommender, lifted “app event conversions,” “ad clicks,” and “conversions.” Even there, monetization depended on measurable human-response signals rather than raw delivery, and despite “advertising business had never grown faster,” Meta’s “net income dropped 8 percent to 15.8 billion dollars,” with the “culprit” being “a 2.4 billion dollar charge tied to legal proceedings” and “1.18 billion dollars in severance costs.”

Publishers are beginning to price machine access itself as a product. Time’s bot-only ad experiment is the clearest example: it has begun serving ads specifically meant for crawlers, with advertiser FAQs invisible to human readers, and is charging for one agent ad per machine-readable page so that every fetch becomes a billable request. That effectively turns scraping into a pay-per-crawl or pay-per-use unit, with advertisers buying the chance that verified, machine-readable material is later retrieved and incorporated into AI answers rather than relying on conventional human-viewed display inventory.

Other experiments are trying to formalize that logic into broader payment rails. Media Copilot described pay-per-crawl as a micropayment model in which a bot “will like make a micro payment every time they use an Article for context,” while a licensing push for Purple Pass argued, “We started with the FT and a bunch of different other Publishers…” because “If you give me that method to purchase, I will buy it, but I'm not paying 250.” Even platform deals are shifting: the-decoder.com reported, “Google Pays One Publisher $1 Million in AI Overview Pilot, Others Earn Under 0.1% of Revenue.”