
What happened
Cloudflare CEO Matthew Prince says bot requests hit almost 60% of web traffic in May, and Cloudflare this week announced a "Disallow AI Training" setting. Apple, Google and Microsoft agreed to it.
Why it matters
The setting lets a site stay indexed for search while blocking training crawlers, giving publishers a way to separate "index me" visitors from "train on me" crawlers.
What to watch
Prince says automated traffic could reach 1,000 times human traffic in five years, so the test is whether AI companies actually pay for content they train on.
WHO IT HITSPublishers, news outlets and dictionary publishers pursuing copyright suits against AI companies gain a practical lever, while AI companies that rely on web content for training face pressure to negotiate payment.
Summaries like this, in your inbox every morning.
Matthew Prince had expected web traffic from automated bots and AI agents to surpass human traffic at the end of next year. Instead, it happened in May, with Cloudflare reporting almost 60% of requests to websites coming from bots. Cloudflare manages traffic for more than a fifth of the web and counts 80% of the top AI companies as customers, so its data offers a rare window into how much of the web is now machine-driven.
The "Disallow AI Training" setting arrives as Sony and Warner Music, news outlets and even major dictionary publishers pursue copyright suits against AI companies over intellectual property claims. Prince frames the problem as two-sided: compensation and recognition. He says the media world of the last 30 years rewarded popularity, while the next 30 might reward credibility, and he worries the industry has not thought enough about how to make sure people get recognized. He suggests there should be an Academy Award or Nobel Prize of knowledge.
Prince describes the world he is playing for as one with 500,000 AI companies rather than five. Whether the new setting changes actual payment flows likely hinges on how many AI companies follow Apple, Google and Microsoft in honoring it, and on whether courts back the publishers' claims.
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
China's Ministry of Industry and Information Technology (MIIT) and National Development and Reform Commission…

Expanse has secured $5.3 million in seed funding for AI infrastructure software that predicts the exact GPU, C…

At an event in Scotland, Nvidia CEO Jensen Huang argued AI should not be regulated like social media

In an open letter to the president of the Royal Society, 42 mathematical fellows — including Fields Medal winn…

Anthropic CEO Daniel Amodei called for slowing the pace of AI development, and Kioxia Holdings shares briefly…

The NYC DSA's Tech Action Working Group posted an Instagram carousel on September 10 declaring AI alarmism and…
