Microsoft trained MAI models on unlicensed web data, contradicting clean-data pledge

What happened

Microsoft's technical paper reveals that its new MAI models were partly trained on unlicensed web data from sources including Common Crawl, despite the company's earlier claim that they used only "enterprise grade, clean and commercially licensed data." Microsoft says it uses a proprietary crawler that respects the Robots Exclusion Protocol (robots.txt) and related controls to manage how content is accessed.

Why it matters

Microsoft marketed its training data as especially clean, but the paper shows it relies on the open internet like other AI companies do. The company relies on a fair-use argument—a legal framework still being contested and sorted out in courts—to justify web scraping. This suggests businesses considering Microsoft's AI offerings should not assume the underlying training data is fundamentally different from competitors'.

What to watch

The paper describes the data as a "mixture of publicly available and licensed human-generated data," placing the burden of protecting content on individual site owners through robots.txt and HTML controls rather than explicit licensing agreements.