Microsoft's Maia 300 chip: 40% more tokens per megawatt

Maia 300 is the successor to Maia 200, which Microsoft announced only in January and which is already running production AI workloads across the company, including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft's own AI models. The company's cloud group chief described Maia 200 as the best performing AI infrastructure in its fleet, measured in real world metrics for AI services.

Microsoft says the new chip is designed for long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step agentic tasks. It uses a chiplet architecture built on TSMC's N2P process node that separates compute, memory controller, and I/O functions, which Microsoft says increases the silicon area dedicated to compute performance and lets each function use the technology best suited to its role.

The chip is also tied to Microsoft's own model ambitions. Its AI chief said the company needs a platform it owns and co-designs with its models, making the work of running Microsoft AI models on Maia a critical focus.