Nvidia, AMD, Cerebras race to lead AI inference chip market

The next frontier of artificial intelligence is not building and training models, but deploying them — the inference phase where trained models execute real-world tasks for users and applications. While training has been the focus of the first wave of AI infrastructure investment and has made Nvidia the dominant chipmaker, inference is expected to eventually become the larger of the two markets. The challenge in inference is different from training: whereas training demands sheer raw computing power, inference depends heavily on fast memory access — getting the trained weights and intermediate values to the compute cores as quickly as possible.

Nvidia, long the leader in AI infrastructure through its GPU monopoly and CUDA software ecosystem, has moved to entrench itself in inference by acquiring Groq and integrating its language processing units (LPUs) into its product line. LPUs use a small amount of on-chip SRAM (static random-access memory) to deliver inference at near-zero latency. Nvidia's inference solution pairs LPUs — which handle the decode phase, generating a response token by token — with GPUs equipped with high-bandwidth memory (HBM), which handle the prefill phase of understanding the user's initial prompt. The company has built complete server racks optimized for this dual-chip workflow.

Cerebras has taken a radically different approach. Recognizing that SRAM is physically bulky and limits how much memory any single chip can carry, the company designed wafer-sized chips approximately the size of dinner plates. These chips cram the equivalent of many standard chips onto a single slab of silicon, enabling the company to claim performance advantages of 6 times over Nvidia's LPUs and 15 times over Nvidia's GPUs on inference tasks. However, the wafer-scale design comes with drawbacks: defects in manufacturing are costly, so Cerebras builds in redundant cores to allow chips to work around defective areas. The chips also require specialized cooling and power management, making them impractical to sell as standalone components. Instead, Cerebras bundles them into its complete end-to-end CS-3 server rack system, positioning it as an expensive premium offering. Despite the high cost, the company has secured major deals: a $20 billion（約3.2兆円） contract with OpenAI and a separate arrangement with Amazon Web Services, both of which could accelerate mainstream adoption.

AMD, the third competitor, is using a more incremental approach. Rather than on-chip SRAM, it relies on chiplet design — modular chip architecture that allows more memory to be packaged alongside its GPUs. AMD has also recently acquired MEXT, a memory optimization software company that tackles a pressing problem in data center economics: high-bandwidth memory (HBM) is in short supply, and DRAM prices have been surging, making data center construction increasingly expensive. MEXT's core innovation is a predictive AI engine that analyzes memory access patterns in real time, anticipates what data an application will need next, and automatically offloads rarely-accessed data to much cheaper flash storage. This approach allows AMD to offer inference servers that save customers money while maintaining performance. Beyond inference, AMD is also positioned to win from the rise of agentic AI — autonomous AI agents that require far more CPU processing than other AI workloads. As agentic AI grows, the ratio of GPUs to CPUs in data centers is expected to compress from about 8 to 1 toward 1 to 1, and AMD is a leading CPU maker for AI data centers, giving it exposure to this second wave of growth. The company already has inference deals with OpenAI and Meta Platforms, and rumors suggest Anthropic may become a major customer.

The AI infrastructure market is entering a new phase. While the initial wave of artificial intelligence investment centered on training large language models — where Nvidia's CUDA software platform and GPU dominance created an unassailable lead — the economics of deployment are shifting toward inference. Inference workloads place different demands on hardware: speed of memory access matters far more than raw computing throughput, and all three competitors have recognized this constraint and designed fundamentally different solutions.

Nvidia's strategy has been to acquire its way into the space, absorbing Groq's language processing units and layering them into its existing CUDA ecosystem. This preserves its software advantage while addressing the memory bottleneck through specialized chip architecture. Cerebras, by contrast, is pursuing a radically different hardware path: wafer-scale silicon that consolidates more transistors on a single chip, achieving faster performance but at the cost of complexity, premium pricing, and the requirement to sell complete systems rather than individual chips. AMD is taking a third route, using chiplet modularization to combine memory more flexibly with its existing GPU and CPU designs, and augmenting this with software optimization (MEXT) to reduce the need for expensive high-bandwidth memory.

The body suggests that AMD's dual exposure — both to inference growth and to the explosive demand for CPUs driven by agentic AI — gives it two growth drivers where Nvidia and Cerebras each have one. Cerebras's $20 billion（約3.2兆円） OpenAI contract and AWS partnership could fast-track adoption, but the premium pricing and vendor lock-in inherent in a complete-system model may limit its addressable market. AMD's simpler, more modular approach and existing customer relationships (OpenAI, Meta, and rumored discussions with Anthropic) position it to capture a broad share of inference deals.