AMD buys Taalas to challenge Nvidia in AI inference chips

On August 6, AMD announced its agreement to acquire Taalas, a chip startup focused on artificial intelligence inference. While the financial terms were not disclosed, the deal represents AMD's latest move in a broader campaign to compete with Nvidia in AI inference—a phase of AI deployment that has become increasingly critical as applications scale. Vamsi Boppana, senior vice president of AMD's Artificial Intelligence Group, stated that "AMD is building a full-stack AI platform that gives customers the flexibility to deploy the right compute solutions for every AI workload," underscoring the company's intent to offer integrated systems combining specialized inference chips with its Instinct GPUs rather than standalone processors. AMD has pursued this strategy aggressively over recent months, having acquired MK1 (an AI software startup specializing in high-speed inference) in November, MEXT in June, and added FastFlowLM to its AI group in July. Together, these acquisitions aim to strengthen AMD's inference capabilities against established competitors. Taalas brings distinctive advantages to AMD's portfolio. The company's HC1 demonstrator reportedly delivers around 17,000 tokens per second per user when running Llama 3.1 8B—a measure of throughput. More significantly, Taalas claims its systems cost 20 times less to build and use 10 times less power than competing solutions because they avoid HBM (high-bandwidth memory), advanced packaging, and liquid cooling, all of which are expensive and power-intensive components in traditional GPU-based inference systems. The timing of AMD's acquisition follows closely on Nvidia's $20 billion deal with Groq for inference technology and talent, announced mere months earlier. This competitive rush reflects the industry's recognition that inference computing—the process of answering user queries—has emerged as a major battleground. While Nvidia chips have dominated AI model training for years, inference presents a different challenge: as AI assistants, coding agents, and similar applications handle billions of queries, latency, throughput, and power efficiency become increasingly critical. Nvidia's traditional graphics processors now face greater competition from central processing units and custom processors designed specifically for inference, a shift that has prompted both Nvidia and AMD to invest heavily in specialized inference solutions.