IBM and Together AI strike $240 million inference cluster deal

IBM and Together AI have announced a $240 million agreement to build and operate an Nvidia-powered AI inference cluster hosted on IBM Cloud. The deal addresses the infrastructure challenge facing AI companies as they move from research into production: running trained models at sufficient scale and speed to serve real users.

Inference — the computational process in which an AI system processes user input and generates output — has emerged as a critical bottleneck. Unlike training, which happens once per model, inference occurs every time a user interacts with the system, and the volume can be enormous. Companies offering AI services need infrastructure that can handle variable demand without unacceptable delays. Together AI, which provides AI model serving and infrastructure, benefits from access to production-grade compute; IBM gains a customer anchor for its AI infrastructure strategy and a foothold in the high-margin AI services market.

The partnership leverages Nvidia's hardware, which remains the industry standard for both training and inference. By embedding this capacity on IBM Cloud, the arrangement allows IBM to compete with hyperscalers (large cloud providers like AWS, Google Cloud, and Microsoft Azure) on a specialized front: dedicated inference capacity rather than broad compute offerings. For Together AI and the developers and companies it serves, the cluster provides a scalable on-demand alternative to building private infrastructure or renting from public cloud providers on spot-market terms.