Agentic AI shifts focus from training to storage, memory, power efficiency

What happened

At RAISE Summit, infrastructure leaders revealed that agentic inference—longer-running AI agent tasks—is creating new bottlenecks in storage, memory, and power consumption. Solidigm's Greg Matson explained that storage has moved into the critical path, becoming "a whole new storage tier that's being created to extend the memory for the system." AMD is optimizing across CPUs, GPUs, and networking rather than individual chips; Tensordyne's Napier inference chip uses logarithmic math to cut power draw to 30 kilowatts (versus 150 kilowatts for a comparable Nvidia system); and d-Matrix is pairing purpose-built accelerators with GPUs for heterogeneous inference in production.

Why it matters

As enterprises move from single-task AI workloads to long-running agentic systems, the infrastructure bottleneck has shifted from raw compute speed to keeping GPUs continuously fed with data and managing power costs. Storage positioned near accelerators is now critical to prevent idle GPU time—which wastes capital since GPUs are the most expensive part of the infrastructure. This reshaping of the AI stack means businesses cannot rely on GPU capacity alone; they must rethink storage, memory staging, and power architecture to stay competitive.

What to watch

Data sovereignty and financing are emerging as infrastructure concerns alongside hardware. Argentum AI is addressing capital constraints by securing customer contracts first, then financing construction—a model that treats "power, compute and capital" as an integrated product. Agentcy Labs and Neo4j are advancing knowledge graphs as a way to give enterprises deterministic control and explainability alongside large language models, positioning sovereignty (territorial, operational, and stack control) as a core infrastructure requirement for agentic deployments.