Weka cuts AI GPU load with cached tokens, launches storage hardware

GPU memory has become the critical constraint in scaling AI inference. Long context windows—the amount of text a model can consider at once—and multi-turn conversations create a compounding problem: models must repeatedly recompute information they have already processed, consuming precious GPU memory and compute capacity. This means fewer concurrent users can be served per GPU, forcing costly hardware expansion. Weka's answer is to bypass this constraint by extending GPU memory into the storage tier. The company's NeuralMesh 6 platform, released alongside Wekapod 3 hardware, implements what Weka calls the Augmented Memory Grid—an architecture that aggregates NAND flash storage to function as GPU memory, but at a fraction of the cost. The key insight is that flash storage can cache 100% of an AI model's pre-calculated tokens, eliminating the need to recompute them on every turn of a conversation. This allows the same GPU to serve more users or handle longer sessions without additional silicon investment. Wekapod 3 marks Weka's entry into custom hardware, a signal that the company believes its software-plus-hardware stack offers an integrated advantage. The timing reflects industry momentum: Dell, NetApp, Pure Storage, and VAST have all repositioned toward AI infrastructure over the past two years, each competing for the same opportunity to reshape how enterprises handle the compute-memory tradeoff. Weka's claim is that it has built from the ground up for this challenge, but the crowded field suggests customers will have multiple architectures to evaluate before settling on a vendor.

The bottleneck Weka addresses is real and acute. In production AI systems, GPU memory is not only the costliest resource but also the scarcest—long context windows and multi-turn conversations force models to repeatedly recompute information they have already processed, wasting both GPU memory and compute cycles that could serve additional users or generate new outputs. Weka's strategy is to treat this not as an inevitable constraint but as an opportunity to extend GPU memory with cheaper flash storage technologies, using flash as a high-speed cache layer for pre-calculated tokens. This shift in architecture—from GPU-centric to hybrid GPU-plus-storage—reflects broader industry recognition that traditional infrastructure design leaves compute underutilized when memory becomes the limiting factor. However, Weka does not enter this space alone. Dell, NetApp, Pure Storage, and VAST have all shifted their positioning toward AI infrastructure within the past two years, each claiming similar purpose-built solutions. The emergence of this crowded competitive field suggests both strong market demand and uncertainty about which vendor's approach will dominate, making execution and customer adoption critical for Weka's differentiation.