ScaleFlux's Balaji: AI factories stall as storage runs almost 800 times slower than GPUs need

The starting point for ScaleFlux's argument is a simple imbalance: a GPU operates at about 20 to 22 terabytes per second, but each tier below it adds delay. As Balaji put it, DRAM is almost 37 to 40 times slower than what the GPU wants, and storage is almost 800 times slower. The bottleneck, in his description, is not any single component but the act of data moving from the GPU into memory and then into storage, where the delay gets amplified.

That framing is what leads ScaleFlux to push NAND flash beyond its usual role. Rather than treating flash as a passive archive, Balaji said an intelligent SSD controller or a CXL memory controller can exploit NAND's characteristics and use it as a memory device, particularly for key-value cache offload — the store of conversational context that inference workloads rely on. The company's stated evidence is more than 200 flexible data placement streams and 7 to 10 drive writes per day on standard triple-level-cell NAND, which it says it can enable alongside a full reference design and a contract manufacturing partner. Because ScaleFlux is NAND agnostic, customers can build around whatever flash they can source in a tight supply market — a practical consideration given the supply conditions Balaji referenced.