Nvidia's Ian Buck ties AI factory economics to tokens per watt

Buck's argument is that the unit of account for an AI data center has changed. Rather than counting servers, cars or devices, he told theCUBE Research's Dave Vellante and John Furrier that the output is tokens, describing them as appreciating, revenue-generating, fungible, durable and productive parts of an economy. That framing matters because it shifts attention away from the chips themselves and toward the networking, storage, processors and software that must work together at scale.

He also pushed back on the idea that inference replaces training. As companies use models, he said, they refine, align and add data to them, which amounts to a little bit of training, visible in reinforcement learning and online alignment work. A second economic tier is emerging for low-latency workloads where faster reasoning commands a higher value, which is where the Groq 3 LPX and Vera Rubin combination fits.

The stakes appear to hinge on execution rather than demand. Power capacity caps how much infrastructure a site can deploy, so the pressure is on vendors to keep raising tokens per watt with each generation. Buck pointed to Blackwell as evidence of that pace and to CoreWeave's menu of configurations and higher-level inference services as a way to spare customers from overwhelming choices. Whether that partner-led model holds as agentic systems pull on multiple models, databases and tools is likely to shape how quickly AI factories convert power into revenue.