Agentic AI uses 10 to 100 times more tokens per task: Futurum

The Futurum report, "The Off Ramp From Per-Token Pricing," was sponsored by neocloud provider QumulusAI Inc. and arrives as enterprises are already shifting how they buy AI compute. According to Futurum's survey of 824 AI decision-makers, reserved and owned infrastructure account for 66% of AI compute consumption, compared with 19% for on-demand cloud, and 59% of respondents primarily run AI workloads outside hyperscaler public clouds, in their own data centers, colocation facilities, or with bare-metal providers.

The report also notes that the offramp works well for open-weight models a company can deploy on infrastructure it controls, which is why Amberd.ai built on private, open-source large language models. But many enterprises have standardized on frontier models available only through their developers' application programming interfaces or hyperscaler marketplaces. For those workloads, no bare-metal alternative exists, so the pricing lever rests with the model provider.

Whether the offramp pays off may hinge on operational skills, since reserved infrastructure only saves money if it stays busy. The report acknowledges that these environments require more custom engineering, limiting the operating margin gains for teams without the hardware expertise. Enterprises that lack deep bench strength in serving engines, batching, quantization and key-value cache management may find the offramp leads to a different kind of cost overrun.