Rebellions' Marshall Choy: commoditize AI inference to grow the market

Choy's argument runs against what he calls the conventional wisdom that commoditization destroys value. He notes the technologies that reshape industries rarely remain scarce, pointing to electricity, broadband, cloud computing and storage — all of which saw demand explode as they became more affordable, reliable and easier to deploy. He applies the same logic to AI inference.

The current state of AI, as Choy describes it, treats AI as a precious resource. Engineering teams ration token usage, throttle API calls and cap deployments to keep cloud bills from spiraling, and even Microsoft Corp. reportedly limits AI usage. Choy's analogies — the $27 artisanal truffle versus the 99-cent chocolate bar, and the multi-million-dollar dragster versus the Toyota Camry and Ford Transit — are meant to show that a mass market rewards consistency, affordability and everyday reliability over peak performance under ideal conditions.

The stakes hinge on whether the industry actually changes how it defines and measures performance. Choy argues that generated lines of code, requests per second and benchmark scores are engineering metrics, not business outcomes, and that businesses invest in AI to get more done rather than to produce more tokens. Whether infrastructure providers and buyers align on outcome-based measures like task completion rates and time or money saved will determine how quickly inference becomes embedded in everyday operations, according to Choy's reading.