UC Berkeley: harness cuts AI answer cost 71%

The study's finding arrives as companies weigh how much to spend giving every task access to a state-of-the-art model. The article's example of two startups bidding on the same $250k contract — one calling a state-of-the-art model on every step, the other reserving it for the few steps that need it — shows how that choice flows straight to gross margin: 38% against 75%. The cheaper startup also pays back its sales cost in half the time, which the article frames as the difference between hiring twice as fast and not.

The article argues the cost advantage is hard to copy. Routing to a cheap model is only safe if you know which tasks it clears, and that knowledge comes from watching thousands of versions of the same work — the cost advantage and the moat are the same asset. The more expensive setup loses money on every evaluation past 8,600, forcing it to ration usage just as the customer finds it most valuable.

The article is careful not to promise 100% margins: as inference gets cheaper, buyers will ask more of their agents, shifting the equilibrium. What persists, it says, is the gap between the two companies. Whether that gap holds across models and tasks beyond the 42 comparisons is the open question.