GPU prices double to $8.08/hr as AI gets cheaper

The article frames a puzzle: GPU rental prices have doubled in six months, to $8.08 per GPU-hour from $4.40, yet AI inference keeps getting cheaper. Tom Tunguz points to the buildout's rising input costs — concrete, copper, credit — with electricity as the limiting factor. Oracle's force majeure on its New Mexico campus, after the natural-gas pipeline feeding it slipped by six months, illustrates how physical delays ripple through plans.

At the same time, demand from inference and model companies is posting record-setting growth, emboldening investors to value them more richly and bid up limited GPU-hours. Efficiency is improving just as fast: the same benchmark threshold that cost $0.55 eighteen months ago now clears for $0.0015, a 377x reduction. Capital itself is more expensive too, and the historical relationship between the 10-year Treasury and tech valuations has flipped — the correlation went from −0.50 over the full history to +0.39 over the last two years, while rates climbed from 3.63% to 5.18% and the NASDAQ rose roughly 78%.

Tunguz's reading is that the industry is holding both forces in balance for now, with gross profit dollars per GPU-hour as the ultimate test. Whether that balance holds seems to hinge on whether efficiency gains — like Microsoft's 90% more tokens per GPU year over year, concentrated in smaller models — continue to run neck and neck with rising GPU costs. If they do not, the economics for the companies paying those GPU-hour rates could shift.