Cast AI data shows companies provision 20 times more GPU capacity than they use, with average GPU utilization at 5%

Cast AI analyzed 23,000 clusters across thousands of companies and found that average GPU utilization across enterprise servers is at 5%, meaning roughly 95% of provisioned GPU capacity is not being used; CPU utilization is similarly low at 8% of total capacity.

Cast AI President Laurent Gil attributed the overbuying to long-term GPU contracts driven by limited supply and fear of missing out, rather than actual demand. Unlike traditional cloud computing where resources can be scaled up and down on demand, companies commit to fixed GPU contracts.

An unused GPU can waste several dollars per hour, compared to a few cents per hour for an idle CPU. GPUs can be up to 50 times more expensive than comparable CPU-based machines. Cast AI's report identifies a healthy GPU utilization rate as around 50%.