AI hit math gold-medal level 5 years early, experts badly missed

The Forecasting Research Institute has been collecting forecasts on AI progress since mid-2022, drawing on panels that include senior specialists, economists, AI policy experts, and superforecasters with a proven track record. Those predictions were largely gathered before ChatGPT launched, but the pattern of underestimation held afterward too, according to FRI.

The gap was widest in math, but it showed up elsewhere. In virology, experts predicted AI models wouldn't match a top team of virologists on a troubleshooting benchmark until 2030, while superforecasters said 2034 — FRI says that likely happened as early as April 2025. Economic forecasts also proved too conservative: experts put the median for the highest annual recurring revenue of any AI company at the end of 2026 at $20 billion, yet FRI cites roughly $100 billion for Anthropic in September 2026 as a figure that has likely already been reached.

Not every forecast ran too low, though. In a controlled trial on biological lab tasks, only 5.2 percent of participants succeeded with a language model and internet access, compared with 6.6 percent using the internet alone — far below what biosecurity experts and virologists expected. Self-driving car forecasts may also have been too high. Even as respondents revised their expectations upward — the average probability assigned to AI becoming a "technology of the century" rose from 31 to 36 percent for experts over nine months — FRI flags that its interim report is naturally tilted toward finding cases where forecasters were too cautious, since underestimates become obvious the moment reality overtakes a prediction, while overestimates only become clear once a deadline passes.