AI Index overhauled after GPT-6 Astra scoring skepticism

The revision comes after earlier evaluations, including OpenAI's own, showed Astra far ahead of the field, while Artificial Analysis initially scored it just on par with its predecessor. This discrepancy drew skepticism, prompting the index overhaul. The new version adds benchmarks like AA-Briefcase for real-world knowledge work and GDP.pdf for PDF document analysis, and drops GPQA-Diamond because models have solved it. By making private test data 40 percent of the weighting, the index aims to resist gaming.

For business readers, the update matters because it provides a more accurate picture of frontier model capabilities, which is useful for procurement and strategy decisions. The fact that Astra now shows a four-point gain and uses fewer tokens per task suggests cost and efficiency advantages, which could influence vendor choices. However, as the index is still evolving, and version 5 will roll out in stages, these rankings may continue to shift. The top of the leaderboard is moving quickly, and this was an interim adjustment to keep pace.