What happened
Artificial Analysis released Intelligence Index v4.2 on September 4, adding AA-Briefcase and GDP.pdf tests while dropping GPQA Diamond, which had saturated scores.
Why it matters
Private data now makes up 40% of the index, double v4.1's share, aiming to prevent AI developers from gaming benchmarks; v5 will raise it further.
What to watch
GPT-6 Astra rose 4 points from GPT-5.6 Sol, and showed higher output token efficiency than other frontier models.
WHO IT HITSEnterprise AI buyers and technical evaluators who rely on benchmark rankings to choose between large language models will now see scores less affected by AI developers' test-specific optimization.
Summaries like this, every morning.
Artificial Analysis's v4.2 update arrives as AI developers commonly train models against public benchmarks, pushing scores higher without reflecting real-world usefulness. By introducing AA-Briefcase and GDP.pdf, and removing GPQA Diamond, the firm aims to capture abilities that were not being measured well.
The heavier weight on private test data represents a structural shift in how this index gauges model quality. The publisher states the purpose is to prevent benchmark gaming, and it plans to increase the private-data share even more in v5. This suggests ongoing cat-and-mouse dynamics between evaluators and developers.
For buyers selecting AI models for enterprise tasks, the stakes are in whether the new tests genuinely track practical performance. The outcome hinges on how well AA-Briefcase and GDP.pdf resist optimization compared to older benchmarks, possibly determining whether the top rank reflects true capability or just better test preparation.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Microsoft said on October 21 that attackers tied to China's government are automating intrusions with ARTEX, a…
1Password unveiled 1Password for Agents, which lets an AI agent request credentials from a user's vault and, o…
Developer Simon Edgerson published WorldTUI, which uses a roughly 360M-parameter model to predict the next ter…
A new paper found Raw-RAG—dense retrieval over raw history—ranked first in 6 of 7 benchmark-model settings on…
Nvidia has won a CPU supply order from AI startup Modal for its Vera CPU, moving beyond GPUs into general-purp…
In a Google blog post summarizing the video "The Agent Factory" on Sept 25, 2026, Ryan Lopopolo defined an age…
