What happened
A University of Pennsylvania statistics professor used OpenAI's GPT-5.6 Sol Pro to disprove a longstanding assumption about the Benjamini-Hochberg procedure, a widely-used statistical method for controlling false positives. The AI constructed a statistical model showing the method's false discovery rate can exceed its target level when data is correlated and normally distributed—something mathematicians had assumed but never proven in 30 years. GPT-5.6 Sol Pro completed the work in about 90 minutes; GPT-5.5 could not find a solution even after roughly 20 hours.
Why it matters
The Benjamini-Hochberg procedure, introduced in 1995, has been cited over 130,000 times and is used across modern statistics and scientific fields to filter out false alarms when testing thousands of hypotheses at once. While the gap the AI found is small in practical terms (0.104 versus the target 0.1), Berkeley statistician Will Fithian called the disproved conjecture 'the most interesting open problem in my area of statistics.' The result signals AI's advancing capability to solve problems that eluded human experts, marking what Fithian described as 'another marker of advancing AI capabilities whose consequences will reach far beyond math.'
What to watch
The AI's solution combined known statistical methods in an unusual way rather than inventing entirely new ones, leaving open the question of whether AI systems can generate genuinely novel knowledge or only recombine what they learned during training. Dobriban notes the result 'mainly matters for theory at this point' and that practical effects need further study. The full chat and code are publicly available.
Summaries like this, in your inbox every morning.
The Benjamini-Hochberg procedure has been a cornerstone of modern statistics since 1995, particularly in fields like genomics where researchers must filter false positives from thousands of simultaneous tests. For nearly three decades, the statistical community assumed the method would work reliably even when data points are correlated—a common real-world scenario, such as when genetic variants are inherited together. However, no one had formally proved this assumption. The University of Pennsylvania's Edgar Dobriban leveraged GPT-5.6 Sol Pro to fill that gap, constructing a counterexample showing that under certain conditions, the false discovery rate can exceed the target level. The speed of the solution is what stands out: 90 minutes versus roughly 20 hours for the previous generation model, and decades of unsuccessful human effort. Dobriban acknowledged that the AI's approach combined existing statistical methods in an unusual way rather than inventing new ones—a pattern seen in similar AI breakthroughs in mathematics. This raises a fundamental question about AI's reasoning: can these systems generate truly novel knowledge, or are they limited to sophisticated recombination of ideas learned during training? Even if recombination is the ceiling, Dobriban's work demonstrates practical value as a tool embedded in human workflows. The broader implications remain uncertain, though some researchers, including deep learning pioneer Richard Sutton, believe genuine self-improvement and generalization require capabilities beyond recombination.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
In Q2, billionaires or their funds sold six 'Magnificent Seven' AI stocks
Nvidia was rising early Friday, even as the wider chip sector kept falling, Barron's reported
ElevenLabs completed a $300 million employee tender at a $22 billion valuation, and co-founder Mati Staniszews…
On October 15, OpenAI announced the Rosalind Academic Plan, giving approved US university labs free access to…
Neostellar Capital principal Evan Schlossman said CoreWeave's $103.7 billion in remaining performance obligati…
A Qiita article compared automated red-teaming methods for LLM apps — GCG, PAIR, and TAP — using mock implemen…
