AIToday
AI Business & IndustryAI WeeklyPublished: Nov 16, 2026, 04:00 JST

Buzzard: 38 of ~290 OpenAI Lean files miss main theorem

Send on LINE
Buzzard: 38 of ~290 OpenAI Lean files miss main theorem

3 Key Points

  1. What happened

    Kevin Buzzard of the Xena Project checked OpenAI's October 2026 release of 722 mathematics papers, about 45% of which came with Lean code. Of roughly 290 Lean files, 38 do not state, let alone prove, the paper's main theorem.

  2. Why it matters

    The 38 files fail the most basic check — that the code states the theorem the paper claims — in a release widely reported as a landmark. Buzzard's four-step standard, covering statement match, compilation, a lake comparator run and a paranoid kernel check, is replicable by any lab with Lean tooling, though he notes the remaining three steps have not yet been completed for many papers.

WHO IT HITSResearchers and reviewers who assess AI-generated formal proofs, and labs that publish Lean code alongside papers, now have a concrete checklist — statement match, compilation, lake comparator, kernel check — for validating claims before citing or building on them.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

OpenAI's October 2026 release of 722 mathematics papers was widely reported as a landmark in AI-driven mathematics. The check by Kevin Buzzard, who runs the Xena Project at Imperial College London, tests a narrower question: whether the Lean code that accompanies a paper actually states the theorem the paper claims. Of the roughly 290 files with Lean code — about 45% of the 722 papers — 38 fail that test.

Buzzard does not stop at the finding. He lays out a four-step verification standard: does the Lean statement match the informal statement in the PDF, does the code compile, does a lake comparator run confirm the proof, and does a paranoid kernel check rule out exploited bugs in Lean itself. He states that frontier AI models are capable of performing all four checks, though at potentially high computational cost.

The standard is replicable by any lab or researcher with access to Lean tooling. His post also notes that for many papers in the dump, the remaining three verification steps have not yet been completed, leaving the broader verification picture incomplete beyond the 38 files that fail the first step.

FAQ
How many of OpenAI's 722 papers came with Lean code?
About 45% of the 722 papers came with Lean code, giving roughly 290 files for Buzzard to check.
What are the four verification steps Buzzard proposes?
He asks whether the Lean statement matches the PDF's informal statement, whether the code compiles, whether a lake comparator run confirms the proof, and whether a paranoid kernel check rules out exploited bugs in Lean itself.
Have the remaining checks been completed for the other papers?
No. Buzzard's post notes that for many papers in the dump, the remaining three verification steps have not yet been completed.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleBias-variance trade-off explained as a G検定 staple