What does PROOF-Gen do differently?

Instead of discarding failed training attempts, PROOF-Gen analyzes them and writes corrective guidance to help the teacher model produce a passing example. This guidance is removed before training the student model.

What results did PROOF-Gen show?

On τ2-bench, it recovered 93% of failed scenarios. A model fine-tuned with it improved from 0.132 to 0.529 on Pass@1, and a deployed pipeline saw a +6.3pp gain in goal completion.

Large Language ModelsAI Business & IndustryApple Machine LearningAug 26, 2026Aug 27, 2026, 04:00 JST

Apple's PROOF-Gen turns AI training failures into wins

Send on LINE

3 Key Points

What happened
Apple researchers introduced PROOF-Gen, a method that recovers successful training examples from failed AI attempts. On τ2-bench, it recovered 93% of failed scenarios, and models fine-tuned with it showed big gains, like Qwen3-4B improving from 0.132 to 0.529 on Pass@1.
Why it matters
Standard AI training discards failed attempts, but most failures are near-misses—two-thirds have only one decisive error. PROOF-Gen uses these to create better training data, which boosts performance in deployed systems, including a +6.3pp improvement in goal completion and positive results on-device.
What to watch
The method improved non-English locales by an average of +1.48pp, suggesting it could make AI assistants more capable globally. This could be a practical way for companies to improve AI without relying solely on expensive frontier models.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Get news like this every morning, free

Context & Analysis

This research tackles a hidden inefficiency in AI development: most training pipelines discard failed attempts, even though two-thirds of those are near-misses with just one mistake. By recovering these failures, PROOF-Gen turns wasted compute into useful training data, which is a more cost-effective way to improve models.

The method's success on non-English locales is particularly notable, as it suggests a way to improve global AI performance without needing more human-curated data in every language. This could make AI assistants more reliable for users worldwide, a key goal for companies like Apple that ship products internationally.

For businesses, the implication is that smarter use of existing data—even failures—can yield significant performance gains. This could reduce the need to constantly rely on the most expensive frontier models, making AI development more accessible and sustainable.

FAQ

What does PROOF-Gen do differently?: Instead of discarding failed training attempts, PROOF-Gen analyzes them and writes corrective guidance to help the teacher model produce a passing example. This guidance is removed before training the student model.
What results did PROOF-Gen show?: On τ2-bench, it recovered 93% of failed scenarios. A model fine-tuned with it improved from 0.132 to 0.529 on Pass@1, and a deployed pipeline saw a +6.3pp gain in goal completion.

Apple Machine LearningRead Original Article

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Get it free

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.