Apple's new method, PROOF-Gen, turns AI training failures into useful data.
It recovers 93% of failed scenarios on a benchmark.
This improves model performance significantly, especially for non-English users.
What happened
Apple researchers introduced PROOF-Gen, a method that recovers successful training examples from failed AI attempts. On τ2-bench, it recovered 93% of failed scenarios, and models fine-tuned with it showed big gains, like Qwen3-4B improving from 0.132 to 0.529 on Pass@1.
Why it matters
Standard AI training discards failed attempts, but most failures are near-misses—two-thirds have only one decisive error. PROOF-Gen uses these to create better training data, which boosts performance in deployed systems, including a +6.3pp improvement in goal completion and positive results on-device.
What to watch
The method improved non-English locales by an average of +1.48pp, suggesting it could make AI assistants more capable globally. This could be a practical way for companies to improve AI without relying solely on expensive frontier models.
Ask the AI about this article →
This research tackles a hidden inefficiency in AI development: most training pipelines discard failed attempts, even though two-thirds of those are near-misses with just one mistake. By recovering these failures, PROOF-Gen turns wasted compute into useful training data, which is a more cost-effective way to improve models.
The method's success on non-English locales is particularly notable, as it suggests a way to improve global AI performance without needing more human-curated data in every language. This could make AI assistants more reliable for users worldwide, a key goal for companies like Apple that ship products internationally.
For businesses, the implication is that smarter use of existing data—even failures—can yield significant performance gains. This could reduce the need to constantly rely on the most expensive frontier models, making AI development more accessible and sustainable.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
Ask AI anything about this article. Q&As are published on this page for other readers too.
Netflix is making a competition reality show based on Willy Wonka & the Chocolate Factory, and it will use AI…
JPMorgan has softened its stance on share-backed loans, as reported by Private Banker International, to target…
Palantir CEO Alex Karp said at a company boot camp last week that society cannot accept all AI value going to…
Goldman Sachs has deployed AI agents in its banking operations, but they are proving difficult to fully replac…
Google has moved its AI-responsibility team out of the DeepMind lab, according to an exclusive report by The W…
Mark Zuckerberg had a bold plan to replace Meta staff with AI, but the plan imploded, according to the article
