Apple's RLTL;DR turns failed attempts into a 14–31% Pass@1

The paper addresses a known problem in reinforcement learning with verifiable rewards: when tasks are so difficult that the agent has a low or even no chance of success, standard training cannot find successful attempts to optimize toward. The authors filtered datasets to Pass@128 = 0 specifically to study this regime, where no teacher models or example solutions are available to distill from. In that setting, standard GRPO training of a Qwen 3.5 9B Thinking policy stays flat at a Pass@1 of 0% to 1%, a learning barrier the paper sets out to break.

The proposed method changes where the learning signal comes from. Instead of learning only from successful rollouts, the policy is shown the verifier outputs after each failure and writes its own TL;DR insight, with subsequent rollouts conditioned on all previous insights. The authors report that backpropagating on these in-context insights—internalizing a direct task → insight mapping—is the key, since the policy still achieves a 12–13% Pass@1 at eval time when no insight is in context. The compacted SFTL;DR variant, trained on only 4k (task, insight) tuples, recovers almost the full performance, which the authors present as evidence for a training paradigm of the form 'on this sort of task, keep this sort of thing in mind'.

What the result ultimately means hinges on whether this task → insight internalization generalizes beyond the tool-calling and coding datasets used here; the paper positions the compacted paradigm as an inspiration for future research rather than a settled replacement for existing approaches.