Apple's SSR cuts agent reasoning latency over 90%

The researchers frame the problem plainly: multimodal agents typically generate freeform reasoning before each action, and small models with limited capacity often produce long reasoning that gives little useful guidance while costing a lot at inference time. SSR attacks that mismatch by turning reasoning into a choice among pre-specified, reusable natural-language candidates, selected by their likelihood given the current context, with no auxiliary task head.

That design choice enables a specific technical payoff: parallel scoring. Because the reasoning traces are pre-specified, teacher-forced prefilling can compute token likelihoods concurrently within and across candidates using a shared context KV cache. The evaluation covered seven multimodal search benchmarks with 2B and 4B models, across multiple reinforcement learning objectives and supervised fine-tuning.

The results were gains in efficiency without a drop in task performance: an average success rate competitive with leading search agents of the same scale, per-turn reasoning latency reduced by over 90%, and total per-question model inference latency reduced by 28–54%. The work was done in part while several authors were at Apple, with academic affiliations including Carnegie Mellon University and the University of Illinois Urbana-Champaign.