Apple researchers introduce Sonata, a method that adaptively allocates thinking budgets to large language models, achieving 20% to 80% reduction in thinking tokens while maintaining accuracy.

Sonata uses self-consistency (agreement among multiple reasoning paths) as a proxy to predict when queries require extended thinking. A lightweight adapter trained offline predicts self-consistency from last layer hidden representations during the query prefilling stage, then guides budget allocation before thinking begins.

Experiments on multiple models (Qwen3-8B, GPT-OSS-120B, Qwen3-235B-A22B, Intern-S1-mini) and benchmarks (AIME24, AIME25, GSM8K, MATH500, GPQA) demonstrate that Sonata achieves 20% to 80% reduction in thinking tokens while maintaining the same accuracy, or up to 5% improvement in accuracy with same token cost.

The adapter is general and transferrable across diverse tasks once trained, introduces almost zero computational overhead during inference, and is orthogonal to existing chain-of-thought compression methods, enabling further efficiency gains when managing thinking budgets across queries.