Researchers discover that a 2.6B-parameter looped transformer encodes human preferences through relational comparisons rather than absolute judgments, achieving 95.2% accuracy in preference prediction.

Scientists analyzed Ouro-2.6B-Thinking, a looped transformer with iterative refinement, to understand how it internally represents human preferences across iteration states

Lightweight evaluator heads (~5M parameters) trained on pairwise differences achieved 95.2% test accuracy on 8,552 unseen examples from the Anthropic HH-RLHF dataset

Key finding: preference encoding is relational—linear probes on pairwise differences scored 84.5%, while independent classification on individual states scored only 21.75% (below chance), indicating the model compares options rather than evaluating them in isolation

The frozen base model's preference representation functions as an internal consistency probe, measuring how stably the transformer refines its evaluations through looped iterations