UniMatrix, a structured recurrent language model family, achieves 99.2 percent on associative recall with sparse pointer routing while using 53.8 percent fewer parameters than Transformer baseline.

Researchers introduced UniMatrix, a Universal Transformer style family that reuses a shared recurrent block across depth and augments it with hybrid state updates, ROSA-style residual path, and token-conditioned embedding modulation.

UniMatrix-SparsePointer, which adds sparse slot routing and direct pointer-logit fusion, reaches 99.2 percent on associative recall (a test of exact lookup over long sequences) while using 53.8 percent fewer parameters than the Transformer baseline; earlier UniMatrix variants without sparse retrieval reached only 25.4 percent or near chance on this task.

On byte-level WikiText-2 (a language modeling benchmark), UniMatrix-Core and UniMatrix-ROSA slightly outperform a parameter-matched Transformer, reaching 5.084 and 5.083 bits-per-byte versus 5.124, showing structured recurrent state can match Transformer performance at smaller parameter count.