Researchers develop mathematical framework explaining how LLMs convert continuous internal computations into discrete tokens through latent semantic manifolds

Study introduces latent semantic manifold theory using Riemannian geometry and Fisher information metric to model how transformer architectures internally represent and discretize language

Defines 'expressibility gap' as a geometric measure quantifying semantic distortion caused by vocabulary discretization, with proven rate-distortion lower bounds

Validates findings across six transformer architectures ranging from 124M to 1.5B parameters, discovering universal hourglass intrinsic dimension profiles and linear gap scaling patterns

Reveals smooth curvature structure in hidden state geometry, suggesting fundamental constraints on how any finite vocabulary can encode semantic information