Transformers learn arithmetic structure early but bottleneck in the decoder, causing grokking delays that can be eliminated with targeted retraining.

In encoder-decoder arithmetic models, the generalization delay in grokking reflects limited decoder access to learned structure rather than failure to acquire it

Encoders organize parity and residue structure within the first few thousand training steps, while output accuracy remains near chance for tens of thousands more steps

Transplanting a trained encoder into a fresh model accelerates grokking by 2.75x, while transplanting a decoder actively hurts performance

Freezing a converged encoder and retraining only the decoder achieves 97.6% accuracy versus 86.1% for joint training, eliminating the plateau entirely