New LGSE framework improves language model adaptation for low-resource languages by using morpheme-based embedding initialization instead of random vectors

LGSE (Lexically Grounded Subword Embedding Initialization) addresses vocabulary expansion challenges in morphologically rich, low-resource languages

The framework decomposes words into morphemes and constructs embeddings by averaging pretrained subword or FastText morpheme representations

For tokens that cannot be meaningfully segmented, character n-gram representations are used to capture structural information

Method integrates with Language-Adaptive Pretraining to improve semantic coherence and preserve critical morphological information