Researchers demonstrate that large language models cannot effectively encode the space of possible questions into their weights during training.

The preprint argues that LLM weights have fundamental limitations in capturing the full dimensionality of question-space

This limitation suggests that question-answering capabilities cannot be fully 'baked in' during the model training process

The research implies that LLMs may require dynamic, runtime mechanisms to effectively handle diverse and novel questions rather than relying solely on learned weights