Children vs. AI: the data efficiency gap

The article highlights a long-standing puzzle: while LLMs like ChatGPT have become fluent, they require an 'inhuman amount of data' compared to children. This gap raises questions about whether innate knowledge of grammar, as proposed by Noam Chomsky, is necessary, or whether learning from a small sample is possible. The BabyLM competition, inspired by such questions, tests models trained on child-sized datasets, potentially revealing insights into both AI and human learning.

For AI researchers, closing this gap could lead to more data-efficient models, useful for training on video or serving minority language communities. For cognitive scientists, it offers a way to test hypotheses about how children learn language, such as whether it is purely statistical or involves innate structures. The article suggests that if models can learn from child-scale data, it might challenge the idea that massive data is necessary, potentially reshaping AI development.