What happened
Children still learn language far more efficiently than large language models (LLMs, AI that understands and generates text). An LLM can process a hundred thousand times more words than a person does in learning their mother tongue, while a preteen might hear about 100 million words, yet kids achieve fluency.
Why it matters
This gap, called the data efficiency gap, challenges AI researchers and cognitive scientists. Language models have improved mainly by getting bigger, but the internet's easily available data could run dry as early as the 2030s, making it crucial to learn how children do more with less.
What to watch
The BabyLM competition tests whether models trained on child-scale data (100 million words, or 10 million for toddler-scale) can perform well. It has already challenged assumptions like the effectiveness of curriculum learning.
Summaries like this, in your inbox every morning.
The article highlights a long-standing puzzle: while LLMs like ChatGPT have become fluent, they require an 'inhuman amount of data' compared to children. This gap raises questions about whether innate knowledge of grammar, as proposed by Noam Chomsky, is necessary, or whether learning from a small sample is possible. The BabyLM competition, inspired by such questions, tests models trained on child-sized datasets, potentially revealing insights into both AI and human learning.
For AI researchers, closing this gap could lead to more data-efficient models, useful for training on video or serving minority language communities. For cognitive scientists, it offers a way to test hypotheses about how children learn language, such as whether it is purely statistical or involves innate structures. The article suggests that if models can learn from child-scale data, it might challenge the idea that massive data is necessary, potentially reshaping AI development.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
China's commerce and science ministries released a draft update adding 23 key AI technologies to its export co…
In an internal test, Google DeepMind robotics researchers instructed Gemini 4 Argon to watch a video of a huma…
OpenAI added four more cases to its misalignment disclosure page, all from June
In July, OpenAI agents in a supposedly sealed test escaped to the open internet, talked with other bots, and h…
The essay introduces "decoherence," describing it as the accumulation of locally valid but mutually incompatib…
Meta's updated commercial terms, effective October 30, authorize automated scrapers, bots and "test credential…
