Study finds students using AI to generate answers see decreased understanding, while those using AI for explanations improve

A 2025 paper (AI Meets the Classroom: When Does ChatGPT Harm Learning?) found that students in a programming course who used LLMs to generate answers to questions experienced a significant decrease in topic understanding, even though they perceived an increase in learning.

Students who used AI to complement learning activities—such as asking for explanations rather than answers—saw an increase in understanding. However, nearly half of all solution requests came without the student making a single attempt first.

The author argues that knowledge workers need to deliberately spend time each day exercising cognitive skills near their limits (performing deep System 2 Thinking, a form of effortful reasoning) to avoid skill decay as they rely more on AI agents. The author experiments with allocating at least 30 minutes daily to mentally difficult tasks like writing code by hand or reading difficult papers.