Distinguishing performance gains from learning when using generative AI

Core Argument

This Nature Reviews Psychology piece draws a critical distinction that has been under-theorized in AIED research:

The authors argue that generative AI easily boosts performance but often bypasses the mental processes essential for genuine knowledge acquisition. This challenges the common assumption in AI Tutor Effectiveness Review research that improved task performance equals improved learning.

Implications for AIED Design

This distinction has profound implications for Scaffolding design. Systems that optimize for immediate performance may undermine learning. They must be designed to promote cognitive engagement — for example, through Socratic Method dialogue, constrained feedback, or requiring student articulation before revealing AI output. The piece connects to Over Reliance research showing that AI assistance can reduce independent problem-solving and to the Pedagogy AI Mistakes paradigm that uses AI errors as learning opportunities.

The performance-vs-learning gap is now causally demonstrated in a field RCT: Generative AI Guardrails Harm Learning shows that an unguarded GPT-4 tutor raised practice performance +48% but reduced later unassisted exam scores by 17%, while a guardrailed "hint-not-answer" tutor eliminated the harm.

Theoretical Framework

Generative AI can function as either a cognitive tool (amplifying thinking) or a cognitive crutch (replacing thinking). This maps onto the Self Regulated Learning cycle — performance gains without metacognitive engagement short-circuit the planning-monitoring-evaluating loop.

Connected Concepts

Connected Articles

Citation

Yan, L., Greiff, S., Lodge, J. M., & Gašević, D. (2026). Distinguishing performance gains from learning when using generative AI. Nature Reviews Psychology, 4(7), 435-436. arXiv:2605.13731.