Does Using AI Actually Help My Students Learn?

Sometimes—but improved AI-assisted performance is not the same thing as learning. AI appears most educationally useful when it provides feedback, hints, explanations, practice, or adaptive support while leaving the learner responsible for the cognitive work that the learning objective requires. When AI instead supplies the reasoning, drafting, problem solving, or answers students need to practice themselves, performance can rise while later independent performance falls.

The knowledge base calls this the performance–learning gap. According to the AI Misuse and Learning Harm synthesis, one field RCT with roughly 1,000 high-school mathematics students found that an unguarded AI tutor substantially improved AI-assisted practice performance but reduced later unassisted exam performance; a guardrailed, hint-oriented version avoided that learning penalty. Because this is evidence from a particular mathematics context, it should not be assumed to generalize unchanged to every discipline.

A useful design principle: "scaffold, do not substitute"

Ask students to attempt, retrieve, explain, generate, or make a judgment before AI intervenes; have AI provide hints rather than finished answers where appropriate; and require students to explain, critique, verify, or revise AI output. The knowledge base's Active Learning synthesis similarly emphasizes keeping learners in constructive and interactive forms of engagement rather than having them passively consume AI output.

How to find out whether AI is actually helping in your course

Measure learning after assistance is removed. The AI Ed Evaluation page recommends distinguishing AI-assisted task performance from unassisted learning, retention, and transfer — because what students can do with AI in front of them is not evidence of what they can do on their own.