Does Generative Artificial Intelligence Improve Students' Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments

Synthesis: This meta-analysis integrates 29 experimental and quasi-experimental studies to quantify the overall effect of Generative AI on learners' higher-order thinking (HOT) and examine moderators. GenAI exerts a moderate positive effect on HOT, strongest for Problem Solving, followed by critical thinking, with a relatively limited effect on creativity. Effects are significantly moderated by intervention duration (strongest at 8–16 weeks) and learners' Self-Regulated Learning abilities (higher SRL → more benefit).

Key Findings

1. Moderate overall positive effect on higher-order thinking. Across 29 studies, GenAI improved learners' HOT to a moderate degree.
2. Problem-solving > critical thinking > creativity. The most significant improvement was in problem-solving abilities, followed by critical thinking, while the effect on creativity was relatively limited.
3. Duration matters. Effects were strongest when interventions lasted 8–16 weeks — medium- to long-term integration is recommended.
4. Self-regulated learning amplifies benefit. Learners with higher SRL capacities benefited more substantially, suggesting dynamic, personalized Scaffolding of SRL maximizes GenAI's educational potential.

What this means for practice

• Instructors. Plan GenAI integration across a semester rather than a single session: the effect on higher-order thinking peaked for 8–16-week interventions (ES = 0.759) and fell away both below 8 weeks (ES = 0.494) and beyond 16 weeks (ES = 0.372).
• Instructors. Scaffold self-regulated learning alongside the tool instead of assuming it: students with high SRL gained substantially (ES = 0.863) while those with low SRL gained little (ES = 0.284), so teach goal-setting, monitoring, and reflection as part of the GenAI activity.
• Instructors. Give creative thinking explicit support: the GenAI effect was weakest for creativity (ES = 0.444, against 0.745 for Problem Solving and 0.691 for critical thinking), so pair the tool with ideation and divergent-thinking prompts rather than expecting creative gains to follow.
• Administrators. Move GenAI use out of lecture-only formats: lecture-based instruction produced a small, non-significant effect (ES = 0.396) compared with project-based (ES = 0.717) and blended (ES = 0.525) designs.
• Researchers. Report higher-order-thinking sub-dimensions separately, since the moderate overall effect (Hedges's g = 0.609) conceals a 0.444–0.745 spread across creativity, critical thinking, and problem-solving.

Limitations

• The synthesis covers 29 experiments and quasi-experiments restricted to studies published in Chinese and English, with a narrow range of educational contexts and learner populations — a language restriction the authors state could omit literature from other linguistic contexts and restrict external validity.
• Heterogeneity was substantial (Q = 255.208, p < 0.001; I² = 77.273%), so the pooled effect of g = 0.609 averages studies that differ in design, population, and delivery.
• Egger's regression test approached significance (t = 1.871, p = 0.066) with a right-skewed funnel plot, which the authors read as possible mild publication bias, though 8 of the 29 points fell outside the plot's slope lines.
• Educational level and instructional method were not statistically significant moderators at the between-group level (p = 0.067 and p = 0.232), so the K-12-versus-higher-education and pedagogy contrasts rest on subgroup patterns rather than validated differences.

Citation

Zhao, Y., Yue, Y., Sun, Z., Jiang, Q., & Li, G. (2025). Does generative artificial intelligence improve students' higher-order thinking? A meta-analysis based on 29 experiments and quasi-experiments. Journal of Intelligence, 13, 160.