AI Literacy Interventions in Education: A Meta-Analysis of Effects and Moderators

Synthesis: In brief: Liu, Lin, Ng, Yang, and Dai conduct a three-level meta-analysis of 59 empirical studies (172 effect sizes, 7,211 participants) on AI literacy interventions. They find a large, statistically significant overall effect (g = 0.837, p < .001) with substantial heterogeneity; moderator analyses show interventions in East Asia and Europe outperformed those in North America, and knowledge-focused interventions outperformed those targeting skills, attitudes, or Ethics. They conclude AI literacy education should move beyond knowledge toward skills, practices, Ethics, and attitudes, supported by integrated and reflective pedagogies and GenAI-supported tools.

This meta-analysis answers the question of how — and under what conditions — AI literacy interventions support student development. Synthesizing 172 effect sizes from 59 studies and 7,211 participants, it estimates a large overall effect (g = 0.837) but with a wide 95% prediction interval [−0.292,1.966], indicating effectiveness varies considerably across settings. Moderator analyses identify geographical region and learning-outcome focus as significant: interventions in East Asia and Europe yielded stronger effects than North America, and knowledge-focused interventions outperformed those targeting skills, attitudes, or ethics.

The paper's central argument is that AI literacy education should move beyond knowledge and conceptual understanding toward skills, practices, Ethics, and attitudes. This shift is best supported, the authors suggest, by integrated pedagogies that combine conceptual learning, hands-on making, reflection, and GenAI-supported tools — including constructionist and socio-constructivist approaches such as project- and Problem-Based Learning, inquiry-based learning, and Experiential Learning environments. The geographic variation highlights the need for culturally relevant, context-sensitive interventions that respond to local educational priorities, technological conditions, and learner needs.

Study Design and Analysis

• PRISMA-guided search of Web of Science, Scopus, ERIC, and PsycINFO retrieves 7,330 candidate records
• Hedges' g effect sizes were computed for posttest and pretest-posttest designs; eight outlying values were excluded, leaving 59 studies, 172 effect sizes, and 7,211 participants.
• A three-level random-effects model partitioned variance, with about 71% sitting between studies; coding agreement was strong (Cohen's κ ≈ 0.87) and publication-bias checks (fail-safe N over 190 thousand) were reassuring.

Key Findings

• 59 studies, 172 effect sizes, 7,211 participants synthesized via a three-level meta-analysis.
• Large overall effect: g = 0.837, p < .001, with a wide prediction interval [−0.292,1.966] indicating considerable variation.
• Geography matters: East Asia (g ≈ .78) and Europe (≈ .96) outperformed North America (≈ .66), likely reflecting an access-and-engagement-over-mastery focus there.
• Knowledge-focused interventions outperformed others: knowledge gains (g ≈ .97) exceeded skills (≈ .67), attitudes (≈ .68), and ethics (≈ .64), all significant — durable dispositions lag understanding.
• (Nonsignificant) descriptives: mixed and reflective pedagogies, generative-AI tools, and performance tasks each showed numerically larger effects, tentative signals worth testing.
• Conclusion: AI literacy education should move beyond knowledge toward skills, practices, ethics, and attitudes, via integrated and reflective pedagogies.

What this means for practice

• Instructors. Pair every concept lesson with sustained practice: AI knowledge gains (g = 0.965) far outpaced skills (g = 0.672), attitudes (g = 0.676), and ethics (g = 0.635), so build in problem-solving, critical output evaluation, and ethics discussion rather than single-shot content delivery.
• Instructors. Assess with performance tasks where feasible — they captured the descriptively larger effect (g = 0.901) than self-reported data (g = 0.774), which are more prone to overstating learning.
• Faculty developers. Fit programs to local conditions instead of importing a standard model: Europe (g = 0.960) and East Asia (g = 0.782) outperformed North America (g = 0.658), indicating that culturally relevant design — curriculum, access, and policy context — shapes what works.
• Researchers. Pilot before scaling: with a 95% prediction interval from −0.292 to 1.966, no AI Literacy intervention is guaranteed to transfer to a given classroom.

Limitations

• The synthesis rests on 59 studies, 172 effect sizes, and 7,211 participants, with 71.361% of the variance sitting between studies, so g = 0.837 is an average across heterogeneous studies rather than a common effect.
• Several moderator cells were tiny — Southeast Asia (k = 4), quantified qualitative instruments (k = 3), mixed educational levels (k = 9) — and the authors caution that subgroups with fewer than 10 effect sizes are statistically unreliable.
• Study quality was not incorporated into the weighting, even though the pool combined (quasi-)experimental designs of varying internal validity.
• Residual heterogeneity was left unmodeled: intervention duration, study design, and disciplinary background were not tested as moderators, and eight effect sizes were dropped as outliers.

Citation

Liu, S., Lin, Z., Ng, O. L., Yang, Y., & Dai, Y. (2026). AI literacy interventions in education: A meta-analysis of effects and moderators. Educational Research Review, 52,100813.