Can EdTech Close Learning Gaps? Global Evidence from Digital Interventions

Synthesis: In brief: A World Bank systematic review and meta-analysis pools 191 effect sizes from 14 randomized trials across ten economies to estimate that adaptive and AI-enabled educational technology raises student learning by an average of 0.125 standard deviations relative to traditional instruction — above the median effect for education RCTs and within the range Kraft (2020) calls "large" for field experiments. Crucially, the newer generative-AI tools show no advantage over the adaptive software that preceded them, and gains are driven less by which technology is used than by whether it is embedded in a sound instructional strategy.

The review brings two "generations" of adaptive educational technology into a common framework under common inclusion criteria and on a common effect-size scale: first-generation tools that select from content authored in advance (adaptive computer-assisted learning, intelligent tutoring systems such as Mindspark), and second-generation generative AI tools that generate instructional content at the point of use (Rori in Ghana, GPT-based tutors in Türkiye). Using robust variance estimation (RVE) meta-regression, the authors retain every extracted outcome per study rather than one estimate each — a methodological choice that matters because within-study variation across outcome measures is as wide as variation across studies.

The pooled learning effect of 0.125 sd holds at primary and secondary levels, in high- and middle-income countries, across tutoring systems, computer-assisted learning platforms, and teacher-facing tools, and across both technological generations. The estimated differential for generative over first-generation tools is only 0.022 sd (SE 0.075), an interval wide enough to bound rather than resolve the comparison: the experimental record to date shows no advantage for the newer technology. Gains on socio-emotional outcomes are positive but roughly a quarter the size (0.029 sd, from only five papers), and the review cannot reject the null for teaching practices (three studies). Among AI-powered tutoring interventions specifically, the average effect is 0.12 sd — below the 0.288 sd Nickow et al. (2024) report for human tutoring, but at a small fraction of its cost.

Two features of the evidence base sharply limit what these estimates can justify. The sample is narrow: no included study was conducted in a low-income country, researchers were involved in implementing 16 of 19 interventions, and none was implemented by a government alone — so behavior under public delivery at scale is essentially unobserved. And only two or three studies report per-student costs on a comparable basis, so the field's motivating claim that personalization can be delivered at a fraction of the cost of human tutoring has almost never been measured alongside the effects it is meant to justify.

Key Findings

• Adaptive and AI-enabled EdTech raises learning by ~0.125 sd on average across 191 effect sizes from 14 RCTs in ten economies — above the median (0.10 sd) for education interventions evaluated by randomized trial, and within the range Kraft (2020) reads as large for broad-achievement field experiments.
• Generative AI shows no advantage over earlier adaptive software. The differential for second-generation tools is 0.022 sd (SE 0.075), an interval [−0.15, 0.19] wide enough to bound the comparison rather than resolve it; the experimental record to date shows no advantage for the newer technology.
• Effects are consistent across education levels and income contexts — primary (0.137 sd) and secondary (0.135 sd) are nearly identical, and high- vs. middle-income differences are not statistically significant.
• AI-powered tutoring averages 0.12 sd, below the 0.288 sd pooled effect for human tutoring (Nickow et al., 2024) but at a small fraction of its cost; the strongest results come from tutoring embedded in a broader instructional strategy with clear objectives, curriculum alignment, and safeguards against misuse.
• Gains do not extend equally to other outcomes: socio-emotional/behavioral effects are ~0.029 sd (a quarter of the learning effect, five papers), and teaching-practice effects are indistinguishable from zero (three studies).
• The evidence base is narrow and policy-relevant gaps remain: no low-income-country study, researcher-heavy implementation, no government-alone implementation, and cost data in only ~2–3 of 14 studies.
• Dosage does not order the estimates — cumulative exposure varies over two orders of magnitude without mapping onto the ranking of effects; within-study measure choice shifts an estimate as much as the study chosen (one ITS's four estimates spanned 0.60 sd).

What this means for practice

• Administrators. Choose the instructional design before the product: the review attributes gains less to which technology is used than to whether it sits inside a sound instructional strategy with clear objectives and curriculum alignment, and dosage alone does not order the estimates.
• Policymakers. Do not procure on the promise of novelty — the differential for second-generation generative AI over first-generation adaptive software is 0.022 sd (SE 0.075), an interval wide enough only to bound the comparison — and fund the model that fits the local delivery capacity.
• Policymakers. Require per-student cost reporting alongside effect estimates in every publicly funded trial: comparable cost data appear in only about two or three of the 14 included studies, so the claim that personalization is cheap at scale is largely unmeasured.
• Administrators. Attach misuse safeguards to any AI tutoring rollout and evaluate them on their own terms: in the Türkiye study Guardrails removed the harm without improving scores, and socio-emotional gains averaged only 0.029 sd.
• Researchers. Study public delivery directly, since 16 of 19 interventions were researcher-implemented and none by a government alone, and recruit sites in low-income countries, where no included trial took place.

Limitations

• The pooled estimate rests on 14 randomized studies (191 effect sizes); several subgroup cells rest on a single cluster, the moderators are correlated with one another, and only five papers inform the socio-emotional estimate and three the teaching-practice estimate, so the heterogeneity results are descriptive rather than tests.
• No included study was conducted in a low-income country, and researchers were involved in implementing 16 of the 19 interventions reviewed, so how these tools behave under government delivery at scale is essentially unobserved.
• Only about two or three studies report per-student costs on a comparable basis, leaving the cost-effectiveness claim that motivates the literature almost never measured alongside the effects it is meant to justify.
• The generative-AI studies are very recent, so their published effects may reflect early adopters, novelty, and models that have since been superseded, giving those estimates a shorter shelf life than the first-generation results against which they are compared.

Citation

Burneo, A., Dinarte-Diaz, L., Lopez, C., & Molina, E. (2026). Can EdTech close learning gaps? Global evidence from digital interventions. World Bank Policy Research Working Paper.