A meta-analysis of the effect of generative AI on productivity and learning in programming

Core Contribution

Maier, Gunzenhäuser & Schweisthal (2026) conduct a meta-analysis synthesizing evidence on how generative AI tools affect both programming productivity and learning outcomes. This is a confidence: high paper due to its synthesis design across multiple studies, addressing the central tension between short-term efficiency gains and long-term skill development.

Key Findings

The meta-analysis examines the productivity-learning trade-off that sits at the heart of ai-learning-transfer debates: when AI tools boost immediate coding output, do they simultaneously undermine the development of foundational programming skills? This directly connects to the broader question of whether tutoring-specific-vs-general-ai matters — general AI coding assistants may optimize for productivity at the expense of learning.

Significance for AIED

This paper provides the highest-level evidence synthesis currently available on the impact of generative AI in programming education. It bridges RCT-level findings with practical implications for:

Curriculum design: When should AI tools be introduced in programming courses?

Assessment: How do we validly assess programming skill when AI is available?

Policy: What guidance should regulation frameworks provide for AI in CS education?

The meta-analytic approach provides more robust conclusions than individual studies like collaborative-ai-tutoring (ProPACT) or agentic-education-coding work on teaching Claude Code. It also connects to programming-its research on traditional intelligent tutoring for coding, providing a baseline for comparing AI-augmented approaches.

The findings have implications beyond programming — the productivity-learning tension applies to writing-education, language-learning, and any domain where AI can produce acceptable output without the learner developing underlying skills (the llm-fallacy-misattribution problem).