Comprehensive Review of Intelligent Tutoring Systems — Journal of Computers in Education (2025). A systematic literature review covering 2010–2025 that analyzes the deployment and effectiveness of Intelligent Tutoring Systems (ITS) in real educational settings. The review examines the full landscape of ITS research — pedagogical strategies, natural language processing, adaptive learning mechanisms, student modeling approaches, and domain-specific applications — and arrives at a nuanced verdict: the evidence for ITS effectiveness is mixed, revealing a complex landscape of genuine advancements alongside persistent challenges in scientific rigor and real-world impact.
Authors: Zerkouk et al. · Venue: Journal of Computers in Education (2025) · arXiv: 2507.18882
Key Findings
This review provides the most comprehensive mapping of the intelligent-tutoring-systems field since the emergence of LLM-based tutoring approaches transformed the landscape. Spanning a 15-year window (2010–2025), it captures both the pre-LLM era of traditional ITS and the post-LLM era that has fundamentally reshaped what is technically possible.
Mixed effectiveness evidence. The review's central finding is that ITS effectiveness is neither uniformly positive nor categorically negative. Some systems demonstrate substantial learning-gains, particularly in well-structured domains like mathematics and programming where student-modeling and knowledge-tracing techniques are most mature. Other deployments show negligible or context-dependent effects. This mixed picture challenges both the optimistic narrative that AI tutoring is a proven solution and the pessimistic narrative that it is ineffective. Instead, it calls for more nuanced questions: which systems, for which learners, in which contexts, produce which outcomes? This aligns with the ai-tutor-effectiveness-review literature's emphasis on contextual factors.
Pedagogical strategies. The review catalogs the range of pedagogical approaches embedded in ITS, from socratic-questioning and scaffolding to mastery-learning and adaptive-learning pathways. A key finding is that many ITS implementations lack explicit pedagogical grounding — the tutoring behavior is often driven by technical capabilities (what the system can do) rather than pedagogical principles (what the system should do). This echoes concerns in the pedagogical-llm-training literature about the gap between technical sophistication and pedagogical intentionality.
NLP and adaptive mechanisms. The integration of educational-nlp techniques — including automated-question-generation, short-answer assessment, and dialogue management — has advanced substantially over the review period. However, the review notes that many NLP components are evaluated in isolation rather than as integrated parts of tutoring systems that actually interact with learners. Similarly, adaptive-learning-systems show promise but often rely on narrow student models that fail to capture the full complexity of learner cognition and affect — a gap that the affective-tutoring and multimodal-ai-tutoring communities are beginning to address.
Student modeling challenges. student-modeling remains both the foundation and the bottleneck for ITS. While knowledge-tracing techniques (including Bayesian approaches like stanbkt-bayesian-knowledge-tracing and deep learning variants) have improved, the review identifies persistent gaps in modeling higher-order cognitive processes, metacognition, and motivational states. This connects to the engagement-intensity-learner-modeling and metacognition literatures.
Scientific rigor deficit. One of the review's most important contributions is its methodological critique. Many ITS studies suffer from weak experimental designs — small sample sizes, absence of control groups, short intervention durations, and inadequate statistical analyses. The authors call for greater scientific rigor, including randomized-controlled-trials-education where feasible, pre-registration of study designs, and transparent reporting aligned with educational research standards. This methodological critique connects to broader concerns in ai-ed-evaluation about the quality of evidence in AI education research.
Implications
For researchers, this review provides both a taxonomy of the field and a methodological call to action. The mixed effectiveness findings underscore the need for rct-quality studies that can disentangle the effects of specific ITS features from confounds like novelty effects, instructor quality, and student self-selection. The review's emphasis on real educational settings — as opposed to laboratory studies — pushes the field toward ecologically valid research designs that mirror the ai-ed-implementation challenges faced by practitioners.
For system designers, the review's catalog of pedagogical gaps suggests concrete priorities: ITS should be built on explicit pedagogical frameworks rather than retrofitted onto technical architectures; domain-modeling should incorporate richer representations of learner cognition and affect; and learning-analytics should be embedded from the start to enable continuous improvement and evidence generation.
The review's identification of persistent student modeling limitations connects directly to emerging work on llm-student-modeling-memory and history-aware-student-simulation, which offer potential pathways toward more sophisticated learner representations. Similarly, the call for adaptive-learning that goes beyond simple difficulty adjustment toward truly personalized pedagogical experiences connects to the learnmate2-llm-adaptive-learning and nie-personavlm-long-term-personalization-2026 research streams.
For the edtech-platform industry and educational-policy-ai community, this review serves as a sobering reminder that technical capability does not guarantee educational effectiveness. The mixed evidence base should temper claims of AI tutoring as a silver bullet while also validating continued investment in rigorous ITS research and development. The path forward requires ai-ed-evaluation frameworks that match the sophistication of the systems being evaluated.
Related Pages
- intelligent-tutoring-systems — Core concept: the ITS field
- adaptive-learning — Adaptive learning systems and approaches
- student-modeling — Student modeling in educational technology
- knowledge-tracing — Knowledge tracing techniques
- educational-nlp — NLP applications in education
- ai-tutor-effectiveness-review — Reviews of AI tutor effectiveness
- ai-ed-evaluation — Evaluation frameworks for AI in education
- learning-analytics — Learning analytics and data-driven improvement
- domain-modeling — Domain modeling for educational systems
- adaptive-learning-systems — Adaptive learning system architectures
- randomized-controlled-trials-education — RCT methodology in education
- edtech-platform — Educational technology platforms