Scaffolding Systematic Reviews in Learning Design and Technology Through Mentoring and AI Integration

Synthesis: Wang et al. (2026) treat systematic reviews (SRs) in Learning Design and Technology (LDT) as intentional learning experiences rather than mere methodological procedures. Drawing on their interdisciplinary team of novice and experienced researchers, they reflect on the realities of each SR stage, showing how scaffolding through mentoring, peer collaboration, and judicious AI integration helped manage ambiguity, foster team consistency, and sustain methodological rigor. Automation tools (chiefly screening) reduced procedural burdens, but interpretive decisions required substantial human oversight — a clear instance of Human In The Loop AI.

Core Finding

Conducting a systematic review is an inherently iterative, human process that is best learned through practice, supported by mentoring and collaboration. Methodological rigor rests on transparent, predefined, reproducible procedures, yet the practical challenges of interdisciplinary literature — variable terminology, reporting conventions, and study formats — demand collective judgment. Automation advanced most in abstract screening, but data extraction, synthesis, and reporting still depended heavily on human expertise. The authors found that hands-on engagement, mentoring support, and peer discussion were central to sustaining team cohesion and methodological rigor, and that automation should be treated as a complement rather than a substitute for human judgment.

The challenge of interdisciplinary synthesis in LDT

Because LDT spans education, instructional design, psychology, information science, computer science, media studies, and more, its literature varies widely in terminology, theory, and reporting conventions. This complicates locating, integrating, and interpreting studies. The authors frame SRs as a way to reduce conceptual ambiguity and build a shared language across disciplines — an inherently collaborative undertaking that benefits from developmental support for novice researchers.

Navigating ambiguity at each SR stage

• Planning: Choosing a viable topic requires prior familiarity with the literature and assessment of field maturity; developing shared inclusion/exclusion criteria demands iterative discussion, calibration exercises, and concrete decision rules for gray-area cases.
• Execution: Abstract and full-text screening face high volume, sparse and inconsistent abstracts, and diverse document formats. Strategies included conservative screening with "maybe" categories, keyword-exclusion filtering, training in academic article structure and reading strategies, and pairing less experienced researchers with more experienced members during double screening.
• Data extraction: Variable reporting locations and conventions reduced interrater reliability. Structured, multiple-choice coding items, practice rounds, double coding with third-reviewer adjudication, and shared meeting logs supported consistency.
• Reporting: Open-ended and mixed-format data were hard to synthesize; close collaboration between synthesis methodologists and content experts, with topic-modeling triangulation, balanced interpretation with efficiency.

Mentoring as emotional and methodological scaffolding

Mentoring functioned as an integrated support system combining methodological guidance with emotional scaffolding. It normalized uncertainty, created spaces for co-reflection with experienced researchers, and made expert reasoning visible through discussion — supporting both technical competence and confidence. When one-on-one mentoring was not feasible, peer-based discussion groups offered mutual support. This is a strong example of Scaffolding applied to a professional research process, and of human–AI collaboration where people remain the interpretive center.

Balancing automation with human judgment

While ML and LLM tools (e.g., ASReview, SWIFT-Review, Covidence, AIScreenR, MetaMate) reduced procedural burdens such as abstract screening, tasks requiring contextual interpretation, reconciliation of ambiguous reporting, or integrative synthesis remained hard to automate. The authors used automation selectively and verified its outputs, concluding it is best treated as a complement rather than a substitute — a clear Human In The Loop AI position consistent with viewing Generative AI as a supportive, scaffolding tool that augments rather than replaces human thinking.

Relevance to the wiki

This paper significantly contributes to the wiki's Scaffolding and Human In The Loop AI threads by treating a research workflow as an intentional learning experience. It provides practice-informed guidance for novice researchers and interdisciplinary teams, illustrates how Collaborative Learning and mentoring sustain rigor, and models a principled division of labor between automation (procedural burden) and human judgment (interpretive decisions) — of direct relevance to Instructional Design and AI-assisted research practice.

Connected Concepts

• Scaffolding
• Human In The Loop AI
• Human AI Collaboration
• Collaborative Learning
• Instructional Design
• Faculty Development
• Generative AI
• Learning Analytics

Connected Articles

• Wang Safety Gap Productive Struggle 2026 — The Safety Gap: Restoring Productive Struggle Through Generative AI
• Lukesova Clue Before Correction 2026 — Clue Before Correction: ChatGPT for Autonomous Learning

Citation

Wang, X., Dadashipour, F., Basori, Maeda, Y., & Richardson, J. C. (2026). Scaffolding systematic reviews in learning design and technology through mentoring and AI integration. Educational Technology Research and Development.