Human-in-the-Loop AI for Education

Educational AI systems that strategically interleave automated generation with human expert judgment, preserving pedagogical quality while scaling production. Two recent implementations illustrate distinct architectures:

CODE-GEN: Human-in-the-Loop MCQ Generation

Duan et al. (2026) built a RAG-based agentic system with two agents:

Generator Agent — Produces multiple-choice coding questions aligned with course learning objectives

Validator Agent — Assesses quality across seven pedagogical dimensions

Evaluation: 6 SMEs judged 288 AI-generated questions. Human-validated success rates: 79.9%–98.6% across dimensions.

AI-Strong Dimensions (low human burden):

Question clarity, code validity, concept alignment, correct-answer validity

Human-Required Dimensions (high human burden):

Pedagogically meaningful distractor design

High-quality explanatory feedback

Strategic insight: Human effort should be concentrated where instructional judgment is irreplaceable; computational verification can be fully automated.

MAIC: Human-in-the-Loop Script Generation

Yu et al. (2024) deployed a multi-agent classroom (Teacher Agent, TA Agent, classmate archetypes) at Tsinghua University with >500 students and >100,000 learning records. Human instructors participate in script generation and oversight, ensuring that mass-scale AI augmentation does not displace pedagogical expertise.

Synthesis

Human-in-the-loop design is not merely a safety measure—it is a resource-allitution strategy. The frontier question is not whether to include humans, but where in the pipeline their judgment has highest marginal value.