Training Pedagogical LLMs for Tutoring

Domain-specialized optimization can transform a mid-sized open-source model (Qwen3-32B) into a pedagogical domain expert that outperforms far larger proprietary systems — but only when training rewards guiding rather than answering.^Singh Eduqwen Pedagogical RL 2026 Classical instructional design theory (ADDIE, Dick & Carey) combined with modern ReAct reasoning achieves the highest performance in automated instructional design.^Jeon Isd Agent Bench 2026

General-purpose LLMs are optimized for helpfulness: users want quick, correct answers. Tutoring requires the opposite: the goal is not to provide the answer, but to help the student get to the answer themselves. This creates a fundamental incentive mismatch.

Approach 1: RL-SFT-RL Pipeline for Pedagogical Reasoning (EduQwen)

Singh et al. (2026) developed a three-stage pipeline transforming Qwen3-32B into EduQwen, achieving 96.52% on the CDPK Benchmark and surpassing Gemini-3 Pro (90.55%).

Stage 1: Initial RL (EduQwen 32B-RL1)

Algorithm: DAPO (Decoupled Advantage Policy Optimization) with asymmetric clipping

Reward model: Prioritizes guiding responses over direct answers

Curriculum learning: Progressive difficulty; hard-negative mining excludes questions the base model already solves perfectly

Extended rollouts: 5→8 steps to capture multi-step pedagogical decisions

Result: 94.13% (already SOTA)

Stage 2: Synthetic SFT (EduQwen 32B-SFT)

RL1 model generates 40,000 synthetic responses

Gradient-based selection retains only hard examples

Difficulty-weighted sampling: easy questions → one example; hard questions → all, weighted up

Result: 96.20%

Stage 3: Final RL (EduQwen 32B-SFT-RL2)

Second DAPO round, reusing the original hard-negative set

Model now solves problems it originally found challenging

Result: 96.52% (definitive SOTA)

The Pedagogy Benchmark: Evaluating Pedagogical Knowledge

Lelièvre et al. (2025) introduced The Pedagogy Benchmark, measuring Cross-Domain Pedagogical Knowledge (CDPK) and Special Education Needs and Disability (SEND) knowledge from real teacher professional development exams. Across 97 models, accuracy ranged from 28% to 89%—revealing that pedagogical knowledge is not automatically acquired in general pretraining.

EduQwen connection: Singh et al.’s EduQwen achieved 96.52% on CDPK, demonstrating that targeted RL+SFT optimization can close the pedagogical knowledge gap that Lelièvre et al. document. The benchmark serves as both a diagnostic (showing most models fail at pedagogy) and a training target (showing optimization works).

Live leaderboards track cost-accuracy Pareto frontiers: rebrand.ly/pedagogy

Approach 2: Theory-Grounded Instructional Design Agents (ISD-Agent-Bench)

Jeon et al. (2026) created a benchmark for LLM agents automating Instructional Systems Design (ISD), testing whether classical pedagogy theory improves agent performance.

Architecture	Performance	Why
Hybrid: theory + ReAct	Best	Classical ADDIE/Dick & Carey frameworks provide structure; ReAct enables flexible multi-step reasoning
Pure theory-based	Moderate	Structured but inflexible
Technique-only (pure ReAct)	Worst	Flexible but lacks pedagogical grounding

Key insight: Theoretical quality strongly correlates with benchmark performance. Theory-based agents excel in problem-centered design and objective-assessment alignment.

Benchmark Design

25,795 scenarios from Context Matrix (51 variables × 5 categories × 33 ISD sub-steps)

Multi-judge protocol across diverse LLM providers to mitigate LLM-as-judge bias

High inter-judge reliability achieved

Synthesis: What Makes Pedagogical Training Work

Principle	EduQwen	ISD-Agent-Bench
Reward/guide, don't answer	DAPO reward model penalizes direct solutions	Theory-enforced ISD steps require alignment between objectives and assessment
Curriculum by difficulty	Hard-negative mining + progressive rollouts	Context Matrix systematically varies complexity
Multi-step reasoning	Extended rollouts (5→8 steps)	ReAct-style reasoning chains
Validate with theory	CDPK benchmark measures pedagogical knowledge	ADDIE/Dick & Carey frameworks ground design decisions
Iterative refinement	RL → SFT → RL pipeline	Multi-judge evaluation reduces bias

Relationship to Safety and Design

Training for pedagogy is not just about accuracy — it is a safety intervention:

A model that rewards "guiding" over "answering" is less likely to commit answer over-disclosure harms

Theory-grounded agents (ISD-Agent-Bench) align with pedagogical principles that prevent metacognitive suppression

However, training on pedagogical benchmarks does not guarantee multi-turn safety; SafeTutors shows even specialized models degrade over sustained dialogue