Training Pedagogical LLMs for Tutoring

Domain-specialized optimization can transform a mid-sized Open Source model (Qwen3-32B) into a pedagogical domain expert that outperforms far larger proprietary systems — but only when training rewards guiding rather than answering.(Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised) Classical instructional design theory (ADDIE, Dick & Carey) combined with modern ReAct reasoning achieves the highest performance in automated instructional design.(ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents)

Questions to Consider

• General-purpose chatbots are optimized to give quick, correct answers. Why is that the opposite of what a tutor needs, and what does that 'incentive mismatch' suggest about off-the-shelf AI as a teaching tool?
• A benchmark found 97 models scored between 28% and 89% on pedagogical knowledge—meaning it's not automatically learned in pretraining. Does that surprise you, and what does it imply about trusting a general Large Language Models (LLMs) to teach?
• EduQwen's training explicitly rewards 'guiding' over 'answering.' Before reading the methods, can you think of how you'd tell an AI to prefer guiding—and how you'd measure whether it actually did?
• The page shows classical design theory (ADDIE) combined with flexible reasoning beat both pure theory and pure technique. Why might 'structure plus flexibility' outperform either alone when an AI designs instruction?
• Training pedagogy into a model costs time, data, and compute. For your context, what would convince you the investment is worth it versus just prompting a general-purpose model with 'act like a tutor'?

Introduction

General-purpose LLMs are optimized for helpfulness: users want quick, correct answers. Tutoring requires the opposite: the goal is not to provide the answer, but to help the student get to the answer themselves. This creates a fundamental incentive mismatch.

Approach 1: RL-SFT-RL Pipeline for Pedagogical Reasoning (EduQwen)

Singh et al. (2026) developed a three-stage pipeline transforming Qwen3-32B into EduQwen, achieving 96.52% on the CDPK Benchmark and surpassing Gemini-3 Pro (90.55%).

Stage 1: Initial RL (EduQwen 32B-RL1)

• Algorithm: DAPO (Decoupled Advantage Policy Optimization) with asymmetric clipping
• Reward model: Prioritizes guiding responses over direct answers
• Curriculum learning: Progressive difficulty; hard-negative mining excludes questions the base model already solves perfectly
• Extended rollouts: 5→8 steps to capture multi-step pedagogical decisions
• Result: 94.13% (already SOTA)

Stage 2: Synthetic SFT (EduQwen 32B-SFT)

• RL1 model generates 40,000 synthetic responses
• Gradient-based selection retains only hard examples
• Difficulty-weighted sampling: easy questions → one example; hard questions → all, weighted up
• Result: 96.20%

Stage 3: Final RL (EduQwen 32B-SFT-RL2)

• Second DAPO round, reusing the original hard-negative set
• Model now solves problems it originally found challenging
• Result: 96.52% (definitive SOTA)

The Pedagogy Benchmark: Evaluating Pedagogical Knowledge

Lelièvre et al. (2025) introduced The Pedagogy Benchmark, measuring Cross-Domain Pedagogical Knowledge (CDPK) and Special Education Needs and Disability (SEND) knowledge from real teacher professional development exams. Across 97 models, accuracy ranged from 28% to 89%—revealing that pedagogical knowledge is not automatically acquired in general pretraining.

EduQwen connection: Singh et al.’s EduQwen achieved 96.52% on CDPK, demonstrating that targeted RL+SFT optimization can close the pedagogical knowledge gap that Lelièvre et al. document. The benchmark serves as both a diagnostic (showing most models fail at pedagogy) and a training target (showing optimization works).

Live leaderboards track cost-accuracy Pareto frontiers: rebrand.ly/pedagogy

Approach 2: Theory-Grounded Instructional Design Agents (ISD-Agent-Bench)

Jeon et al. (2026) created a benchmark for LLM agents automating Instructional Systems Design (ISD), testing whether classical pedagogy theory improves agent performance.

Architecture	Performance	Why
Hybrid: theory + ReAct	Best	Classical ADDIE/Dick & Carey frameworks provide structure; ReAct enables flexible multi-step reasoning
Pure theory-based	Moderate	Structured but inflexible
Technique-only (pure ReAct)	Worst	Flexible but lacks pedagogical grounding

Key insight: Theoretical quality strongly correlates with benchmark performance. Theory-based agents excel in problem-centered design and objective-assessment alignment.

Benchmark Design

• 25,795 scenarios from Context Matrix (51 variables × 5 categories × 33 ISD sub-steps)
• Multi-judge protocol across diverse LLM providers to mitigate LLM-as-judge bias
• High inter-judge reliability achieved

Approach 3: Pedagogical Instruction Following (LearnLM) and Authentic-Data Post-Training (TeachLM)

Two complementary post-training strategies for embedding pedagogy into foundation models:

• Pedagogical instruction following (LearnLM). Google's LearnLM reframes education-model training as pedagogical instruction following: training and evaluation examples carry system-level instructions describing the desired pedagogical behavior, letting developers/teachers specify tutor behavior without committing to any single definition of pedagogy. Mixed directly into Gemini's post-training (SFT + reward-model + RLHF stages) via co-training, LearnLM was preferred by experts over GPT-4o (+31%), Claude 3.5 Sonnet (+11%), and base Gemini 1.5 Pro (+13%) across scenario-guided multi-turn evaluations. Key finding: RL is substantially more effective than SFT alone for following nuanced pedagogical instructions in long conversations.

• Authentic-data post-training (TeachLM). TeachLM argues that prompt engineering is a stopgap and that the scarce ingredient is authentic learner–tutor interaction data. Trained on 100,000 hours of one-on-one Polygence sessions (rigorously anonymized), it builds a fine-tuned authentic student model enabling synthetic multi-turn evaluation, and the teacher model doubles student talk time, improves questioning style, and increases dialogue turns by 50%.

Synthesis: LearnLM shows that instruction following + RLHF is a viable route when training data is scarce; TeachLM shows that when authentic longitudinal interaction data is available, post-training on it directly outperforms both prompting and synthetic-only data. Together they frame the pedagogical-training design space as a choice between scalable instruction-conditioned post-training and data-driven fine-tuning on real tutoring interactions.

Rubric-guided prompting as a lightweight alternative

Not all pedagogical shaping requires retraining. Yaşar et al. (2026) showed that rubric-guided prompting — treating the rubric as a semantic interface between human pedagogical intent and machine inference — can push a general-purpose LLM toward human-like evaluative judgment without fine-tuning: iterative rubric co-refinement raised LLM–human agreement on student design work from 54.75% to 81.25% (Cronbach's Alpha 0.393 → 0.798), and role-aware prompting (instructor, peer-reviewer, grant-reviewer) produced distinct evaluative feedback. This complements the training-based approaches above: where TeachLM argues prompt engineering is a stopgap and authentic-data post-training is the scarce ingredient, Yaşar et al. demonstrate that a well-engineered rubric can itself be a powerful, low-cost lever for aligning LLM evaluation with pedagogical intent — though human-in-the-loop oversight remains essential, as models can still misinterpret nuance, hallucinate rationale, or blend roles. A further lightweight alternative is prompt-level role-play customization without retraining: Zhuang and Zhang (2025) used the OpenAI custom-GPT feature to simulate a misconception-holding middle-school math student, and found that a refined, literature-grounded prompt (specifying three ratio-reasoning Misconceptions about AI) elicited the target conceptual errors far more reliably than a broad algebra prompt (0.98 vs. 0.40 presence) — evidence that careful prompt design can substantially steer an off-the-shelf model toward a desired pedagogical persona, even while the simulated agent retained authenticity limitations (teacher-like tone, role confusion).

The lightest intervention in this family is not prompting but parameter-efficient adaptation. Lu et al. (2026) built 360 system–user–assistant dialogues from a Linear Control Systems course, restructured answers into a Solution–Method–Teaching-Points format, and applied LoRA to Qwen2.5-3B and 7B at ranks 4, 8 and 16. Structured-output coverage moved from near zero at base to roughly 1.00, and the best configuration (7B, r = 16) reached ROUGE-L 0.4093 with bootstrap confidence intervals for the gain entirely above zero — but gain per million adapter parameters fell monotonically as rank rose, so course-level alignment is a scale-and-rank trade-off rather than a free upgrade. The metrics measure similarity and formatting, not derivational accuracy.

Approach 4: Training Simulator Roles, Not Only Tutors

The same post-training machinery is now pointed at the learner side of the interaction, and the results say the supervision budget matters more than the prompt. Liu et al. (2026) instruction-tuned three small models to acquire algebra misconceptions in two roles — a Novice Student Misconception Model holding one misconception, and an Expert Tutor Misconception Model holding ten — and measured both misconception accuracy and correct-solving accuracy. The student role showed a trade-off no prompt could fix: the learned error overgeneralised beyond its applicable problem types until correct examples were explicitly mixed into the training data, at ratios as low as one correct example per four misconception examples. The tutor role showed no such cost, with correct accuracy stable or rising from 93% to 98% when ten misconceptions were trained jointly, though classroom-scale samples were insufficient and rare misconceptions would require cross-institution data. Most decisively, neither role acquired anything when trained on final answers alone — misconception accuracy stayed below 30% at every data size — so step-level solution traces, not more examples, are the binding requirement. SWIM (Do, Kontak and Sachan, 2026) reaches the mirror conclusion for a writing simulator: rubric-grounded prompting gave limited proficiency control (best average trait QWK 0.577 for Claude Sonnet, 0.422 for GPT-5.4, near zero for prompting an open 7B model), supervised fine-tuning lifted a 7B model to 0.474 ± 0.023, and GRPO against an automated-essay-scoring-derived reward lifted it further to 0.618 ± 0.005 across every trait and prompt, with the reward designed as a dense trait-normalized accuracy because exact-match rewards are too sparse in the multi-trait setting.

Synthesis: What Makes Pedagogical Training Work

Principle	EduQwen	ISD-Agent-Bench
Reward/guide, don't answer	DAPO reward model penalizes direct solutions	Theory-enforced ISD steps require alignment between objectives and assessment
Curriculum by difficulty	Hard-negative mining + progressive rollouts	Context Matrix systematically varies complexity
Multi-step reasoning	Extended rollouts (5→8 steps)	ReAct-style reasoning chains
Validate with theory	CDPK benchmark measures pedagogical knowledge	ADDIE/Dick & Carey frameworks ground design decisions
Iterative refinement	RL → SFT → RL pipeline	Multi-judge evaluation reduces bias

Relationship to Safety and Design

Training for pedagogy is not just about accuracy — it is a safety intervention:

• A model that rewards "guiding" over "answering" is less likely to commit answer over-disclosure harms
• Theory-grounded agents (ISD-Agent-Bench) align with pedagogical principles that prevent metacognitive suppression
• However, training on pedagogical benchmarks does not guarantee multi-turn safety; SafeTutors shows even specialized models degrade over sustained dialogue
• Grounding and validation can substitute for — or complement — training. Reddig, Arora & MacLellan (2025) found that a frontier untrained GPT-4 produced ~35% too-general, incorrect, or answer-revealing hints when authoring ITS feedback, and that its own automated quality checks misaligned with human judgment — leading the authors to conclude that LLMs lack an internal model of instruction and that robust validation or domain-specific training is required before unsupervised learner-facing use, supporting the case that grounding and quality control are themselves pedagogical interventions alongside reward design.

Sycophancy reduction as a training objective

Because tutoring requires corrective friction — challenging a student's incorrect claim rather than affirming it — reducing sycophancy is a core objective for pedagogical LLM training. EduFrameTrap shows that models which resist context-switch attacks still capitulate under authority or social-affective pressure, withholding corrective feedback; its authors argue "kind-but-correct" behavior should be an explicit training requirement, not a usability preference. Training that rewards guiding over answering (as in EduQwen's DAPO reward model) is one structural lever against sycophantic answer-giving. Yet contextual sycophancy persists even after prompting/alignment training — learners' errors still propagate into AI advice — so sycophancy mitigation in trained tutors must combine reward design, alignment against sycophancy benchmarks, and system-level safeguards rather than rely on any single stage.

Open Questions

1. Does pedagogical RL training generalize across subjects, or is subject-specific tuning (as SafeTutors suggests) always needed?
2. Can the RL-SFT-RL pipeline be combined with longitudinal memory (see PersonaVLM: Long-Term Personalized Multimodal LLMs) for personalized tutoring?
3. Would ISD-agent theory improve general tutoring conversation, or is it limited to macro-level curriculum design?

Connected Concepts

• Intelligent Tutoring
• Scaffolding
• Adaptive Learning
• Metacognition
• Affective Tutoring
• Human-in-the-Loop
• Personalized Learning
• Learner Modeling and Adaptive Instruction
• Self-Regulated Learning
• Pedagogical Safety
• Formative Assessment
• Large Language Models (LLMs)
• Authentic Assessment
• AI Sycophancy
• AI Feedback Quality
• Bias Mitigation
• Technologies — Umbrella: AI technologies and techniques (models, LLM training, robotics, RAG, agentic)

Connected Articles

• Comprehensive Review of Intelligent Tutoring Systems
• Civic education in the age of AI: Should we trust AI-generated lesson plans
• Cognitive Agent Compilation for Explicit Problem Solver Modeling
• The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human-AI Collaboration
• Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact
• EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
• MathBuddy: Affective Math Tutoring
• A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large Language Models: An Exploratory Study of Educational Potential
• Multimodal Learning with Generative AI
• Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling
• NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models
• Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised
• Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks — Sycophancy is an educational safety risk: Why LLM tutors need sycophancy benchmarks
• TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring
• LearnLM: Improving Gemini for Learning — LearnLM: Improving Gemini for Learning
• TeachLM: Post-Training LLMs for Education Using Authentic Learning Data — TeachLM: Post-Training LLMs for Education Using Authentic Learning Data
• From evaluation to emulation: LLMs as agents of iterative pedagogical design — LLMs as agents of iterative pedagogical design
• Generating In-Context, Personalized Feedback for Intelligent Tutors with Large Language Models
• Integrating ChatGPT in Mathematics Teacher Education: AI-Based Simulation Role-Playing to Support Practice-based Teaching
• LoRA Fine-Tuned Models for Control Systems Course Q&A: A Multidimensional Evaluation of Model Scale and Rank Effects — LoRA Fine-Tuned Models for Control Systems Course Q&A: A Multidimensional Evaluation of Model Scale and Rank Effects
• Misconception Acquisition Dynamics in Large Language Models — data composition, correct-example mixing and step-level supervision for misconception-aware models
• SWIM: Student Writing Simulation via Proficiency-Conditioned Generation — supervised and reward-based training beat rubric prompting for proficiency control
• OmniEdu: Open Foundation Models for Learning and Teaching — OmniEdu: Open Foundation Models for Learning and Teaching