LLM Student Simulation and Misconception Faithfulness

Synthesis

This paper exposes a critical failure mode in using LLMs as simulated students for Intelligent Tutoring development and evaluation. The authors introduce misconception faithfulness — the property that a simulated student holds a coherent, misconception-driven belief state and updates it only when feedback addresses the underlying misconception — and show that across seven LLMs (4B to 120B parameters), simulators exhibit near-zero faithfulness.

The core finding is a sycophantic failure mode: when given any corrective signal, LLM simulators abandon their assigned misconception persona and re-solve the problem from internal knowledge. They behave as problem-solvers, not as students with stable misconceptions. Using the novel Selective Flip Score (SFS), the authors quantify this: simulators flip their answers at similarly high rates regardless of whether feedback is targeted, misaligned, or generic.

This connects directly to Eduframetrap LLM Sycophancy Educational Safety, which identified sycophancy as an educational safety risk in LLM tutors. Here the sycophancy is inverted: simulated students capitulate to feedback rather than maintaining authentic misconception-driven behavior. Both papers together establish sycophancy as a bidirectional problem in AIED — affecting both tutor and student roles.

The post-training pipeline — combining supervised fine-tuning, preference optimization, and RL with SFS-aligned rewards — achieved SFS gains up to +0.56, demonstrating that misconception faithfulness is trainable. This has implications for AI Tutor Safety Harms: if student simulators used for tutor safety testing are themselves unfaithful, safety evaluations conducted on them may systematically miss harm patterns that real students would exhibit.

For Student Experience and benchmark development, this paper motivates a paradigm shift from static output matching toward interactive, belief-aware student modeling — a theme that also resonates with LLM Student Modeling Memory and the behavioral evaluation framework in AI Tutor Behavioral Evaluation.

Connected Concepts

Connected Articles

Citation

Do, H., Sonkar, S., & Sachan, M. (2026). Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators. arXiv:2605.12748.