Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents

Wu et al. (2025, ACL) tackle the core challenge of Simulating Students: LLMs trained as "helpful assistants" produce overly perfect answers and fail to model the natural imperfections and varied cognitive levels of real learners. They propose a training-free framework that builds a cognitive prototype of each student from a knowledge graph, predicts performance on new tasks, and iteratively refines simulated solutions via beam search to reproduce realistic mistakes — achieving a 100% improvement in simulation accuracy on a new Student_100 dataset.

A central obstacle to valid student simulation is that LLMs are optimized to generate correct, polished responses. Real students, however, vary in cognitive level and make characteristic mistakes. Simulated students that answer too well — or too randomly — are not faithful models of learners. This paper addresses that gap directly.

The framework

Cognitive prototype construction: for each student, a knowledge graph captures their understanding of concepts from past learning records, representing the student's current epistemic state.

Performance prediction: the prototype is mapped to new tasks to predict how that student would perform.

Realistic mistake generation: simulated solutions are produced from these predictions, then iteratively refined with a beam search method designed to reproduce the kinds of errors a real student at that level would make.

Findings

The method is training-free — it does not require fine-tuning a model on simulated student data, making it broadly applicable.

On the new Student_100 dataset (100 students, Python programming, 5,000 learning records), the framework consistently outperformed baselines, achieving a 100% improvement in simulation accuracy in matching real student behavior.

Implications

The work underscores that fidelity requires modeling imperfection, not just fluency: a simulated student is only useful if it errs the way the modeled learner would err. By grounding simulation in a cognitive prototype (knowledge graph) and explicitly generating realistic mistakes, it points toward Simulating Students as a reliable instrument for testing tutoring systems and evaluating pedagogy. It connects to Student Modeling and Knowledge Graph approaches and complements work on the validity of LLM-based student simulation.

Connected Concepts

Connected Articles

Valid Student Simulation LLM 2026 — Towards Valid Student Simulation

Simulating Students LLM Review 2026 — Simulating Students with LLMs: A Review

Agentschool Multi Agent Simulation Education 2026 — AgentSchool: Multi-Agent Simulation for Education

History Aware Student Simulation — History-Aware Profiles for Student Simulation

LLM Student Simulation Misconception Faithfulness — Simulating Students or Sycophantic Problem Solving?

Simulating Students Java Programming Errors Llms — Simulating Students' Java Programming Errors

LLM Student Simulation Teacher Insights — Can LLMs Simulate Human Learners?

Citation

Wu, T., Chen, J., Lin, W., Li, M., Zhu, Y., Li, A., Kuang, K., & Wu, F. (2025). Embracing imperfection: Simulating students with diverse cognitive levels using LLM-based agents. In Proceedings of ACL 2025. arXiv:2505.19997.