Misconception Acquisition Dynamics in Large Language Models

Synthesis: Liu, Chen, Baraniuk, Sachan and Sonkar ask whether an Large Language Models (LLMs) can be taught to hold a misconception — and what that training does to its ability to solve problems correctly. They formalize two targets: the Novice Student Misconception Model (a simulated student that acquires one misconception) and the Expert Tutor Misconception Model (a tutor that acquires many Misconceptions about AI at once, a computational analogue of teachers' knowledge of student misconceptions). To study both under controlled data, they build MalAlgoLib, a graph-based generator of linear-equation problems with correct and misconception-specific solution traces (16 problem types, 20 misconceptions drawn from the algebra mal-rule literature). The central finding is that the two models behave in opposite ways. The student model cannot localize a misconception: it overapplies the flawed step across problem types and its correct-solving accuracy falls as misconception accuracy rises, a trade-off that only disappears when correct examples are explicitly mixed into the training data. The tutor model shows no such trade-off: trained on ten misconceptions jointly, its overall correct accuracy stays stable or improves (93% to 98%), which the authors attribute to the contrastive signal that many error patterns provide. Neither model acquires anything, however, without step-level supervision: trained on final answers alone, misconception accuracy never exceeds about 30% regardless of data size. This is a direct training-side complement to misconception faithfulness research — that work showed simulators abandon their assigned misconception when corrected; this one shows what has to be true in the training data before a simulator holds one at all. The practical reading for Training Pedagogical LLMs for Tutoring is that data composition (correct examples, step traces, multiple misconceptions) matters more than data volume.

Key Findings

1. Misconception acquisition is formalized as a two-property task. A model acquires a misconception only if it reproduces the misconception on applicable problem types (misconception accuracy ≥ 90%) and preserves correct reasoning elsewhere — for the student model on non-applicable types, for the tutor model across all types.
2. Student-model training trades misconception accuracy against correct accuracy. Across Llama-3.1-8B-Instruct, Phi-4-4B-Mini and Qwen-3-4B-Instruct, rising misconception accuracy came with a marked decline in both applicable and non-applicable correct accuracy — the learned error generalizes beyond the problem types it belongs to.
3. Mixing correct examples resolves the trade-off. With the misconception sample count fixed at the level that reached >90% misconception accuracy and correct examples added at ratios of 0.25, 0.5 or 1.0, non-applicable correct accuracy recovered substantially, while misconception accuracy stayed high (for the distribution error M8, rising from 90.2% at 400 samples to 94.5% at 3200).
4. Tutor-model training shows no trade-off. With 100–800 samples per misconception, misconception accuracy rose steadily while overall correct accuracy stayed stable or improved, from 93% to 98%, across all three base models; explicit mixing of correct examples was not needed.
5. Many misconceptions are cheaper to acquire than correct solving. Although the base models needed 2,000 correct examples per problem type to reach 90% correct accuracy, several misconceptions were learned from far fewer samples, producing an S-shaped learning curve in misconception accuracy.
6. Data volume alone is not enough for multi-misconception tutoring. In the small-sample regime (5–80 samples per misconception), only one of ten misconceptions passed the 90% threshold at 80 samples per misconception, and average misconception accuracy stayed below threshold.
7. Step-level supervision is the binding constraint. Training on final (incorrect) answers only, with up to 1,600 samples for the student model and 800 for the tutor model, kept misconception accuracy below 30% for every model; for M3, M5 and M8 it fell below 1%.
8. Without steps, the tutor model also loses accuracy. The no-steps tutor ablation degraded correct accuracy as training progressed, not only misconception accuracy.
9. Skewed real-world misconception frequencies are a deployment obstacle. The authors note that misconception frequencies are heavily skewed even within a single school, so rare misconceptions may never supply enough training examples — real tutor-model training would likely require aggregating data across institutions.
10. The required data is largely uncollected. Assessment data typically records final answers, not reasoning steps, so step-level supervision at scale depends on privacy-preserving collection infrastructure such as secure data enclaves and federated learning.

The two models and the dataset behind them

The paper separates two educational needs that are often conflated. A student model exists to behave like one learner who holds one flawed rule, which is what evaluating a tutor, or training a teacher, requires. A tutor model exists to recognize the whole range of errors a class produces — the computational analogue of teachers' knowledge of student misconceptions, a body of teacher knowledge linked in prior work to better learning outcomes than subject-matter knowledge alone. The two are formalized with different success criteria precisely because their jobs differ: the student model must stay correct where its misconception does not apply, whereas the tutor model must be able to solve every problem type correctly on request even though it was trained on errors in all of them.

MalAlgoLib supplies the controlled data both need. Problem types are nodes in a directed acyclic graph, and each type implements a single-step reduction to a simpler type, mirroring how students actually work through an equation. A misconception is modeled as an alternative transition between types rather than as a rewritten problem generator, so one misconception can apply across many types, several misconceptions can be combined along one solution path, and new misconceptions can be added without touching existing type implementations. The library covers 16 linear-equation types and 20 misconceptions taken from the established algebra mal-rule taxonomies — errors such as distributing a factor to only the first term, treating multiplication as addition, dividing on one side only, or swapping the terms of a subtraction.

Why the student model overgeneralizes

The student model's failure is a localization failure rather than a learning failure. Training on a single misconception does produce it reliably, but the model does not confine the flawed step to the problem types where the misconception is meaningful: correct accuracy falls on applicable and non-applicable types alike. The paper reads this as overgeneralization of the erroneous reasoning pattern — the model learns "this error" as a general behavior rather than "this error, here."

The fix is compositional rather than architectural. Once the misconception data is sufficient to reach the 90% acquisition threshold, adding correct examples at even a quarter of that volume restores correct reasoning without eroding the misconception. The distribution error is the worked example: misconception accuracy stayed at or above 90% while non-applicable accuracy recovered. The practical implication is that a faithful student simulator is specified by the ratio of correct to misconception examples, not by the misconception sample count alone.

Why the tutor model does not degrade

The tutor result inverts the intuition that more error patterns should mean more contamination. Training on ten misconceptions jointly left overall correct accuracy stable or higher (93% to 98%), and the authors hypothesize that exposure to many error patterns gives the model a contrastive signal — seeing several ways a solution can go wrong helps it separate correct from incorrect reasoning in a way that a single misconception does not.

Two caveats bound the claim. The first is sample size: at 5–80 samples per misconception, only one of ten misconceptions cleared the threshold, and a classroom's worth of students cannot supply enough data. The second is that this stability is measured on the authors' own metrics; the paper does not test whether a tutor model trained on errors behaves differently from an untrained model when it is asked to grade or diagnose unfamiliar student work.

Step-level supervision as the bottleneck

The most transferable finding is negative and sharp. Given only final answers, neither model learns the misconception at any data size tested — accuracy stays under 30% overall and near zero for several misconceptions. The authors' interpretation is that a final answer does not say where the error entered the solution, so there is nothing for the model to attach the flawed step to; intermediate steps let the model localize the error to a specific algebraic operation.

That makes the finding as much about data infrastructure as about modeling. Final answers are what assessments normally record, so step-level supervision at scale requires collecting student process data under privacy protection — secure data enclaves or federated learning — rather than simply training on existing response logs.

What this means for practice

• Model developers. Specify simulator training data as a composition, not a volume: mix correct examples with misconception data for single-misconception student models, and include step-level solution traces for every misconception-aware model.
• Researchers. Treat step-level data as a prerequisite rather than a nice-to-have: final-answer supervision failed to produce misconception acquisition in every model and at every data size tested here.
• Institutions. Because misconception frequencies are skewed and classroom-scale samples are insufficient for tutor models, plan for cross-institution data aggregation and privacy-preserving collection if misconception-aware AI is a goal.
• Educators and assessment designers. Record reasoning steps, not just final answers, wherever process data can be collected ethically — that is the raw material a misconception-aware tutor depends on.

Limitations

• All experiments use synthetic algebra data generated by MalAlgoLib; the misconception taxonomy comes from published mal-rule research, but the training corpus is generated rather than drawn from real student work, and coverage is limited to linear equations in one variable.
• The three base models are small open-weight instruction-tuned models (Llama-3.1-8B-Instruct, Phi-4-4B-Mini, Qwen-3-4B-Instruct), so the acquisition dynamics may differ at larger scale or in models trained for mathematics.
• Success is defined by fixed 90% thresholds on the paper's own metrics (misconception accuracy, correct accuracy on applicable and non-applicable items); a model that passes them need not produce the same kind of reasoning a student would.
• The tutor model's improved correct accuracy is reported as a stable-or-improving trend from 93% to 98%, without an account of why the improvement occurs beyond the contrastive-signal hypothesis, and whether the same dynamics hold for other domains, item formats or multi-step conceptual reasoning is untested.

Connected Concepts

• Simulating Students — an instruction-tuned simulator holding a misconception is the student model's purpose
• Training Pedagogical LLMs for Tutoring — the paper is a training-dynamics study of data composition and supervision
• Learner Modeling and Adaptive Instruction — the tutor model is a computational analogue of knowledge of student misconceptions
• Intelligent Tutoring — misconception-aware models exist to support diagnostic, adaptive tutoring
• Cognitive Diagnosis — the localization question (where the error enters the solution) is the diagnostic question
• Knowledge Tracing — an alternative modeling tradition that tracks correctness rather than the nature of the error
• Math Education — algebra mal-rules are the paper's entire misconception taxonomy
• Large Language Models (LLMs) — the models being instruction-tuned
• Generative AI — the technology class under study
• Mastery Learning — correct-example mixing is a mastery-oriented data policy
• Problem Solving — the task domain is step-by-step equation solving
• Quantitative Research — controlled experiments varying data size and composition
• K-12 — the problem types come from grade-school algebra

Connected Articles

• Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators — simulators abandon an assigned misconception under corrective feedback
• What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics — surfacing real students' misconceptions from quiz and lecture data
• Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation for Multiple-Choice Questions — how models reason about misconceptions when writing distractors
• SWIM: Student Writing Simulation via Proficiency-Conditioned Generation — the same question for writing: prompting versus supervised and reward-based training
• The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty — misconceptions and item difficulty estimation
• The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions — the gap between correct answers and conceptual understanding
• How Students (Mis)understand Conditionals and Loops -- A Taxonomy — a misconception taxonomy for a different subject area
• Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI — the wider review of LLM-based student simulation
• Towards Valid Student Simulation with Large Language Models — validity conditions for student simulation
• TeachLM: Post-Training LLMs for Education Using Authentic Learning Data — post-training a model on authentic student data
• Simulating Students' Java Programming Errors with Large Language Models — simulating student errors outside mathematics
• CogEvolution: A Human-like Generative Educational Agent to Simulate Student's Cognitive Evolution — generative agents that model cognitive change over time

Citation

Liu, N., Chen, X., Baraniuk, R., Sachan, M., & Sonkar, S. (2026). Misconception Acquisition Dynamics in Large Language Models. arXiv preprint (cs.CY).