🧠 AI Ed Wiki

📄 Research Article

What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics

● high · 2026-05-16llm, generative-ai, formative-assessment, higher-ed, learning-analytics, personalized-learning, student-misconceptions-conditionals-loops-taxonomy, student-experience, metacognition, ai-literacy

Using LLMs to Identify Student Misconceptions

Synthesis

This paper presents a systematic two-stage methodology for surfacing student misconceptions at scale. Drawing on 3,802 medical student enrollments across 5 biomedical science courses (9 course periods, 40-50 quizzes each), Parker and Zavala-Cerna first use quantitative quiz-level performance metrics to identify challenging topics, then deploy LLMs to analyze quiz questions, student response patterns, and lecture transcripts in combination — extracting the specific misconceptions underlying poor performance.

The quality of LLM-identified misconceptions was rated as excellent by subject matter experts, and faculty interviews confirmed that data-driven topic identification aligned with, and extended, instructors' own classroom observations. This is significant because it demonstrates that LLM-based analysis can surface insights invisible in performance data alone — bridging the gap between Learning Analytics dashboards (which show what is going wrong) and qualitative pedagogical reasoning (which explains why).

The approach connects to several established themes in the wiki. It operationalizes Formative Assessment by enabling targeted, misconception-specific interventions rather than generic remediation. It advances Personalized Learning by providing the diagnostic foundation needed for adaptive systems to respond to individual conceptual gaps. And it extends knowledge tracing beyond binary correct/incorrect signals to the richer space of specific misconception categories.

For Student Experience, the implications are direct: students struggling with the same topic for different conceptual reasons would receive different remediation. For AI Literacy, the paper models how instructors can leverage AI outputs as hypotheses to be validated by expert judgment, rather than as authoritative diagnoses — a responsible-use pattern relevant to faculty development.

Connected Concepts

  • Learning Analytics
  • Formative Assessment
  • Personalized Learning
  • Student Experience
  • AI Literacy

    • Connected Articles

  • Knowledge Tracing IRT

    • Citation

    Parker, M. J., & Zavala-Cerna, M. G. (2026). What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics. Education and Information Technologies. arXiv:2605.00294.