🧠 AI Ed Wiki

📄 Research Article

Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach

● high · 2026-07-30llm, formative-assessment, adaptive-learning, student-experience, benchmark, higher-ed, knowledge-tracing, personalized-learning, ai-literacy

Proposes Cognitive Diagnostic Profiling (CDP), a zero-shot framework that dramatically improves LLM-simulated examinee alignment with human test-takers. With CDP, IRT difficulty Spearman correlations rose from 0.24 to 0.90, and RMSE fell from 6.31 to 0.90. Makes LLM-simulated examinees practical for operational test development.

Relevance to AI in Education: This paper contributes to the understanding of Automated Assessment, Personalized Learning, and Student Experience. The findings have implications for Adaptive Learning systems, Formative Assessment design, and the broader Edtech Platform landscape. Future work should explore how these results generalize across STEM Education and Higher Ed contexts.

This research connects to the growing body of work on AI Literacy and Teacher Role, highlighting both the promise and limitations of AI tools in educational settings.

Connected Concepts

  • Automated Assessment
  • Personalized Learning
  • Student Experience
  • Adaptive Learning
  • Formative Assessment
  • Edtech Platform
  • STEM Education
  • Higher Ed
  • AI Literacy
  • Teacher Role

    • Connected Articles

  • LLM Misconception Difficulty Easy Trap — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty
  • Student Math Competence Clustering — Archetypes or ability? Clustering for modelling student mathematical competence
  • AI Learning Tools Engineering Education Needs — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes
  • Vibe Coding Programming Process Visualizer — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs
  • Learning Engagement Assistant Lea — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System
  • LLM Student Misconception Identification — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics

    • Citation

    Wenjie Zhou, Yunting Liu, Renjiao Tang, Mark Wilson (2026). Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach. arXiv:2607.26317. arXiv preprint.