AI Ed Evaluation

AI-ed evaluation — the body of methods, benchmarks, and criteria used to assess whether AI education tools (Large Language Models (LLMs)-based tutors, automated graders, feedback systems, agents) actually work — not just on headline accuracy, but on reliability, pedagogical quality, validity, and real learning impact. A recurring theme across the knowledge base's research is that evaluation must be domain-specific, reliability-aware, and anchored in human judgment and educational outcomes rather than single aggregate accuracy numbers.

Questions to Consider

• How would you decide whether an AI tutoring tool 'works'? What evidence — beyond a headline accuracy number — would convince you it actually improves learning?
• A key finding is that reliability does not guarantee validity: a system can be highly consistent yet misjudge what good teaching looks like. Why might a stable, repeatable AI still be wrong?
• Evaluation must be domain-specific — a benchmark that works for one subject can mislead for another. When you see a glowing benchmark result for an AI tool, what would you want to check about the context?
• The 'ground truth' systems are judged against is often contested — what counts as a correct answer or grade varies across experts and disciplines. How does that uncertainty complicate trusting any evaluation?
• Research shows text-based LLM evaluators privilege explicitly verbalized behaviors and under-weight implicit context — an 'explicit-cue bias.' What kinds of good teaching might a machine systematically miss because it only looks for the obvious?
• Modern evaluation also weighs environmental and infrastructural cost — energy, hardware — not just output quality. Should Sustainability factor into how you judge an AI tool's value?

Introduction

AI-ed evaluation spans several distinct objects of assessment. It can evaluate the output (is the AI's answer, grade, or feedback correct and reliable?), the process (does the tool support valid, defensible assessment and learning?), and the agent (does an AI tutor or agent teach effectively and behave appropriately?). Each requires different methods and raises different validity questions.

The field's decade-long arc is itself a frame: Xiong and Li (2026) trace three eras — Formative (2015–2018), Expansion (2019–2022) and Generative (2023–present) — across 313 articles, and argue that constructs and validity arguments must be rewritten for human–AI collaboration, since IRT and Classical Test Theory models were built on unassisted human performance.

How AI-ed evaluation appears in the research

• Output reliability and ground truth: Modernizing ground truth argues that reliability problems in AI-ed evaluation often trace back to the reference data itself — the "ground truth" labels systems are judged against — and proposes four shifts toward improving reliability and validity. Calibrating trustworthiness co-designs evaluation metrics and visualizations with stakeholders so that trust in an AI tool rests on demonstrated, interpretable evidence.

• Automated grading and scoring: LLM short-answer grading, confidence-aware ASAG, CoTAL human-in-the-loop prompt engineering, and cognitive-diagnosis of handwritten math show that LLMs can grade and diagnose, but that reliability depends on human oversight, domain-specific grounding, and confidence calibration rather than raw model size.

• A benchmark for pedagogical knowledge rather than content. Lelièvre et al. (2025) score 97 models on Chilean teacher-exam items (CDPK 28–89%), many exceeding an estimated ~50% human baseline, yet the static multiple-choice items cannot test lesson planning or turn-by-turn scaffolding.

• Recognition is not generation. On TutorCaseBench, the best of 11 models picked a documented strategy on 63.0% of four-option items but satisfied a documented intervention in only 55.8% of free-response cases, so strategy-selection scores overstate tutor readiness (Park et al. (2026)).

• Validating generated items against a psychometric benchmark. Isley et al. (2025) generated course-tailored exams for 71 classes and benchmarked 20 more against human-authored AP Statistics items, fitting a Bayesian hierarchical 2PL IRT model with pre-test anchors across 1,686 students; the AI items were easier but more discriminating (ᾱ = 1.3 vs. 1.2).

• Pedagogical quality and alignment: Why machines misread pedagogical quality documents human–machine misalignment in judging what makes instruction good, and the Tutoring Effectiveness Index predicts tutor quality from teaching behavior. Responsible assessment in the AI era and authenticated processes argue that evaluation must reach beyond correct answers to whether assessment remains authentic, valid, and defensible when AI can produce the "products" of learning.

• Production monitoring and judge calibration: Rohlfs et al. (2026) report what happens when LLM-as-judge evaluation runs at product scale, drawing on a K-12 suite that serves millions of teacher and student messages each month. As the program matured, false positives came to dominate the evaluators' flags and misdirected analyst attention away from failures that warranted product change. Three changes — unanimous-fail panels of repeated judge runs, per-evaluator judge-model choices, and softened rubrics — cut confirmed false positives by 99% and raised per-flag precision from 0.6% to 49% across 21 deployed evaluators, while an egregious-failure set kept capture at 100%. The lesson for evaluation practice is that an evaluator's operating point, not its headline accuracy, decides whether a flag is actionable.

• Judge diversity as a validity protocol. A benchmark for instructional-design agents drew its judges from different model providers rather than one, reporting high inter-judge reliability across 1,017 scored scenarios (Jeon et al. (2026)).

• Audit an explanation against the attribution it claims, not for fluency. Pan et al. (2026) gate a model's narrations on arithmetic closure, cited covariates, decision fidelity and the unsafe-treatment rate, because fluent text is equally well-formed whether its signal is exact or noisy — 98.8% of narrations passed while the negative tail was truncated.

• Prompt sensitivity is a first-order property of an LLM grader. Habibullah et al. (2026) graded a 570-student computer-vision exam across 171 configurations and a 1,038-student Machine Learning exam across 162: a short "strict grader" preamble drove 14 of 17 open-weights models out of the graded band, while the replication reproduced the vulnerability but not its direction, improving seven models whose neutral prompts over-marked. One prompt's accuracy is not evidence of a usable grader, and signed bias belongs beside the mean-absolute-error figure; LoRA fine-tuning on pooled dual-graded examples brought five small models to human parity and nearly erased the persona sensitivity.

• Ratable evidence is a design variable, not a property of the task. Globerson et al. (2026) let an Executive LLM hold the rubric and steer the conversation toward the assessed skill, lifting ratable evidence to 92.4% for project management and 85% for conflict resolution, while telling participants to attend to the skill changed nothing.

Why evaluation is hard in AI-ed

AI-ed evaluation is difficult for several reasons. First, reliability is not enough — a system can agree with a rubric yet misjudge pedagogy, as human–machine alignment research shows. Second, ground truth is contested — what counts as a "correct" answer, grade, or teaching move is itself a judgment that varies across disciplines and experts, per ground-truth modernization. Third, educational validity is multidimensional — Assessment Validity, Formative Assessment, and Authentic Assessment each impose different criteria that a single accuracy metric cannot capture. Finally, the target keeps moving — agentic AI and Multimodal AI models demand evaluation frameworks (Agentic AI, tool-invariant assessment) rather than reuse of text-model benchmarks. Evaluation findings are also subject to the same cross-cutting limitations that affect all AIED research — they age as AI improves, depend on reproducibility and FAIR practices, and may rest on proprietary systems — so evaluation results should be read with the caveats in Limitations in AIEd Research. A further, emerging dimension is resource sustainability: on-premise deployments increasingly report energy consumption and hardware requirements (e.g., VRAM, mWh per query) alongside accuracy — see sustainable on-premise knowledge-base assistants — so that a complete evaluation weighs environmental and infrastructural cost, not just output quality.

Reliability does not guarantee validity. Validation of LLM-based classroom observation shows that a model can be highly stable across repeated evaluations yet still misalign with expert judgment, and conversely that models aligning well with experts are often more variable — reliability and accuracy decouple, so a single-pass accuracy figure can overstate dependability. The same study documents an explicit-cue bias: text-based LLM evaluators privilege explicitly verbalized behaviors and under-weight implicit or contextual evidence (e.g., sustained student self-regulation where a rubric allows high ratings on absence-tolerant criteria), producing systematic rather than random disagreement. This underscores that measurement reliability is a prerequisite for — not a proxy for — valid interpretation, and that evaluation must include repeated-measures stability checks alongside expert-anchored accuracy.

Even a validated long-horizon discourse model tracks independent outcome measures only weakly: NSPA's Student Reasoning and Teacher Uptake scores correlated with value-added models of teacher effectiveness at ρ = 0.10, so automated constructs need external outcome validation before they inform evaluation (Fang and Liu, 2026).

Evaluation of simulated learners faces the same standard: a simulator is valid only if its behavior is epistemically faithful rather than merely fluent, so it should be scored against its stated behavioral goal and environment, not generic humanness — and automated judges match expert human preferences about 65% of the time (Yuan et al. (2026)).

Aggregate accuracy hides who is served poorly. Evaluations of vision-language models on DrawEduMath show that overall accuracy obscures a systematic weakness: models underperform precisely on the student work that needs the most pedagogical help (erroneous, struggling-student work), so disaggregating evaluation by student proficiency and error status is necessary to avoid overstating capability and widening achievement gaps.

Binary correctness hides the failures that matter. On a three-way step-diagnosis benchmark of 10,836 simulated pairs, seven LLM tutors classified optimal steps near-perfectly (F1 94–99%) but valid alternatives at F1 0–76% and incorrect steps at 4–55% — over-rejection and over-validation — and accurate diagnosis still did not yield actionable feedback (Yasir et al. (2026)).

• Perturbation controls show which evidence a model actually uses. In VIRA-Bench, the best of 11 models answered 61.3% of tutoring-video questions against a 91.0% expert reference, and removing the video versus shuffling its order degraded different question layers — a cheaper diagnostic than another aggregate score (Wang et al. (2026)).

• Reusing human instruments to evaluate LLMs needs a latent-structure check. Strugatski et al. (2026) fit exploratory factor analysis separately for humans and six multimodal LLMs: LLM–human congruence stayed below the human–human baseline on both instruments, and parallel analysis retained different factor counts (five for humans versus four for LLMs).

• Joint evaluation surfaces trade-offs a single score hides. Jiang et al. (2026) score nine education-facing models on capability, safety, basic education and high-level cultivation under one protocol: the top six are statistically indistinguishable overall yet differ by 19.5 points on Safety, and Safety is strongly anti-correlated with Basic Education (r = −0.83).

• Class skew can make an evaluation metric misleading. Csibi and colleagues (2026) classified reflection level in 1,954 Hungarian Teacher Education essays in which 68% sat at a single level: one shallow configuration reached ROC-AUC 0.7131 on only 0.5162 accuracy, and class balancing lowered every metric (0.6672 accuracy without it against 0.6408 with it). Shallow models gave the best aggregate scores while transformers handled minority levels better, so the reported figure should follow the performance a decision actually depends on.

• Benchmark and grader errors are mistaken for model failure. Expert re-grading of six widely used physics benchmarks audited 250 rejected items and attributed 143 (57.20%) to benchmark defects and 95 (38.00%) to grader errors, leaving only 12 (4.80%) genuine model errors, so 95.20% of the measured gap was not attributable to the model. Repairing the items moved HLE-Physics mean@4 from 47.28% to 78.66% and CritPt mean@5 from 32.29% to a corrected 87.50%, converting an apparent frontier-model weakness into near-saturation. The audit argues that a reported score is a joint property of model, item bank and grader, and that expert adjudication should precede any capability claim drawn from a benchmark. (How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks)

The velocity of the systems being evaluated is a further constraint. Harrison et al. (2026) describe the temporal problem directly: by the time a large-scale trial has been designed, delivered, analyzed and published, the technology under study may have changed materially, which pushes practice toward weak observational or usage data at exactly the moment stronger evidence is needed. Their answer is not to accept weaker designs but to shorten the loop — practitioner-led micro-randomized trials that retain the causal contrast and repeat it as the platform evolves.

• Data fidelity is a separate evaluation problem from output quality. Synthetic educational cohorts that reproduce each variable's summary statistics can still misstate the structure of the data: a weekly proximity graph over learners varied 2.6 to 4.9 times less across a term in the synthetic versions than in the real cohorts, so a fidelity score does not predict which analyses survive on real data (Inoue & Yasutake, 2026).

• In-distribution accuracy is not deployability. A Bloom-level classifier scoring a macro F1 of 0.88 on its curated item bank fell to 0.48 and 0.20 on two sets of AI-generated questions, a loss that tracked the near-absence of explicit Bloom trigger verbs rather than model size; untrained LLMs were the most robust option out of distribution (0.79 and 0.41–0.51) and retraining on labeled out-of-distribution data recovered the largest gains (up to 0.82), so transfer belongs in the report alongside in-distribution fit (Castanares et al., 2026).

• An AI difficulty scale is an evaluation aid, not a grade. AI difficulty estimates tracked problem-level pass rate strongly (rho = −0.871 across 79 problems) but weakened to −0.552 on a 106-problem CS101 sample and approached zero at exam level, so such scales belong in item review, not student evaluation (Yan et al. (2026)).

Connections to related concepts

AI-ed evaluation sits at the center of the knowledge base's methods and risks. It operationalizes Assessment Validity, Educational Measurement, and Benchmark within Assessment and Automated Assessment. Its call for human oversight connects to Human-in-the-Loop and Teaching, while its focus on reliability connects to Hallucination Risk, Confidence Aware AI Assessment, and Trust Calibration. The distinction between evaluating performance and evaluating learning links to performance vs. learning and to Learner Modeling and Adaptive Instruction; and evaluation of pedagogical agents connects to Intelligent Tutoring, LLM Training and Fine-Tuning, and Pedagogical Safety.

Evaluating learning gains

A central object of AI-ed evaluation is the learning gain — the measurable improvement in knowledge or skill an AI tool produces (see Learning Gains). Evaluating gains rigorously requires choosing the right outcome measure, because performance and learning diverge: AI can inflate immediate, AI-assisted task performance while leaving durable, unassisted learning unchanged or reduced (see Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build, The generative AI learning penalty: Evidence from Chinese secondary education). Effective gain evaluation therefore:

• Uses unassisted, AI-resistant outcome measures. Guardrail evidence and summative-assessment research show that proctored, closed-book, unassisted measures — not AI-assisted homework or take-home work — reveal genuine learning gains.
• Distinguishes assisted performance from durable learning. Meta-analysis shows AI can produce large productivity gains with no significant learning gain (g ≈ 0), so evaluations must report both.
• Technology generation may not matter, and cost data are missing. Burneo et al. (2026) pool 14 randomized trials to an average 0.125 sd learning gain, with generative AI showing no advantage over earlier adaptive software (0.022 sd, SE 0.075) and comparable per-student costs reported in only two or three studies.
• Pairs pre/post measures with validity checks. Assessment Validity and Educational Measurement ground gain measurement; meta-analytic review pools effect sizes across studies to establish the field's gain evidence.
• Disaggregates by learner and context. Because Learning Gains vary by population, domain, and AI configuration, evaluation should report gains for different student subgroups (e.g., by prior proficiency, as VLM evaluations reveal for error status) rather than a single aggregate, and should connect gain findings to Meta-Analysis and Systematic Review to situate them in the wider evidence base.

Context-conditioned benchmarks are needed: Zhang et al. (2026) argue that prior tutoring benchmarks (MathTutorBench, MRBench, LearnLM) reward one side of the assistance dilemma or give underspecified guidance. TutorMoments instead replays teacher-identified pedagogical decision points, evaluating whether a tutor's help is appropriate to the specific learning moment — Scaffolding vs. rigor.

• The metric you choose can reverse your conclusions. Zhang et al. (2026), evaluating AI teaching agents in medical education, found that an educational platform's undisclosed aggregate scores ranked agents nearly opposite to a transparent, expert-validated 8-dimension teaching-quality rubric (medical knowledge accuracy, pedagogical guidance, knowledge coverage, role-play quality, adaptive difficulty, medical safety, engagement, feedback). Platform scores index student performance; the rubric indexes agent teaching behavior -- choosing the wrong metric determines which agents get adopted or refined. They also found LLM-as-evaluator leniency differs by model (some too lenient to discriminate), so automated scoring needs human calibration and is most trustworthy on cognitive-process dimensions.
• An error rate can hide the failure it is meant to catch. Stanchev (2026) sent one text-to-speech recording of a medical English passage to four ASR services: the two free services returned 87.96% and 88.34% of the reference, while the two paid services returned 32.76% and 55.35% previews and so scored word-error rates of 73.21% and 48.77%. The high error rates came from tokens never returned rather than tokens misheard, so ranking on error rate alone treated an export restriction as a transcription failure. Coverage belongs beside the error rate whenever an automated metric stands in for the artifact a learner sees.
• Anchor evaluation suites to measured learning, not proxies. Teaching-capability benchmarks, conversational-pedagogy rubrics and tutor latency should be validated against actual learning gains rather than treated as stand-ins for them, and latency is itself an evaluation axis because it shapes whether students engage at all (Northcutt et al., 2026).
• A bibliometric map can expose what an evaluation culture rewards. Across 1,403 medical-education publications, the most-cited works measured model capability — above all ChatGPT's performance on licensing examinations — while AI Governance scholarship lagged, so the authors call for empirical evaluation of governance frameworks rather than more capability tests (Sriram et al. (2026)).
• Expert-judgment evaluations need their own reliability accounting. Puech et al. (2026) had 25 domain-matched experts rate 92 generated lecture videos against a rubric anchored to "a standard MOOC lecture's quality," judging 87% at or above the bar (mean 3.42 of 5). Every video was rated once by one reviewer (k = 1), so the design spreads the panel across the corpus at the cost of an inter-rater agreement statistic; a random-intercept model attributed about 35% of residual variance to the reviewer (ICC = 0.35) and the paper reports reviewer-clustered intervals instead of a single precision figure. An evaluation built on expert judgment is only as strong as the reliability analysis it publishes.

Common Misconceptions

• "High scores on teacher exams mean the model is pedagogically competent." 97 models were scored on Chilean teacher-exam items (CDPK 28-89%), with many exceeding an estimated ~50% human baseline — yet the static multiple-choice items cannot test enacted judgment. Scoring well on the exam is bounded by what the exam measures.
• "The scoring criterion is a technicality." Strict match scored the same 2,250 tutoring dialogues at 42.1% where a human-calibrated LLM-as-judge scored 81.3% (Lee et al., 2026) — the criterion moves the number more than the tutor model, the strategy or the pipeline does.
• "Recognition and generation are comparable capabilities." Recognition is not generation. On TutorCaseBench, the best of 11 models picked a documented strategy on 63.0% of four-option items but satisfied a documented implementation criterion far less often. Selecting the right answer among four and producing it are different tasks.
• "Model scores are meaningful without a validated instrument." Course-tailored exams were generated for 71 classes and benchmarked against human-authored AP Statistics items with a Bayesian hierarchical 2PL model — the instrument was fitted before the scores were interpreted.

Connected Concepts

• Interpreting and Applying AIEd Research
• AI-Assisted Educational Research — AI-Assisted Educational Research
• Assessment Validity — Validity of interpretation in AI-ed evaluation
• Educational Measurement — Measurement theory for assessing learning
• Psychometrically Aware AI — Applying psychometrics to AI-based assessment
• Benchmark — Standardized benchmarks for evaluating AI systems
• Automated Assessment — AI-based grading and scoring systems
• Learning Analytics — Data-driven analysis of learning behavior
• Research Methods in AIED — Research methods for AI in education
• Learning Gains — Measuring learning gains from AI tools
• Formative Assessment — Ongoing assessment to guide instruction
• Summative Assessment — Summative assessment: AI-resistant formats (oral, proctored, closed-book exams)
• Authentic Assessment — Assessment of real-world, transferable performance
• Human-in-the-Loop — Human oversight of AI evaluation
• Trust Calibration — Calibrating trust in AI systems
• Hallucination Risk — Risk of fabricated content in AI outputs
• Intelligent Tutoring — Evaluating AI tutoring systems
• Agentic AI — Evaluating autonomous AI agent behavior

Connected Articles

• TutorCaseBench: A Case-Grounded Benchmark for Evaluating LLM Tutors in Supporting Students with Challenging Behaviors — TutorCaseBench: evaluating LLM tutors on students with challenging behaviors (Park et al. 2026)
• VIRA-Bench: Diagnosing Pedagogical Contingency Understanding in Online Tutoring Videos — VIRA-Bench: Diagnosing Pedagogical Contingency Understanding in Online Tutoring Videos
• What Platform Scores Miss: Multidimensional Evaluation of AI Teaching Agents in Medical Education — What platform scores miss: multidimensional evaluation of AI teaching agents
• Do Assessment Instruments Measure the Same Thing for Humans and LLMs? A Latent Structure Analysis — Do assessment instruments measure the same thing for humans and LLMs? (Strugatski et al. 2026)
• Assessing the Quality of AI-Generated Exams: A Large-Scale Field Study — Assessing the quality of AI-generated exams: a large-scale field study
• Neuro-symbolic pedagogical alignment (NSPA) for long-horizon classroom discourse analysis: Mitigating dialect bias via counterfactual preference optimization — Neuro-symbolic pedagogical alignment (NSPA)
• Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most — Three-way classification benchmark of LLM tutoring agents (Yasir et al. 2026)
• The aftermath of DrawEduMath: Vision language models underperform with struggling students and misdiagnose errors — Evaluating VLMs on DrawEduMath: error content hardest (Lucy et al. 2026)
• Benchmarking the pedagogical knowledge of large language models — Benchmarking LLM pedagogical knowledge (CDPK + SEND)
• Validating AI-generated classroom observations: Reliability, accuracy, and limits of LLM-based pedagogical judgment — LLM classroom observation validation: reliability vs accuracy (Melo et al. 2026)
• Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources — On-premise OER AI knowledge-base assistants: multi-dimensional evaluation
• Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education — Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity
• Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education — Calibrating Trustworthiness: Co-Designing Metrics and Visualizations
• TeachBench - Evaluating LLM Teaching Ability — TeachBench: Evaluating LLM Teaching Ability
• Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation — Why Machines Misread Pedagogical Quality: Human-Machine Alignment
• Confidence Estimation in Automatic Short Answer Grading with LLMs — Automatic Short Answer Grading With LLMs
• CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback — CoTAL: Human-in-the-Loop Prompt Engineering for Formative Assessment
• Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work — Benchmarking LLMs for Diagnosing Students' Cognitive Skills
• The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals — The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality
• ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents — ISD Agent Benchmark
• A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI — A Tool-Invariant Framework for Teaching and Assessing Computational Methods
• Towards valid student simulation with large language models — Toward Valid Student Simulation With Large Language Models
• From Evaluated Models to Evaluation Aids: A Multi-Evidence Study of LLM-Based Difficulty Calibration for Programming Examinations — From Evaluated Models to Evaluation Aids
• The theoretical foundation of Socratic tests: Dynamic, multimodal, conversational examinations — The Theoretical Foundation of Socratic Tests
• Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference — Responsible Assessment in the AI Era
• From authentic products to authenticated processes: a systematic conceptual review of authentic assessment in AI-rich higher education — From Authentic Products to Authenticated Processes
• Generative AI technologies and educational outcomes: a comprehensive meta-analysis comparing traditional and AI-driven approaches — Meta-analysis of generative AI educational outcomes
• When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle — When Help is Unhelpful: evaluating AI tutors for productive struggle
• ELBench: A multi-dimensional benchmark for education-facing large language models — ELBench: education LLM benchmark
• Large-scale AI grading of handwritten physics assessments: Score agreement and Olympiad team selection outcomes — AI grading of handwritten physics assessments (Olympiad)
• Distilling Black-Box Machine Learning into a Small, Self-Explaining Language Model for Learning Analytics — Distilling self-explaining LM for learning analytics
• Can EdTech close learning gaps? Global evidence from digital interventions — Meta-analytic evaluation of adaptive + AI EdTech
• A decade of reflection and thematic review on artificial intelligence's impact on educational measurement — AI's role across scoring, psychometrics, assessment
• Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomised Trials of an AI Tutoring Platform in GCSE Science — Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomized Trials of an AI Tutoring Platform in GCSE Science
• ProIQA: A Process-Based Framework for Fine-Grained Math Item Quality Assessment — ProIQA: Process-Based Math Item Quality Assessment
• Towards Scalable Measurement of Durable Skills — Toward Scalable Measurement of Durable Skills
• From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning — From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning
• StudentBench: AI and human tutoring yield equivalent GRE learning gains — StudentBench: AI and human tutoring yield equivalent GRE learning gains
• What fidelity metrics miss: A structural check on synthetic educational data — What Fidelity Metrics Miss: A Structural Check on Synthetic Educational Data
• Evaluation of pre-trained models for pedagogical assessment of novel AI-assisted educational questions — Evaluation of pre-trained models for pedagogical assessment of novel AI-assisted educational questions
• When Evaluators Cry Wolf: Lessons from Production LLM-as-Judge Evaluation in Educational AI — When Evaluators Cry Wolf: Lessons from Production LLM-as-Judge Evaluation in Educational AI
• Ethical and responsible use of artificial intelligence in medical education — A bibliometric map can expose what an evaluation culture rewards
• MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring — MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring (Lee et al. 2026)