Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education

Thomas, Borchers, Vanacore, Koedinger & Kizilcec (2026) — CMU & Cornell. Accepted as full paper at AIED 2026.

Core Argument

The AIED community over-relies on inter-rater reliability (IRR) — typically a single Cohen's κ coefficient — as a mechanical gatekeeper for "ground truth." This practice is insufficient and potentially misleading for the complex, noisy realities of educational data. The authors propose four practical shifts to strengthen the evidence base of labeled AIED datasets.

The Problem

Noise vs. Bias in Educational Labeling

Human judgment is subject to both noise (random variability) and bias (systematic directional error). While bias and fairness have received extensive attention, noise is an underexamined obstacle in AIED. Education is inherently noisy — assigning grades, defining engagement, and identifying giftedness all involve subjective interpretation.

Why κ Alone Fails

Educational settings present specific challenges that undermine threshold-based IRR heuristics:

High-inference constructs: Complex concepts like "engagement" or "understanding" require deep judgment beyond surface-level agreement

Skewed label distributions: Class imbalance inflates or deflates κ in misleading ways

Temporally segmented multimodal data: Video, audio, and log data over time create segmentation challenges where raters may agree on the label but disagree on when to apply it

The LLM Annotation Risk

The growing use of LLMs as annotators introduces new threats:

Automation bias: Over-trusting LLM output without verification

Circular validation: Using the same model to evaluate its own output

Silent degradation: LLM annotators may produce deceptively consistent but subtly wrong labels

The Four Shifts

1. IRR as Diagnostic, Not Gatekeeper

Stop treating κ > 0.8 as a binary stamp of approval.

Instead, use IRR to localize disagreement — identify where and why raters disagree, then refine constructs and codebooks accordingly. Disagreement is information, not failure.

2. Transparent Annotation Reporting

Require thorough documentation of:

Rater expertise and training: Who labeled the data and what qualifies them?

Codebook development: How were categories defined and iterated?

Reconciliation procedures: How were disagreements resolved — majority vote, discussion, expert arbitration?

Segmentation rules: For multimodal/continuous data, how was it divided into labelable units?

3. Mitigate LLM Annotation Risks

Bias audits on LLM annotators before deployment

Verification workflows with human-in-the-loop checks

Guard against automation bias and circular validation through independent evaluation

4. Complement Agreement with Validity Evidence

Go beyond agreement statistics with:

Uncertainty-aware labeling: Assign multiple labels to the same item to capture ambiguity and nuance

Criterion-related checks: Predictive tests to verify labels forecast intended educational outcomes

Close-the-loop evaluations: Test whether systems trained on these labels actually improve learning beyond a reasonable control condition

Case Studies

The paper illustrates these shifts through case studies of multimodal tutoring data, demonstrating how the four-shift framework applies to real AIED annotation challenges.

Connection to the Wiki

This paper is a methodological backbone for much of the research in this wiki. Many studies summarized here rely on labeled data; this paper provides the framework for evaluating whether those labels are trustworthy.

assessment-validity: The four shifts extend validity concerns from assessment instruments to the labeled datasets that train and evaluate AIED systems

ai-k12-evidence-base: Stanford SCALE found only 20/818 papers met causal inference standards — Thomas et al. address the complementary problem of measurement quality in the remaining papers

automated-grading: Automated grading systems depend on ground truth labels; the four shifts provide a quality framework for evaluating those labels

human-in-the-loop-ai: Shift 3 (LLM annotation risks) directly operationalizes HITL principles for AIED annotation workflows

educational-vlm-evaluation: Evaluation methodology for vision-language models in education connects to the multimodal segmentation challenges discussed

ai-literacy-assessment-misalignment: The disconnect between self-report and actual skills exemplifies why agreement alone (students agree with each other about their skills) doesn't guarantee validity

educational-llm-alignment: Hardy & Kim's finding that benchmarks misalign with teaching quality parallels the argument that IRR alone misaligns with ground truth validity

formative-assessment: Formative assessment systems that rely on AI-generated labels need the validity framework proposed here to ensure labels support learning

Practical Recommendations

1. Always report multiple IRR metrics (κ, α, percentage agreement) and discuss their limitations given the data characteristics 2. Make codebooks and annotation guidelines public whenever possible 3. Treat LLM annotations as hypotheses to verify, not as ground truth 4. Include at least one validity check beyond agreement in every labeled dataset paper 5. Design annotation workflows that surface ambiguity rather than forcing binary decisions

Open Questions

What is the "minimum viable validity evidence" for different AIED use cases (high-stakes assessment vs. formative practice)?

How should the four shifts be adapted for fully automated annotation pipelines with no human raters?

Can uncertainty-aware labeling be standardized across the field, or is it inherently task-specific?

llm-cognitive-diagnosis-handwritten-math — MathCog benchmark: 18 LLMs evaluated on cognitive skill diagnosis from handwritten math; all F1 < 0.5; systematic over-attribution and hallucination of evidence (2025)

rubric-aware-grading-rec-cbm — 2 of 8 papers in May 28 scan

reliable-programming-kt — Attention-dimension misconfiguration as reliability threat

self-referential-l2-writing-llm-assessment — Rasch-model calibration for more rigorous AIED measurement

short-answer-scoring-quality-degradation — Quality-conditioned agreement addresses scoring reliability concerns

ai-tutor-effectiveness-review — Systematic review whose conclusions depend on the quality of labeled evaluation data

ai-tutor-safety-harms — Safety taxonomies require reliable annotation of harm categories

automatic-short-answer-grading — Grading systems that depend on ground truth label quality

knowledge-tracing-irt — Interpretable knowledge tracing that requires reliable ground truth for training- aied-carbon-footprint-reporting — Parallel under-reporting of computational costs and ground truth methodology

human-vs-llm-ordered-coding

Citation

APA: Thomas, D. R., Borchers, C., Vanacore, K. P., Koedinger, K. R., & Kizilcec, R. F. (2026). Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education. arXiv:2603.29141. Accepted to AIED 2026.

📎 1 other page tagged ground-truth-reliability-aied

Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis