Explainable Artificial Intelligence in Education (XAI-ED)

Synthesis: This paper introduces XAI-ED, a framework for explainable AI that is purpose-built for education. It argues that while XAI in education shares common ground with broader XAI (accountability, fairness, privacy), education has distinctive needs: learning data is noisy at many levels, explanations can directly support metacognition and self-regulated learning, and different stakeholders need fundamentally different kinds of explanations.

The XAI-ED Framework — Six Aspects

1. Stakeholders

XAI in education must serve multiple audiences — learners, teachers, administrators, parents, researchers, developers, and policymakers — each with different explainability requirements. A learner needs to understand why a hint was given; a teacher needs to understand which students are at risk and why; a developer needs to debug model behavior. This connects to Teaching and Student Experience research on multi-stakeholder design.

2. Benefits

Explanations serve multiple purposes in educational contexts:

• Trust — building confidence in AI-driven recommendations and assessments
• Fairness — detecting and mitigating algorithmic Bias Mitigation
• Debugging — helping developers and researchers identify model flaws
• Usability — making AI interfaces more transparent and actionable
• Learning — explanations as pedagogical tools that support Metacognition
• Regulatory compliance — GDPR right to explanation and similar mandates (Privacy, AI Regulation in Education)
• Adoption — reducing resistance to AI tools among educators

3. Approaches for Presenting Explanations

The paper catalogs multiple explanation modalities: visual (heatmaps, decision trees), textual (natural language), example-based (counterfactuals, nearest neighbors), feature importance rankings, rule extraction, and model simplification. The key insight is that the optimal approach depends on the stakeholder and the pedagogical context — an insight directly relevant to Scaffolding design and Intelligent Tutoring interface research.

4. Classes of AI Models

The framework maps explanation approaches to model types:

• White-box (decision trees, linear models, rule-based systems) — inherently interpretable
• Black-box (neural networks, ensemble methods) — require post-hoc explanation methods
• Glass-box — newer approaches that balance accuracy with transparency

This taxonomy connects to the Adaptive Learning literature's ongoing tension between model complexity and interpretability.

5. Human-Centered Design

Explanations are not purely technical artifacts — they are communication acts. The interface must be designed for the specific stakeholder's cognitive needs, not just optimized for technical accuracy. This aligns with AI Literacy research showing that technical transparency without pedagogical framing often fails to support actual understanding.

6. Potential Pitfalls

The framework identifies risks specific to educational XAI:

• Explanation overload — too much information overwhelms users, undermining the benefit
• Misleading explanations — post-hoc explanations may not reflect actual model reasoning
• Confirmation bias — users may selectively attend to explanations that confirm existing beliefs
• Over-trust — explanations can create false confidence in flawed systems, connecting to Over-Reliance
• Gaming the system — students may exploit explanations to circumvent learning, a known risk in Intelligent Tutoring

Four Case Studies

OnTask — Instructor-Facing Learning Analytics

A learning analytics platform that provides instructors with interpretable student risk indicators. XAI-ED applied to surface the rules and features driving each risk flag, enabling instructors to make informed interventions. Demonstrates the Teaching shift toward data-informed decision-making.

OATutor — Adaptive Tutoring System

An Open Source ITS where XAI explains mastery predictions and hint selections to both students and instructors. Connects to Intelligent Tutoring and Adaptive Learning research on transparency in automated instruction.

SRES — Student-Facing Recommender

A learning resource recommender that explains why specific resources are suggested based on the learner's knowledge state and goals. Supports Self-Regulated Learning by making the recommendation logic visible and actionable.

CLA (Connected Learning Analytics) — Multimodal Collaboration

Analytics for Collaborative Learning environments that surface group dynamics and individual contributions. XAI helps students and instructors understand team interaction patterns — connecting to Collaborative Learning and Learning Analytics research.

Significance for AIED

The XAI-ED framework has become a foundational reference in AIED, cited across the The Evidence Base on AI in K-12: A 2026 Review, SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems, and AI Literacy literatures. It bridges the gap between technical XAI research (which often ignores pedagogical context) and educational practice (which often treats AI as a black box). The framework's emphasis on distinctive educational needs anticipates later work on Pedagogical Safety and Human-in-the-Loop design.

What this means for practice

• Instructors. Ask who the explanation is for before asking whether it exists: XAI-ED treats learners, teachers, administrators, parents, researchers, developers and policymakers as stakeholder groups whose explainability requirements differ, so a learner's "why this hint" is not a teacher's "who is at risk."
• Instructors. Use explanations as teaching moves rather than as transparency features: the framework lists learning among the benefits of XAI and positions explanations as support for Metacognition and Self-Regulated Learning.
• Instructors. Pair each explanation with a guardrail against its failure modes — explanation overload, misleading post-hoc rationales, confirmation bias, and over-trust — because the framework names these as risks specific to providing explanations in education.
• Instructors. Match the presentation approach to the model class and the audience instead of defaulting to one format: white-box models can be surfaced directly, while black-box models need post-hoc methods such as counterfactuals, feature importance rankings, or rule extraction.
• Instructors. Plan the compliance pathway before deployment: the GDPR right to explanation and similar mandates are among the framework's listed benefits of explainability in tutoring and learning analytics systems.

Limitations

• This is a conceptual paper, not an empirical study: it proposes six aspects of explainability and illustrates them with four case studies (OnTask, OATutor, SRES, and Connected Learning Analytics) but reports no measurement of whether any explanation improved learning, trust, or decision quality.
• The framework is a synthesis of existing XAI literature and educational tool practice, so its six aspects function as organizing categories rather than validated causal claims.
• The four case studies are drawn from pre-existing educational AI systems and show fit to the framework, not the framework's effectiveness; the paper reports no comparative data across them.
• The pitfalls it names — explanation overload, misleading explanations, confirmation bias, over-trust, and gaming the system — are each described conceptually with no reported incidence or threshold at which they appear, and the mapping of explanation approaches to white-box, black-box, and glass-box model classes is argued rather than tested.

Citation

Khosravi, H., Buckingham Shum, S., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable Artificial Intelligence in education. Computers and Education: Artificial Intelligence, 100074