How Does AI Bias and Unfairness Show Up in Education, and What Can We Do About It?

Bias and unfairness in educational AI are not one problem. The research in this knowledge base documents bias entering at nearly every stage of a system's life — training data, model behavior, prompts, scoring, feedback, allocation decisions, and the infrastructure beneath them — and it affects instructors, developers, researchers, and administrators differently. The pattern across all of it is consistent enough to state plainly: ask for subgroup numbers, not averages; define "disadvantaged" by outcomes, not by group size; and treat fairness as something you design and audit, not something you bolt on afterward.

Where bias and unfairness show up

Scoring and assessment. AI scoring can penalize the language a student writes in rather than the understanding it expresses: physics scoring systematically underestimates students whose explanations are less fluent, punishing non-native and less confident writers for what they know (language bias in automated scoring). Automated marking also compresses toward the middle of the distribution, exposing the strongest and weakest students the most, and its disagreement with human markers varies with attainment level and with surface language features like vocabulary range (OpRaise, 2026). For the full grading picture, see the grading-fairness FAQ.

Tutoring, feedback, and adaptive systems. An adaptive tutor's internal model can disadvantage whole groups invisibly: a Bayesian knowledge tracing fairness audit of MATHia's logs from 8,549 students over two school years found aggregate statistical parity of −0.077 that concealed 36 of 50 skills biased against emerging readers, with skill-level bias as large as −0.262 — the tutor had no reading-skill component, so students who read less fluently lost ground in math (Stinar et al., 2026). Feedback tools have their own documented skew: four widely used LLMs shift writing feedback in stereotype-aligned ways when fed a student's identity — more praise, less critique, assumptions of limited ability — even for identical essays (Tan et al., 2026). LLM tutors also vary with the learner's demographic label: in EduFair-Bench, corrective tone excursions exceeded 0.10 for every model in chemistry and physics, with step scaffolding correlating up to |r| = 0.144 with demographic conditions in math, and the tone tracked the label rather than the quality of reasoning (EduFair-Bench). Even the demographic context invisibly present in a conversation history shifts outputs: implicit conditioning inflated automated essay scores by up to 1.57 points for one model because the context, not a stated attribute, carried the signal (Rooein, Benedetto and Hovy, 2026).

Dashboards and early warning. McConvey et al. (2026) stress-tested six post-hoc fairness interventions on a replicated early-warning system built from 168,550 student records: none delivered consistent gains. Disparities were redistributed, not reduced; two methods used group size to define disadvantage and directed all 706 corrections at international students, who already had the higher predicted-success rate (0.916 vs 0.756); and one method satisfied its gender equal-opportunity target by driving the female false positive rate from 0.561 to 0.837. Students collapsed into "Unknown Gender" had the lowest true positive rates under every method.

Generative output: text, images, and avatars. Models reproduce dominant norms unless explicitly checked. A systematic review of 31 studies of text-to-image tools in education found images consistently centered white, male, Western, thin, and non-disabled figures, with age, body, and ability diversity largely absent (Alon et al., 2026). Team-formation assistants route students into different experiences: three LLMs placing 11,200 software-engineering assignments sent men at least 80% less often than women to Interface Design rather than Core Development, and nationality shifted placements independently of merit (Entezami et al., 2026). Even an avatar's identity cues change trust: White avatars were rated more credible and competent across two experiments (N = 396), while older Black female avatars were penalized, with the effect amplified in STEM tasks (Anthis and Kyriakidou-Zacharoudiou, 2026). Writing tools can transfer bias into students' own work: a gender-biased prompt produces more gender-stereotypic essays, so the tool's bias becomes the learner's output (gender bias transfer in LLM-assisted writing).

Data, labels, and infrastructure. Bias is present before any model trains. Off-the-shelf name-detection tools missed African names at recall 0.7647 and Asian names at 0.8640 against 0.9024 for European and 0.9091 for American names, gaps a fine-tuned model eliminated (Ji et al., 2025). Training corpora and tokenization structurally disadvantage speakers of underrepresented languages (Structural Silence), LLM-generated learner profiles carry regional and gendered priors that propagate downstream (López-Pernas et al., 2026), and the ground-truth labels behind models deserve the same scrutiny as the models themselves (ground-truth reliability).

Detection tools hit multilingual writers hardest. Because detectors classify on textual predictability, the low perplexity and formulaic density of additional-language writing trigger false "AI" flags: documented studies misclassified large shares of authentic TOEFL essays by non-native speakers (Angelier, 2026), and editing a non-native paper shifted verdicts both ways — 10.7 points less likely to be flagged under one detector, 5.8 points more likely under another (Park, Jeong and Kim, 2026).

Access and tool quality are equity issues too. The digital divide is foundational (digital access), and quality divides compound it: the same model's free tier was inaccurate in 49 of 150 responses (32.7%) against 18 of 150 (12%) for the premium tier, leading teachers to call it "borderline useless for mathematics" (Canonigo, 2026). Faculty overwhelmingly expect generative AI to widen inequities — 81% of 1,057 surveyed faculty said so, 58% saying a lot (Watson and Rainie, 2026). And the benefits of AI tutoring are not automatically distributed: gains concentrate where allocation follows need, as when proactive human tutoring aimed at students farthest below a cutoff produced marginally higher growth (75%, p = .065) in exactly that group (Gurung et al., 2026), or when expert-verified AI study materials moved the largest share of a 2.34-mark gain to the bottom quintile (Dang and Nguyen, 2026).

What to do about it — by your role

If you are an instructor: ask any tool you adopt what its error rates were for the subgroups you actually teach, not just its overall accuracy. Check multilingual writers' work carefully before accepting a detector verdict — those tools misclassify additional-language writing. Use neutral prompts when you generate teaching materials (a neutral prompt largely avoids inducing gender-differentiated language), and compare AI output against your own standards rather than treating it as authoritative (How Do I Teach Students to Verify AI Output?). For grading, keep a human in the loop on borderline cases.

If you are a software developer: audit across identity conditions — language, gender, culture, disability — using inputs that vary only the student's context (the implicit-signal result shows stated attributes are not the only channel). Test on held-out subjects, not just held-out items: fairness gains that hold on a validation split evaporated for a real student-attention model in 4 of 10 runs (Fragkiadakis et al., 2026). Prefer mitigation in the model — fine-tuning, dialect-invariant training, fairness-aware objectives like GroupDRO — over post-hoc output adjustments (bias-mitigation approaches). Never infer disadvantage from group size, and report subgroup metrics including the groups you had to exclude because the sample was too small (Ye et al., 2026 excluded English learners and IEP students for exactly this reason — say so when you do).

If you are a researcher: report a fairness metric, period — a scoping review of 421 studies applying NLP to student evaluations found a formal fairness metric in only 8 (1.9%) (Eicher and da Silva, 2026), and of 12 AI-versus-human scoring studies, ten were silent on demographic, linguistic, or social bias (Xu, Lu and Abdi, 2026). Test across demographic dimensions, not just gender; report small-group exclusions; and evaluate equity alongside accuracy rather than as an afterthought. When you audit models as history tutors or tutors in different contexts, use identity-condition testing before drawing conclusions (the paternalistic filter is the cautionary example).

If you are an administrator: make subgroup reporting and audit-before-adoption contract terms when you procure predictive or generative tools — the early-warning study's authors frame procurement governance, not metric optimization, as the primary safeguard (McConvey et al., 2026). Watch for quality-tier divides that leave under-resourced classrooms with materially worse tool performance, and pair access with support and training rather than issuing licenses alone. Use the levers in institutional AI policy to require disclosure and equity reporting, and route the R&D and oversight discussions through the equity, ethics and safety research conventions.

What does not work

Overall accuracy claims. Aggregate parity of −0.077 coexisted with 36 of 50 skills biased against a group; a balanced-looking model missed struggling students almost entirely on an imbalanced split (Hooshyar et al., 2026 found 8%, 12%, and 16% recall for low performers at 80% precision). Averages are compatible with concentrated harm.

Post-hoc parity fixes. Every intervention in the early-warning study either redistributed errors, corrected the already-advantaged, or achieved parity by making a group less visible. Fixing the pipeline beats patching the output.

One mitigation done once. Bias re-enters through data, prompts, scoring, and deployment; the research shows complementary strategies (fair training objectives, prompt design, dialect invariance, scoring that separates understanding from language, explainability, human oversight) precisely because no single fix covers the pipeline (Bias Mitigation).

Where to go deeper

The Bias Mitigation page catalogues sources and mitigation approaches across the pipeline; Equity covers the access, representation, and outcome dimensions with evidence for each; Automation Bias covers the reader-side failure — over-trusting a biased or merely wrong recommendation; and Differential Effects Across Learner Groups collects the studies that trace how the same tool lands differently on different groups.