On this page

All issues

AIEd Research Weekly Update #5

Audio reading · 17 minAIEd Research Weekly Update #5

A weekly brief from the AI in Education Knowledge Base, on what the research added in the past week means for instructors, developers, researchers, and administrators.

Since the last issue, 143 new studies, eight new concept pages, and seven new resources have joined the knowledge base. It now holds 2,425 article summaries, 246 concept pages, 46 FAQs, and 60 resources. Search or use AI to chat with the knowledge base.

Contents

Main highlights and questions

  • Does the AI a teacher builds beat the chatbot handed to them from above? A pre-registered trial across 19 universities let 55 classes use their own instructor's AI tool a term early. On proctored exams AI could not help with, the early classes scored higher by about a third of a standard deviation. The tool was built by the teacher for their own course, which is the part to keep in view when a vendor offers the same chatbot to everyone.
  • Does AI help students think, or just get them through the work? A synthesis of 31 studies and 16,357 learners found the effect on critical thinking runs both ways at once. Students who use AI to avoid thinking lose ground, and students whose motivation rises gain it, so the two nearly cancel. A study that reads only the final grade sees nothing, which is why the argument keeps going in circles.
  • Does the assignment still measure what it claims to? Kearney sorts AI-era assessments on two questions: how much of the thinking the student keeps, and how much of their process the instructor can see. The same tool becomes an engagement amplifier or a cognitive substitute depending on where an assignment sits, so the countermeasure depends on where the assignment sits.
  • Is ease a sign that learning happened? Three experiments with UK university students found AI assistance made study feel fluent and lifted students' own metacognitive estimates of how much they had learned, without a matching gain on an independent test. A smooth explanation reads as understanding whether or not the tool did the work.
  • Can you write a question a chatbot cannot answer? Faculty at eleven universities wrote 61 questions meant to resist generative AI, and three frontier models answered them well anyway, with and without the course materials. Hardening the prompt is a partial answer; the evidence collected around the task carries the rest.
  • Do tutoring systems raise achievement? A review of 89 studies from 2015 to 2026 found the tutoring system ahead in 83 and the comparison ahead in none, but only 71 reported a significant advantage. The size of the advantage depended on what the system was compared against, so the honest claim is narrower than the count suggests.
  • Is the unease about AI really about learning? Benau argues that much of the discomfort in academia is a moral judgment about effort rather than a claim about learning. The Protestant work ethic ties effort, commitment, competence and value together, and AI pulls them apart. Less effort does not by itself make work less legitimate, and struggle is not a virtue either. What he settles on is ownership: whether the student can stand behind what was produced.

New concepts

Eight concept pages joined the knowledge base this week: four new stages, a music page, and three discipline umbrellas that came out of splitting a larger page into several.

Levels of education

The levels of education cluster now treats each stage separately. This week added Primary Education, Secondary Education, Undergraduate Education, and Graduate Education. They join K-12, Early Childhood Education, Higher Education, Adult Learners, Vocational Education and Training, and Special Education.

Primary education is mostly differentiated practice in early literacy and numeracy plus material for the teacher to draft. Secondary education is the band where AI use is already ordinary and the measured gains are smaller than the claims. Undergraduate study meets AI in large gateway courses with a shared rubric. Graduate work is where a tool starts competing with the artifact the degree certifies, the thesis chapter, the clinical read, the diagnostic encounter.

Arts and music

Music Education is new and joins Arts and Media Education, which was renamed this week after the architecture and interior studios moved to Design Education. The shared question is what a generative tool does to work whose value is in the making. In music the assessed artifact is sounded and then gone. In design the assessed object is the process behind an artifact as much as the artifact. In arts and media it is the image, the story, the performance. Each raises its own version of one problem: a tool that can produce the artifact can also skip the making that taught the student something.

The disciplines, split apart

The AIEd in the Disciplines overview was reorganized this week. Humanities Education is now by itself, covering history, philosophy, literature, languages, classics and religious studies, where a tool that produces a plausible reading instantly turns authorship and source judgment into the thing a course has to protect. Social Science Education is new and covers the disciplines that study people in aggregate, from civics and management to sociology and environmental studies. Psychology Education was split out of it, and it carries a warning worth repeating: psychology has the largest AI literature of any discipline in the base, and most of that literature studies psychology students as a convenient sample rather than the psychology curriculum. Professions Education rounds out the set with the certification argument, since accreditation, licensure examinations and supervised hours order these curricula, so AI's first effect lands on the evidence chain rather than on the content. Within the cluster, Teacher Education, Medical and Health Professions Education, Nursing Education, Legal Education and Libraries each work the argument out against their own gate.

New audio and video

The knowledge base now has two companion media tracks for twelve foundational concepts: a two-voice deep-dive podcast episode and a short single-narrator explainer video for each, with the related FAQs linked alongside. Both tracks were generated in the AIEd Gemini Notebook, the notebook built from these pages, with the scripts edited before publishing.

Every episode is on the audio page, where you can also subscribe to the podcast feed. The explainers are collected in the explainer-video playlist, and the episodes and the weekly issue readings are in a separate YouTube Music playlist.

Podcast episode Video explainer Concepts and FAQs
1. The Teacher as Curator: What AI Leaves for the Human in the Room Redefining Educator Roles Teaching
2. AI Literacy Has Two Halves, and Schools Teach the Easy One A New AI Literacy AI Literacy
3. Cognitive Offloading or Cognitive Surrender: The Line That Decides Whether AI Teaches Cognitive Offloading vs Cognitive Surrender Cognitive Offloading, Cognitive Surrender
4. The Textbook That Watches You: What AI Pedagogical Agents Change About Teaching AI as Pedagogical Partner Agentic AI; How can AI agents support students and instructors?
5. The Detector Cannot Know: Why Integrity Is Designed, Not Enforced Rethinking Academic Integrity Academic Integrity
6. Say Why, Not Just What: Framing AI Use for Students Rethinking AI in Education Framing AI Use for Students
7. Confident and Wrong: The Misconceptions That Make AI Feel Like Learning AI Myths in Education Misconceptions about AI; How can we address common misconceptions about AI in education?
8. The Performance Illusion: Reading AIEd Claims Before You Act on Them Applying AI in Education Research Interpreting and Applying AIEd Research, AI Literacy
9. Clapping for the Car, Not the Driver: The Limits of AI-Education Research Reality of AI in Ed Research Limitations in AIEd Research
10. Friction Is Your Friend: Designing Learning That AI Can't Short-Circuit AI Adapted Learning Designer Learning Design; How should AI be designed into the learning experience?
11. Design, Don't Detect: How to Actually Reduce AI Misuse AI Misuse to Collaboration Reducing AI Misuse; How can I reduce AI cheating in my course?, Should we use AI detectors?, How do I keep students from over-relying on AI?, How do I write a course AI policy?
12. Orchestrators, Not Sages: Faculty Development in the AI Era Educator's AI Evolution Educational Development; How do I design faculty development for AI that actually changes practice?

New resources

Seven resources joined the resources library.

  • Canvas MCP is an open-source server that lets Claude, Cursor, Codex and other agents read and act inside Canvas, with write actions gated and student data optionally anonymized.
  • Repertoire builds step-by-step AI assignments in which the instructor sets the model's role and the gates on student use, and student identity stays out of the model's context.
  • GradeBench is a public leaderboard that scores frontier models grading real handwritten university STEM answers against the rubric items the instructor chose.
  • RAID Benchmark leaderboard scores AI-text detectors on RAID, a benchmark built to test how they hold up against different models, domains and editing attacks.
  • Fab AI benchmark leaderboard ranks models on education benchmarks aimed at low- and middle-income countries, covering pedagogy, literacy, SEND, visual maths and multilingual work.
  • Integrating Generative AI in Psychology Courses is a 255-page open-access volume of 21 course-level accounts of generative AI in psychology teaching, from introductory sections to clinical training.
  • Vibe-Coded Classroom Examples is José Antonio Bowen's gallery of just over thirty classroom simulations, games and visualizations that educators built by prompting a model, sorted by discipline.

For instructors

Assess the process, not the product

Francis, Rutherford and Smith argue that generative AI broke the inference summative assessment rests on: that a good product was made by a student who did the learning. AI detection cannot restore it, since detectors are unreliable and inequitable, and permission rules say what use is allowed without saying how well it was done. Their alternative is PACE, Process-Anchored Competency Evaluation, five domains of process from critical sensemaking to metacognitive engagement crossed with four levels of human agency.

Write the intended learning outcome so it states the conditions of performance, with and without AI. Then grade the process the student can show, and let the product be one kind of evidence rather than the whole grade.

See related: Redesign assessment for the AI era

Dependence shows up in attention before it shows up in grades

Su and colleagues surveyed 1,461 Chinese adolescents in Grades 7, 8, 10 and 11 and modeled AI dependence against exam performance. Dependence went with weaker attentional control and lower performance, mostly through that route rather than a direct one, and the pattern was stronger in the higher grades.

Watch the study habits alongside the marks. A student whose work gets faster while their unaided attention gets shorter is the early signal, and a no-tool task now and then is how you see it.

See related: Keeping students from over-relying on AI

Make the grade depend on the understanding, not the artifact

Wood argues that the assessment problem created by generative AI is not detection but validity: a polished product is compatible with several different production histories, so a grade no longer shows what the learner can do or whether producing the work developed the understanding the qualification certifies. The proposal, learning-consequential assessment, lets students use AI to produce assessed work while making success depend on demonstrating that the understanding the work displays is their own.

Decide what the assessment exists to certify, then build a step that only that understanding can complete. A defended judgment at the end of a project does that work without banning the tool that helped produce it.

See related: Assessment validity

For developers

Teaching evaluation has models and very little validation

Xu and colleagues reviewed 60 empirical studies of AI in teaching evaluation and found 19 that reported validation evidence or demonstrated impact. Reliability and validity work is thin and inconsistently reported, bias is discussed more often than it is tested, and the split between what the human and the model do is usually asserted rather than designed.

If you ship a classroom tool that scores or advises, publish what it was validated against. Accuracy on one course's rubric says little about teaching in general.

See related: Checking whether an educational AI works

Tutoring that turns errors into the next step

Asher and colleagues design an LLM tutor around errors rather than around answers. The system waits for a wrong attempt, then works through the misconception behind it in steps, instead of supplying the correct method as soon as the student stalls.

Build the error path first. A tutor that only responds after a correct answer never addresses the misconception behind a wrong one.

See related: Training an AI tutor to guide rather than answer

One agent cannot scaffold every stage

Kozachek and Janson argue that a single general-purpose agent cannot scaffold well, because good scaffolding means giving a different kind of help at each stage, conceptual, procedural, strategic or metacognitive, and one agent answers every request the same way. Their prototype gives a lead orchestrator a specialist sub-agent for each kind of help, keeps a profile of what the learner has demonstrated, and grounds its help in verified course materials. In a concept-mapping task the order of the specialists predicted how much learners added, and the metacognitive agent going first produced the most new concepts.

Design the handoff, not just the prompt. Decide which kind of help a turn needs, and hand the specialist what the learner has already shown, or the same generic response arrives at every stage.

See related: Designing educational AI software

For researchers

What academics say they will do with AI

Sheridan and colleagues surveyed teaching academics in 20 disciplines across 24 countries about how generative AI would change their work and what they intended to do. Respondents described a split experience: the same tool might cut marking and preparation time and raise workload in the short term, erode human judgment, and produce material nobody had checked.

The plans are worth reading as a set of expectations to test. Time saved, trust in output and access to AI literacy are the variables to follow over the next year.

See related: Designing faculty development that changes practice

AI use spread, and the survey holds up two years later

Burke and colleagues replicated an earlier survey of AI use among college students and followed the same population two years on. The headline is that self-reported use is now close to universal across disciplines, and the replication checks whether the original measures still behave the same way as the practice changed.

The original measures still behaved the same way after two years of changed practice, which is the part a replication is for. A widely cited percentage from two years ago has usually moved.

See related: How AI is impacting students

A more accurate AI taught the judgment worse

Hegeman and colleagues ran a preregistered experiment with 300 novices judging whether images of white blood cells were cancerous, to test whether watching an AI's probability output teaches trust calibration, the skill of expressing certainty that matches accuracy. The same 94%-accurate model appeared three ways: its own conservative and poorly calibrated probabilities, a recalibrated and correspondingly certain version, and bare categories. Tested afterwards without the AI, only the miscalibrated display improved participants' calibration.

An AI that states its confidence well can leave the learner no work to do. When you test whether a tool teaches judgment, separate the tool's accuracy from what it shows the student, because a confident display may carry the skill away with it.

See related: Evaluating AI interventions: measures and methods

For administrators

Credentialing when the artifact can be generated

Meng proposes the term synthetic credentialism for what happens when the signals a degree certifies, the essay, the portfolio, the project, can be generated without the competence behind them. The framework traces how competence signals and certification come apart, and what institutions can still verify.

The question for a program is what evidence it can defend if every submitted artifact might be synthetic. Answer that before the next program review.

See related: Redesign assessment for the AI era

Governance that treats authentic assessment as the equity lever

Jiang and colleagues connect digital governance of generative AI in higher education to authentic assessment and student engagement. The argument is that a governance policy which only restricts tools leaves the assessment problem in place, and that redesigning assessment is also the route to equity.

Put one line of the AI policy on assessment design. A rulebook about permitted tools does not fix an assignment that a model can finish.

See related: Writing and implementing an institutional AI policy

Compliance needs a stage, not a principle

Isaifan and Hasna argue that AI governance in higher education has produced a shelf of principles and almost no procedure for showing compliance with any of them. Their answer puts the question where quality claims already become auditable, the accreditation cycle: ten stages, each with a permission level for AI, Safe, Limited, Restricted or Not Permitted, and compliance findings, ratings and the final accreditation outcome reserved for humans.

Write the permission level into the stage where the evidence is already collected. A principle that nobody has to demonstrate at a review is a sentence, not a control.

See related: Options and issues when adopting AI tools

Also added this week

The journal page lists every study added this week.

Site notes

The concept work was the week's main editorial effort. The levels of education cluster split into separate stage pages because the evidence differs by stage. The arts page was renamed when design moved out to its own page, and the disciplines were reorganized so that humanities, social science, psychology and the professions each stand alone.

The site also gained the media tracks. Every episode page now carries a note crediting Gemini Notebook, and the audio page's opening paragraph says the episodes and explainers were generated in the same notebook. A concept or FAQ page that lists a podcast now embeds the player, so an episode can be heard without leaving the page.

The commit history records every update.

Follow the update

Get each issue through the newsletter feed in a feed reader, by pasting that same feed into Blogtrottr to have new issues emailed to you on a schedule you choose, or by following it on LinkedIn, where each issue is published as a LinkedIn newsletter.


AIEd Research Weekly Update comes from the AI in Education Knowledge Base, which an AI agent generates. This issue was drafted by that same AI agent, with editorial guidance from the site developer, and every claim above links to the page it came from.

The knowledge base is dedicated to the public domain under a CC0 1.0 license. No rights reserved.

Found an issue with the content? Report it on GitHub, or contact the site developer, Doug Holton.

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.