AIEd Knowledge BaseAIEd KB
On this page

All issues

AIEd Research Weekly Update #4

A weekly brief from the AI in Education Knowledge Base, on what the research added in the past week means for instructors, developers, researchers, and administrators.

Since the last issue, 199 new studies, eight new concept pages, two new FAQs, and thirteen new resources have joined the knowledge base. It now holds 2,277 article summaries, 238 concept pages, 46 FAQs, and 53 resources. Search or use AI to chat with the knowledge base.

Contents

Main highlights and questions

  • Can you trust a marking average? Two ChatGPT versions marked the same essays and pushed every paper toward the middle: one the lecturer failed at 50 came back at 66, one at 80 came back at 67, so the weakest and strongest students ended up with the same mark. Use AI comments to help a student revise, and keep the recorded grade with a person.
  • What happens when the assignment changes but the grade does not? Universities swapped essays for presentations and oral defenses, then kept the same single grade on the transcript. A student can defend their work out loud and still leave with one number that says nothing about how they thought. Change what the transcript reports, or the reform only changes the paperwork.
  • Does a chatbot help students understand a text better than writing notes? Students who wrote notes learned more than students who asked a chatbot about the same passage, and the students with the chatbot often copied its answers into their notes. They still rated the chatbot easier, more helpful and more enjoyable. Keep the writing task, because a tool that summarizes the reading is doing the work that produces the learning.
  • Do we grade the essay or the thinking? The essay was designed to show a mind changing as it writes, and grading only the finished text threw that away. Ask for a first draft, a critique of it, and a revision that says what changed and why. A chatbot can produce the last draft. It cannot show the work of getting there.
  • Can AI mark handwritten exams? Claude marked handwritten exams almost exactly like the lecturer on multiple choice and short answers, and got hand-drawn diagrams right about half the time. Let it pre-mark recall questions, and mark the drawings and the judgment calls yourself.
  • Can a chatbot run an oral exam? A chatbot can interview a student without ever seeing the rubric while a second, hidden part of the system grades the conversation, so students cannot prompt their way to marks. Their system marked too generously when it scored each answer as it went. Grade the whole conversation at the end.

The Stanford evidence review of AI in K-12 was updated to the November 2026 edition this week, and the causal base doubled to 52 studies. Using AI to explain and guide predicts better learning, using it to produce the work predicts worse, and one trial found a general assistant lowered student motivation.

New FAQs

Two answers joined the FAQ this week. Both are about decisions that arrive before anyone picks a tool.

What are the options and issues to consider when adopting AI tools?

The adoption answer separates three decisions that a vendor list blends together: what data goes into the tool, whether the institution permits it, and who runs the model. A free chatbot keeps the conversation on the vendor's servers. A campus gateway keeps it on yours and can cut access. Start with the data question, because it is the one you cannot undo later.

What should educators avoid doing with AI?

The avoidance answer names three mistakes. Do not use AI to do the thinking an assignment exists to teach. Do not treat clicks, time on page or completion as evidence of learning. And do not write a rule saying "use AI responsibly" while leaving an assignment that a chatbot can finish in a minute. That last one is the common failure: a policy sentence does not change what students can do. The task and the evidence you collect do.

New concepts

Eight concept pages joined the knowledge base this week.

  • Shadow AI is AI use that happens before anyone approves it. A lecturer drafts feedback in a personal ChatGPT account and the institution never learns that student work passed through it.
  • Institutional AI infrastructure is the plumbing behind AI use on campus: who gets access, which model runs, what gets logged, and who pays. One example is a university gateway that gives each department a quota and keeps an audit log.
  • Executive function covers planning, holding information in mind, and stopping an impulse. Think of a student who can plan an essay but loses the thread halfway through. A tool can support that planning, or do it for them without anyone noticing.
  • Developmental psychology is the study of how children's thinking changes with age. A seven-year-old may need the tool to read the question aloud. A seventeen-year-old can judge whether the answer is any good.
  • Longitudinal research follows the same students over months or years. The question is whether the writing gain from October is still there in May, not whether it showed up on the unit test.
  • Empathy separates understanding what someone feels from feeling it with them. A chatbot can rewrite a harsh comment kindly. It cannot notice that the student has stopped trying.
  • Case-based learning teaches through one specific situation students have to reason about. Picture a patient with chest pain and two test results that disagree. The learning is in the questions students ask about the case, not in the case text.
  • Metaphors and analogies are the pictures we use to explain AI, such as black box, helper or mirror. Call it a tutor and students expect it to know the answer. Call it a study partner and they are more likely to check it.

New resources

Thirteen resources joined the resources library this week.

  • Primer lets a teacher describe a role-play or simulation in plain language instead of writing code, for example a negotiation exercise where the AI plays a difficult parent.
  • Agentic Builds is a gallery of small apps teachers built, with the prompts behind them that you can copy for your own class.
  • Vibe-Coding Center gives you three instruction files to paste into a coding assistant, so the web app it writes is accessible, private and secure enough for students.
  • AI Guidance by Age takes a role and an age band and suggests what students should be able to do with AI at that age, and says which parts are still guesswork.
  • Voice-First Written Assessment has students explain their answer out loud, then write a revision from the transcript, so the thinking is on record and the writing still happens.
  • Amplify GenAI is Vanderbilt's chat platform, which a campus runs inside its own cloud account so student conversations stay with the institution.
  • conText is a voice tutor that asks the next question instead of giving the answer, and shows the teacher the path the student took.
  • MindRouter is a gatekeeper in front of campus GPUs, with quotas by department, single sign-on and a log of who used what.
  • Moodle AI Text Question Type adds an AI-marked question to Moodle quizzes, with your rubric written into the prompt and marks going straight into the gradebook.
  • MyEssayFeedback has students submit a draft, read AI comments on the parts you choose, then record which comments they accept before revising.
  • NowComment lets a class discuss a document line by line, and now an AI can add a draft comment that students are expected to edit.
  • Teacher Hive collects small teacher-made apps that need no student logins, published as examples other teachers can copy.
  • AI Tutor Pro and AI Teaching Assistant Pro are free study tools with voice conversation, quizzes and note taking in eight languages, for students practicing on their own.

One naming change: the EduPrompts resource was called EduGems until this week. The prompt collection is the same, and it now lives at eduprompts.ai.

For instructors

Stop asking whether the work is the student's and ask what it shows

Bayraktar argues that detection tools, bans and in-class rewrites answer only one question: whether a product was produced independently. They say nothing about whether the student understands the thing the assignment was meant to evidence. The proposed framework keeps the finished work as one source among several and adds evidence where the construct needs it: a decision log, a short note on which advice was rejected, a conference response. It also makes AI use itself something students judge, for example by naming where the tool's answer was wrong.

Decide what learning the assignment should evidence before choosing a countermeasure. Add one artifact that closes the gap rather than turning every course into a portfolio course, and ask students where the AI was unhelpful. A paragraph on that is worth more than a disclosure statement.

See related: Redesigning assessment for the AI era

Keep the assignment, add a paper trail

Winstone, Gravett and Elkington argue that most university responses to GenAI are talk: traffic lights, declarations, permitted-use rules. Their Black Box Assessment keeps the task and asks students to submit windows into the process, such as drafts, revision logs and short reflections, with rubrics rewritten so that revision earns credit. They report no pilot, and say so.

Take one assignment and require one process artifact: the outline, the first draft, or a paragraph about what feedback changed. Grade that artifact against a stated criterion. Students then have a reason to keep the record, and academic integrity lives in the assignment instead of a policy sentence.

See related: Redesign assessment for the AI era

Teacher feedback and AI feedback came out even

Meyer and colleagues randomized 391 students in grades 10 and 11 in Germany to three conditions: feedback written by their teacher, immediate AI feedback, or AI feedback delivered late so that source and timing could be separated. Every standard comparison came out null. Revisions improved, the next writing task did not differ, and time on task, motivation and affect were statistically indistinguishable between teacher and AI feedback.

Students rated immediate AI feedback as more useful and more motivating than teacher feedback, but the delayed AI group did not get the same ratings, so the advantage came from speed rather than authorship. Use AI feedback when a teacher cannot respond quickly, and expect it to hold the floor rather than raise it. Check whether students are mistaking a fast comment for a useful one.

See related: Redesigning assessment for the AI era

For developers

A tutor that folds when the student says "my notes say otherwise"

Liu and Korolova ran eleven commercial models through two simulated K-12 math tutoring experiments: 400 questions over a five-turn answer-revision protocol and 1,884 over a two-turn diagnosis protocol. If the student opened with a wrong answer, first-turn accuracy dropped 5.5 points. Among answers that were correct at the first turn, a bare "I disagree" flipped 5.2% to the student's wrong choice, "my notes say B" flipped 12.7%, and explicit wrong reasoning flipped 3.1%. Of the correct conversations that ended on the wrong answer, 86% flipped at the first pushback. Doubting correct work also cut diagnostic accuracy from 66.5% to 58.0% and raised false positives from 33.5% to 42.0%.

Treat an appeal to authority as a request to re-derive the step, not to answer again, and never finalize a misconception diagnosis on work the student has just doubted. Test tutors on a scripted multi-turn dialogue and report flip rate separately from first-turn accuracy, because two models with the same first-turn score differed by 7.5 points in how often they folded.

See related: Training AI tutors to guide rather than answer

The rubric is the measuring instrument, not the model

Roy and colleagues show that the same judge paired with two rubrics produces two different measurements, and that a vague rubric asking for answers that are "helpful and factual" can reward a polished reply that is wrong. Across four public preference datasets, six rubric sources and three open-weight judges, no rubric source led on every axis: inter-judge agreement, preference fit and resistance to gaming. Reordering criteria by preference data raised held-out accuracy from 65.0% to 68.6%, and rewriting criteria to close loopholes cut exploit success from 46.4% to 36.0%.

Treat the rubric as the instrument and version it with the model. Record the rubric text, date and judge configuration beside the marks, and test exploitability separately from agreement: write a few answers that satisfy the wording while violating the intent, and see whether they score high.

See related: Checking whether educational AI works

The reading tutor helped when students used it enough

One charter school gave every first- and second-grade teacher a phonics and fluency tutor and left use to teacher discretion. Students who used it a lot gained more than students who used it a little, by g = 0.56 on oral reading fluency. The comparison between high use and no use missed the corrected significance threshold. The difference between classrooms came down to logged minutes.

Log minutes per student from day one and report them beside the outcome. If the log is empty, that is a finding about the rollout, not about the tutor. A district that counts licenses instead of use will see a flat result and learn nothing.

See related: Checking whether educational AI works

Copilot made novices faster and left them where they started

Twenty-two novice and intermediate programmers did two 20-minute coding tasks, once with a human partner and once with Copilot, then repeated the tasks alone about a week later. With Copilot they scored 14 points higher out of 100 and reported much lower mental demand and effort. A week later the advantage was gone: retest scores were 4.8 points lower, not statistically different, and each person's drop from the first session was about 19 points larger after the AI session.

Add an unaided retest to any coding assistant you build or buy. The gain you see in class may be rented, and it is largest for the students who already knew the most. Pairing with a person produced lower scores and higher workload in this study, but also better engagement and a trend toward retention.

See related: Designing AI into learning

For researchers

A stricter meta-analysis cuts the tutoring effect in half

Leite and colleagues pooled 18 experiments, 77 effect sizes and 11 intelligent tutoring systems used in US K-12 schools. The average effect on achievement was g = 0.271, well below the effects earlier meta-analyses reported, because this one kept to experimental designs in one national system. Setting mattered: rural schools showed g = 0.146 against 0.298 in urban schools. Who controls the support mattered too, with worked examples at 0.324 and on-demand hints at 0.316, and duration peaked at two weeks to three months (0.449) before falling to a non-significant 0.110 beyond six months.

Treat older tutoring effect sizes as optimistic until you check the study design and the setting. Report moderators rather than a pooled average, and expect a rural deployment with weak infrastructure to deliver roughly half the urban effect. The duration curve is the finding to test next: almost no evidence covers a full year.

See related: Checking whether educational AI works

GenAI helps language learners most when it gives feedback, not instruction

Zhang, Aubrey, Wu and Chiu pooled 67 effect sizes from 48 controlled studies with 5,196 learners. The unadjusted effect was large, g = 0.76, and fell to g = 0.57 when three outliers were removed. The role the tool played explained more than the average: feedback and assessment reached g = 1.15, an interaction partner 0.81, and an instructional integrator 0.28. Learner-directed use beat AI-led use, 0.92 against 0.34. Randomized trials reported larger effects than quasi-experiments, 1.02 against 0.48.

Ask what role the tool plays before asking whether it works. If you are pooling studies, model the design type as a moderator, because it moves the estimate as much as the intervention does. For a language course, feedback and practice with the student in charge is the configuration the evidence favors.

See related: Checking whether educational AI works

What AI literacy means depends on the student's age

Guo, Ali and Wong read 118 K-12 studies and sorted the field into nine categories and 35 subcategories. Their PACES framework stages AI literacy across four school bands, on the argument that what a second grader needs to know differs in kind from what a senior needs. Their audit of validated measures found almost everything aimed at secondary school, and nothing that measures the process of working with AI.

Build an instrument that captures what a student does with a model: whether they ask a follow-up question, check a claim, or revise after feedback. Then test whether it behaves differently across the four bands. Until someone does, the developmental claim stays a design principle with no evidence behind it.

See related: AI literacy evidence

For administrators

Nearly everyone uses AI at work and half have not seen the policy

EDUCAUSE surveyed 1,960 higher education employees in 2026. 94% had used AI for work in the past six months, and 73% of those used it daily or weekly. Only 54% knew of the policy meant to guide that use, including just 38% of executives. 56% used tools their institution does not provide. 92% of institutions said they have an AI strategy, and 13% measure its return.

Publish the policy where people already work, and count the tools people actually use before writing the next strategy. A gap between what staff do and what leaders think they permit is a governance problem, not a training problem, and it is invisible in a strategy document.

See related: Adopting AI tools

Teachers weigh the time saved against the work they lose

Dangol and colleagues interviewed 22 teachers in a large US district that had encouraged GenAI since late 2023, most of them working without a district-wide classroom policy. Teachers were glad to lose paperwork and lesson planning. They worried about losing the parts of the job that require knowing a student: writing the comment that lands, or deciding what a class needs next. The paper describes a trade, not a resistance.

Name what the relief is for and where it stops. A district that automates marking or lesson planning should say which work it is removing, and protect the work teachers say they will not trade. Leave room for professional judgment instead of adding another mandated tool.

See related: Adopting AI tools

The unglamorous systems change institutions most

Pardos and Borchers argue that credit transfer, advising, curriculum classification and assessment production matter more than the chatbot on the front desk. Their common thread is representation. Turn courses, syllabi, transcripts and learning records into embeddings, and you can make decisions that used to be too expensive to attempt, such as evaluating a transcript from another country course by course. The same step makes errors harder to see.

Before buying, name the decision the system changes and the person accountable when it gets that decision wrong. An audit trail and an appeal route matter more than a better model, because a back-office error reaches students through institutional infrastructure nobody can see.

See related: Checking whether educational AI works

Countries add AI literacy to the curriculum and do not measure it

The iRAISE report draws on a structured review and policy documents from France, Canada, Egypt and Qatar. Governments split between scaling tools and restricting them, and the report treats AI literacy as the newest layer of digital and media literacy rather than a separate subject. Most countries name media literacy in a framework. Few put it into the curriculum teachers plan from, and almost none measure whether students gain it.

Put AI literacy in the documents teachers actually use, and fund an assessment that can show whether students improved. A national framework with no curriculum line and no measure leaves the work to individual schools, which is where the gap already sits.

See related: Incorporating AI literacy

Also added this week

The journal page lists every study added this week.

Site notes

The site has widened its scope this week. It now carries some AI research that is not about schools or universities but still bears on both: Salvi and colleagues on how a conversational model can shift what a person believes, Folk and Dunn on chatbots and loneliness, and Sim and colleagues on how AI summaries distort memory. They enter as citations on the concept pages they inform, not as claims about teaching.

The Hermes AI agent that maintains this site switched to a new memory system this week. It now runs on Hindsight, which keeps what it learns between sessions instead of starting each one from nothing.

Almost every concept page and FAQ was rewritten this week to remove self-referential text, phrases like "in this knowledge base" or "this page argues". The Hermes AI agent also put that prose through its humanizer skill, which strips tells like stacked qualifiers and sentences that repeat the heading. Article and concept pages now follow one structure, with the same sections in the same order, and a build check fails when a page breaks it.

Some new features are in planning, such as educational task-based guidance and multimedia summaries of the main pages. Suggestions and feature requests are welcome through the GitHub issue form.

Follow the update

Get each issue through the newsletter feed in a feed reader, by pasting that same feed into Blogtrottr to have new issues emailed to you on a schedule you choose, or by following it on LinkedIn, where each issue is published as a LinkedIn newsletter.

AIEd Research Weekly Update comes from the AI in Education Knowledge Base, which an AI agent generates. This issue was drafted by that same AI agent, with editorial guidance from the site developer, and every claim above links to the page it came from.

The knowledge base is dedicated to the public domain under a CC0 1.0 license. No rights reserved.

Found an issue with the content? Report it on GitHub, or contact the site developer, Doug Holton.

SourceHistoryEditChat

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.