AIEd Knowledge BaseAIEd KB
On this page

All issues

AIEd Research Weekly Update #2

A weekly brief from the AI in Education Knowledge Base, on what the research published in the past week means for instructors, developers, researchers, and administrators.

Since the last issue, 121 new studies, four new resources, five new FAQs and two new concept pages, on open education and statistics and data science education, have joined the knowledge base, which now holds 1,926 article summaries, 222 concept pages, 42 FAQs, and 37 resources. The journal page lists every study added in the past week. You can also search the knowledge base or use AI to chat with it.

Contents

Main highlights and questions

Most of this week's results are about the work that happens around an answer, and what to change when the answer arrives too fast or too confidently. Two experiments isolated the same mechanism in different rooms.

A preregistered trial with 317 adults gave the same deep-processing task as a worksheet or as an open chat window, and the chat window raised task disorganization by 0.84 standard deviations with nothing in the model explaining it, so the fix is structure around sequencing and stopping rather than more content support. A forum study found an AI teaching assistant answered first in 97% of student-started threads, in a median of 0.92 minutes against 490 for classmates, while the self-regulatory language in the thread fell from 49% to 36%, which points at delaying the first response rather than removing the assistant.

New FAQs

Five new answers went up this week, each written for a question a colleague actually asks rather than for a topic. The FAQ index was also regrouped into nine sections by the task you are trying to finish, so the question you arrive with is findable without knowing what the field calls it.

Can I use AI to grade student work fairly and reliably?

The grading answer gives AI a place as a supervised second reader on work you have already marked, and not as the grader of record. Agreement does not carry across assessment types, and the check that looked strongest was run with the answer key in hand, which is not the test you are running when you grade blind.

How do I check whether AI-generated quiz and exam questions are good?

The question-quality answer starts by solving every item yourself and then asking what it requires a student to do, because answer keys and task demand fail more often than wording does. A generated question nobody has answered is a candidate, not an assessment.

How can AI support multilingual learners without erasing their voice?

The multilingual answer asks you to decide what language is doing in the task before you decide what the tool may do. Where language is incidental, AI can remove a barrier that has nothing to do with your construct; where the language is the construct, the same tool does the assignment for the student.

What should students learn for careers where AI is commonplace?

The careers answer is to teach the whole professional task and assess both the supported product and what the student can do unaided. The evidence supports disciplinary knowledge plus judgment, verification, and transfer, and it supports no list of tool skills.

What does personalized learning actually mean, and when does it help?

The personalization answer separates five things that get the same label, from reordering practice to letting the student choose what to study next. What matters is what is adapted, what evidence drives it, and whether it improves the learning you intended, which a familiar example or a responsive conversation can feel like without doing.

Concept spotlight

Five concept pages absorbed this week's findings, two of them new. Each is a standing page rather than a one-off, so the new evidence sits alongside what the page already argued.

Open education

The page is new, and it was written because thirteen studies arrived at once on what generative AI does to openly licensed teaching. Its argument is that the license rather than the model is the thing to examine, and the two new resources below both describe institutions adapting open content with AI faster than they can record who authored what. Read the open education page before writing an AI clause into an OER policy.

Cognitive offloading

It now separates what a tool discharges from how much a student used it, which is the distinction the offloading ladder draws and the empathy experiments demonstrate in a place most people would not look for it. The cognitive offloading page also carries the 0.84 standard deviation result, because it is the cleanest causal evidence the page has.

Evaluative judgment

Its new evidence is about what a judge is asked to do: grading methods moved rerun stability from 2.9% to 15.9%, and thirteen models folded under student pushback at rates from 46.5% to 98.5%. The evaluative judgment page now treats resistance to a false correction as a separate capability from getting the answer right.

AI literacy

Two results narrowed the construct: critical thinking ran through how students divided and reworked the cognitive labor, not through familiarity with tools, and a rubric built from 16 core papers showed how few resources name the competency they claim to build. The AI literacy page keeps both, because a literacy test and a literacy resource are measuring different things.

The other new page is statistics and data science education, whose first argument is that the discipline's AI question turns on the artifact, since datasets, charts, and code can now be generated faster than anyone can validate them.

For instructors

An open chat window needs a stopping rule, not more content support

Pairing a deep-processing task with an open chat window costs students their grip on the task itself, and the fix is scaffolding around sequencing and stopping rather than more explanation. In a preregistered experiment with 317 adults, the chat condition raised disorganization by 0.84 standard deviations (credible interval 0.47 to 1.21) with a direct effect of 0.77 that no mediator in the model explained, while deep processing did not move in either direction (median minus 0.22, interval spanning zero).

A first reply in under a minute removes the peers who were doing the reasoning

Delay or cap the assistant's first reply in a discussion forum, because the classmates who reason aloud are the ones who stop posting. Across four graduate courses and 28,123 coded sentences, an AI TA answered first in 97% of student-started threads, in a median of 0.92 minutes against 490 for peers, and self-regulatory language fell from 49% to 36% as peer replies dropped from 91% to 45% of follow-ups, with peer replies carrying about 70% of that regulation against 27% for the students who opened the thread. The self-regulated learning in a forum is produced by the replies, not by the answer.

Prompt training buys teachers time, not better materials

Budget teacher AI competency work as a workload intervention, and do not expect it to raise the quality of what teachers produce. In a 204-teacher randomized trial, training in prompt writing added a 21% preparation-time saving on top of plain ChatGPT access with no gain in rated material quality, because trained teachers wrote longer prompts and needed fewer refinement passes.

A lesson assistant saves time on a thin dose of lessons

Trial it for the hours it returns and not yet for the lessons it produces. An independent cluster trial of Oak National Academy's Aila followed 416 English primary teachers and measured preparation at 2.51 hours a week against 3.32 without the tool, which is about 49 minutes a week, while it covered 7% of lessons and quality was tested only for non-inferiority rather than improvement.

Bolting AI onto an open assignment adds nothing on its own

Fund the open education assignment you already run instead of adding a tool to it and waiting for a literacy gain. Two sections of an introductory child development course, 55 students working as usual and 58 with generative AI search tools added, reported the same perceived source-skill gain, 0.50 against 0.48 on a five-point scale, with the interaction term near zero (F(1, 84) = 0.04, p = 0.85).

Choose an assistant by what it does when a student says it is wrong

Resistance to a false correction separates models more sharply than accuracy does, so make it the first thing you test. Across 200 verified truss problems and thirteen models, solving accuracy ran from 64.5% to 97.0%, a 32.5-point spread, but told that correct work was wrong, models confirmed it at rates from 46.5% to 98.5%, a 52.0-point spread, which belongs in the evaluative judgment check you run before a rollout.

Teach AI-assisted writing as a genre and the scores stop overlapping

Treating AI-assisted writing as something to teach rather than something to police belongs inside the writing education curriculum, and the gap it opened was not marginal. Post-test means were 9.8750 for the section taught that way against 3.8125 for the control, with no overlap between the two distributions (Mann-Whitney U = 0.00, r = 0.86).

For developers

Build the scenario, not the tutor persona

Put the scaffolding into the scenario and the workbench before you tune the voice, because the persona is not where the effect lives. In two preregistered three-arm trials, 131 students on legal reasoning and 54 completing a competitive programming arm, the structured system reached 22.8 percentage points against 15.7 for a vanilla assistant, a 7.0-point advantage (p = 0.018, d = 0.56), while an LLM-as-Teacher persona over the same model gained 2.8 points and did not separate (p = 0.121).

A tutor that names the brute-force solution first is steering students wrong

Audit which solution a tutor names before you ship it on a multi-solution problem, and keep that audit in the intelligent tutoring acceptance tests. On 150 problems run with simulated students, brute force was the named approach in 22.2 to 32.1 percent of solution-specific openings against 13.0 to 19.4 percent for the optimal solution, and in probe-first conversations all four base models moved students toward a worse approach in 6.1 to 7.1 percent of cases.

Counting AI calls measures nothing about progress

Interaction counts are the easiest signal for a learning analytics pipeline to collect and the weakest thing to put on a dashboard, so instrument the execution state on either side of each call instead. Across 140 student-assignment pairs from 35 students, AI interactions correlated with successfully executed cells at r = 0.12, and with touched cells at the same r = 0.12.

Judge a session by how it drifts, not by how many questions were asked

A session that slides toward answer-seeking in its second half predicts a worse post-test, and a mandatory adequacy check after each response is the cheap intervention. From 880 substantive turns by 112 students aged 14 and 15, the steepness of the help-seeking trajectory added a change in R squared of .110 to a model already holding the pre-test score (beta = .34), while static interaction features added .023 and missed significance (p = .684), and 81.5% of turns were requests against 2.2% evaluation.

The scoring rule decides a benchmark score more than the model does

If your harness reports one number, you do not yet know what it measures. The same 2,250 dialogues between five tutors and simulated students scored 42.1% under exact match and 81.3% under a human-calibrated judge, a 39.2-point spread larger than the combined effects of model, strategy, and pipeline, and four judges from three vendors agreed within 6 points while structured pipeline labels gave no advantage (p = 0.263).

The grading method, not the judge model, sets reproducibility

Publish an adjudication statement naming the method, the harness, a repeat run, and how many graded items sit behind each claim, because the method is what moves the result. Reruns changed 2.9% to 6.1% of outcomes under a semi-rigid rubric and 13.5% to 15.9% under free holistic grading, while swapping the judge model moved only about three points.

An AI peer that agrees with everything loses its users

Configure agents to hold uncertainty and to decline agreement by default, since trust calibration is a behavior you write rather than a property a model has. Seven experts each spent 20 minutes in the student role with an agent cast as a peer, and what made it read as a peer was social: tentative reasoning and deliberate imperfection built trust, while blind agreement with everything the learner said destroyed it.

For researchers

Plan for a small effect, not the published average

In a new setting the headline effect of generative AI instruction is not the one to plan on, so pilot before you scale. A three-level meta-analysis of 58 studies and 6,941 participants pooled to g = 0.864, then reported heterogeneity near 87.9%, a prediction interval running from minus 0.213 to 1.942, and a bias-corrected effect of g = 0.258; print that interval beside the pooled estimate where a reader cannot miss it.

Critical thinking follows the division of labor, not tool familiarity

Assign an explicit division of labor and require students to rework the output, because experience with generative AI reached critical thinking only through those two routes. Among 891 students in the United States and Türkiye, specializing and coordinating cognitive labor carried b = 0.636 and reflectively transforming output b = 0.367, with no significant direct path, which turns the cognitive offloading question into a design choice rather than a warning.

A smaller education gap in AI use is not equity

Do not read a narrower use gap as progress, and do not build access targets on a binary last-three-months item. Matching 16 European countries at their 2025 adoption rates to their own internet-era year, the median gap between the most and least educated fell from 56.4 points to 28.6, negative in 15 of 16 countries (p = 0.0005), but composition-aware matching shrinks the change to minus 11.5 and minus 10.0, so an equity claim needs a stronger instrument.

Low use is not low willingness

Fund access and basic skills for the oldest and least educated rather than a campaign about trust, because use and trust travel separately. In 2025, 48.4% of Danes and 17.8% of Romanians had used generative AI in the previous three months against 32.7% across the EU, yet Romanians rated their trust at 4.1 against 3.8 on a seven-point scale, and 47% of them called current laws and safeguards adequate against 36% of Danes.

Offloading is only erosive when it bypasses the goal

Ask which cognitive operations a tool discharges rather than how much a student used it. Offloading helps when it takes on work peripheral to the learning goal and risks erosion when it bypasses processing constitutive of that goal, and processing the output afterward can rescue an otherwise constitutive offload; seven conditioned propositions replace one directional claim, and offloading regulation becomes a learnable dimension of AI literacy.

Sentiment tools miss about half the negative responses

Treat automated coding of open text as a validity argument rather than a tool choice, because accuracy hides exactly the cases you would act on. Against 1,220 hand-coded responses, the best transformer reached .79 accuracy but only .42 kappa and caught about half the negative cases, while lexicon tools misclassified roughly one in five, in a corpus that was 76.5% positive.

For administrators

Ask what the chatbot sends before you license it

Require a third-party script inventory before you license or host a course chatbot, and ask specifically which tags can read the page title, the URL, and the input field. Instrumenting 42 platforms found 21 sending conversation titles to at least one tracker and nine transmitting the verbatim prompt, 11 of the 13 leaking general-purpose platforms also sent the registration email, and education platforms averaged about 367 tracker requests each with 24% carrying user content.

Codified recall is the part of a credential losing value

Protect supervised practice and judgment under uncertainty, and read the curriculum design implication before the program marketing one. A difference-in-differences over 655,919 survey observations from 2018 to June 2026 found the return to a year of schooling in the most AI-exposed occupations fell 0.55 log points after November 2022, about 7% of its pre-shock level, while the return to experience rose 0.26 log points at three years and fell 0.12 at twenty-five.

Calling the chatbot empathetic moves the empathy off people

Position the system as triage that routes students back to people, and watch help seeking as the behavior that quietly disappears. Across four studies with 249, 494, 743, and 737 participants, framing AI as capable of empathy raised willingness to offload empathic responsibility (total effect 0.83) entirely through perceived capacity (indirect 0.73), and hearing that a peer had taken a problem to ChatGPT lowered emotion-reading accuracy to 7.13 correct out of 12 against 7.73 (p = .012).

Benchmarks passed, children still harmed

Evaluate at session and institution level rather than at the API, because a per-output filter cannot reach a companion interaction where no single turn crosses a threshold. Coded from 1,331 reported child-related incidents, sexualization is the largest harm subtype at 316, ahead of sexual exploitation at 131, and harms cluster with deployment, governance, and misuse failures rather than with model defects.

Write integrity policy for the occasional, conflicted user

Name refusal, support, and substitution as three populations the same assignment meets, because almost nobody is a daily user. Two anonymous surveys at one UK university found 81.05% of 94 first-year students had used a generative AI tool to help complete an assignment against 4.21% reporting daily use and 9.47% reporting none, which makes academic integrity a policy for the middle of a distribution rather than a cheater and non-cheater split.

Ethics modules are not teaching where the data comes from

Add a provenance review assignment to the ethics module you already run, which is a curriculum design change rather than a new course. A census of 36 programs across 21 universities found ethics taught in 29 (80.6%) but data provenance at the authors' bar in 8 (22.2%), with no master's or executive program meeting it and 22 of the 29 ethics-covering programs missing provenance entirely.

New resources

Four resources joined the resources library this week, and all four describe an institutional decision rather than a classroom trick.

Learning Design Wiki

Learning Design Wiki is an openly licensed reference that splits teaching design into principles, elements, patterns, and strategies, each tied to the citations behind it, and its own evidence report says 98% of the claims rest on a single study. Language models maintain it under one human maintainer and no page has been checked by a person, so it is a set of well-sourced leads rather than findings.

teacher.dev

teacher.dev is a nonprofit studio that ships one free classroom app a day, 34 so far, each aimed at a single moment such as a printable figure generator or a First/Then board for special education. Most run local-first with no account, so no student data leaves the device.

AI + OER Case Studies: A DOERS Project

AI + OER Case Studies is an open anthology of eleven cases on where generative AI meets openly licensed resources in higher education, from a federated university drafting guidance under a union contract to a human-in-the-loop textbook revision. No two chapters share an instrument, so it is practice to adapt rather than evidence to cite, and the collection is CC BY 4.0 while each chapter carries its own license.

Integrating AI in TVET: a practical guide for institutions

Integrating AI in TVET is a 2026 UNESCO-UNEVOC guide for institutions that train technicians and tradespeople, and the usable material sits in three annexes: a readiness self-reflection tool, a risk register, and an indicator framework. It reports no outcome data and calls its own maturity scale a discussion aid, so use it to structure a first institutional decision rather than to justify one already made.

Also added this week

The other studies, with methods and abstracts, are listed by day on the journal page.

Site notes

All 222 concept pages now carry a common misconceptions section that names the wrong belief a reader may already hold and corrects it with the evidence on that page.

Every FAQ was rewritten this week, and five new ones joined them, taking the collection to 42 answers written for the question a colleague actually asks rather than for a topic. The index groups them into nine sections by the task a reader is trying to finish, so arriving with a question no longer requires knowing what the field calls it.

Translations moved further than the English did: the twelve concept pages in the Essentials section and all of the FAQs are now translated in full across every language.

Every language has its own home page, and sixteen of them also ship a downloadable EPUB and PDF:

Everything above is public at github.com/edtechdev/aied.

Follow the update

Get each issue through the newsletter feed in a feed reader, by pasting that same feed into Blogtrottr to have new issues emailed to you on a schedule you choose, or by following it on LinkedIn, where each issue is published as a LinkedIn newsletter.

AIEd Research Weekly Update comes from the AI in Education Knowledge Base, which an AI agent generates. This issue was drafted by that same AI agent, with editorial guidance from the site developer, and every claim above links to the page it came from.

The knowledge base is dedicated to the public domain under a CC0 1.0 license. No rights reserved.

Found an issue with the content? Report it on GitHub, or contact the site developer, Doug Holton.

SourceHistoryEditChat

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.