Assessing learning beyond AI assistance: assisted performance and unaided retention in generative AI-supported learning

Synthesis: Wei (2026) sorts the evidence on generative AI in learning by what the test measures: assisted performance, an immediate posttest without it, or a delayed one. Three Web of Science searches returned 670 records, 664 unique; twelve studies were purposively selected, from 32 nursing undergraduates to a randomized trial where ChatGPT-supported study fell behind on a surprise supervised test at day 45. The direction is not uniform: structured tutoring and feedback produced gains, unrestricted assistance produced deficits on unaided examinations, and language contrasts split by skill. What the corpus cannot support is what readers want from it. None of the twelve scored assisted performance and two unaided tests together with verified AI exclusion, so the review documents a gap in the assessment literature rather than a forgetting curve.

Key Findings

1. No study measured all three conditions. None of the twelve scored assisted performance, an immediate no-AI test and a delayed no-AI test together with verified exclusion at both, so measured gains cannot be tied to retention.
2. Unrestricted access can cost unaided performance. In school mathematics, GPT access lifted practice scores but came with an unaided exam score roughly a sixth lower than no-AI instruction; tutor guardrails removed the deficit.
3. A longer interval did not favor AI support. On a supervised test at day 45, ChatGPT-supported study came out clearly behind conventional study on a ten-point scale, with nearly a third of participants lost to attrition and no matched immediate baseline.
4. Retention contrasts depend on the skill tested. One week after lexical tasks, chatbot-supported learners were well ahead on productive collocation retention while receptive vocabulary was level between groups.
5. Designed support looks different from unrestricted access. Theory-informed tutoring and practice-with-feedback conditions produced immediate advantages — one comparison gained thirteen percentage points over lecture — but those studies stopped at immediate or short-interval outcomes.
6. Most studies could not establish AI exclusion at testing. Where test-time access was undocumented it was coded "not established," and a supervised or paper-based test was not automatically classified as verified exclusion.
7. Candidate mediators went unmeasured. Retrieval attempts, elaboration and feedback use are the processes the review can name, and almost none of the twelve recorded them well enough to test as mediators.

Three conditions, three different questions

An assisted score combines what the learner can do with what the tool supplies, and a fluent study session is a poor guide to what survives it. Wei builds the argument on the classic learning-versus-performance distinction: supported scores describe functioning under a particular arrangement, unaided scores describe what the learner produces alone. The practical consequence is that "no-AI testing" is a claim about procedure, not a label. A supervised examination and an unsupervised online questionnaire supply different evidence of restricted access, and the second may leave saved AI responses within reach. Wei therefore codes test-time access three ways: an explicit restriction with a documented compliance check (verified exclusion), a reported restriction or supervised administration without independent confirmation, and "not established" where the procedure is undocumented. Only the first supports the inference that a score reflects learning rather than availability — a validity question before it is an integrity one.

What the twelve studies found

The evidence tracks instructional design more than the tool. Barcaui's randomized trial put the ChatGPT-supported group at 5.75 out of 10 on the surprise supervised day-45 test against 6.85 for conventional study (p = 0.002, reported Cohen's d = 0.68), with 29.2% attrition — but study time also differed (self-reported means 3.2 versus 5.8 h), and the paper's curves interpolate between assumed and observed endpoints rather than estimating a forgetting trajectory. Qin et al. compared three intact undergraduate classes: the group-by-time interaction was significant (p = 0.001), the GenAI-versus-traditional delayed interaction was not (p = 0.129), and the cognitive-linguistics-versus-traditional contrast was (p = 0.013), across packages taught by different instructors. Sezgunsay et al. found no immediate difference among 32 nursing undergraduates but a six-week median advantage of 65 versus 55 (p = 0.044), measured remotely and unsupervised. Zhang and Huang tested 52 high-school learners over 8 weeks and reported descriptive follow-up gaps, but their reported test statistics do not reconcile with the design, so only the means can be read. Gustafsson et al. found nothing after 4 weeks of team-allocated trauma training (p = 0.33) — a null result, not equivalence.

Two further contrasts are worth separating by task. In school mathematics, an unrestricted GPT condition improved practice performance yet sat 17% below no-AI instruction on the unaided examination, and a tutor condition with guardrails removed that deficit — the clearest evidence in the review that the arrangement, not the tool, sets the direction. In vocabulary, one week of lexical work left the chatbot group well ahead on productive collocation retention (6.15 against 3.61 out of 10, p < 0.001) while receptive vocabulary was indistinguishable (7.67 against 7.58, p = 0.863), a split the review reads as skill-specific rather than as general retention. The designed-support studies point the same way: theory-informed tutoring and practice with feedback produced immediate advantages, including a 13 percentage-point gain over lecture (p = 0.009), but none of them carried the comparison to a delayed unaided test.

Why assistance helps and hurts: retrieval, offloading, feedback

The explanatory section is deliberately provisional. When a learner receives an answer before attempting recall, the retrieval processes that build durable memory may never run — the standard argument from desirable difficulties and pretesting research. But the consequences of offloading depend on which operations are delegated. Wei cites Nakamura's distinction between assistance with operations peripheral to the learning goal and delegation of the processing students are meant to master: AI formatting a table leaves the causal reasoning to the learner, while an AI-written interpretation absorbs it. Readymade explanations may also release working memory for harder work, so theory does not settle the direction. Feedback is the third route, and its value depends on whether learners interpret it, locate the error and revise; practice-with-feedback experiments showed immediate memory and generalization benefits without measuring delayed retention. ICAP categories could characterize engagement during assisted tasks, and prior knowledge plausibly decides whether a student inspects a tentative answer or bypasses the attempt.

What a decisive study would look like

The closing proposal is a design specification, and the most portable part of the paper. An informative experiment would score an AI-assisted task, then administer an immediate no-AI assessment and at least two later comparable no-AI assessments, verify restrictions on external assistance at each, distinguish answer exposure from retrieval before feedback, and randomize at the unit of assignment. Alongside scores, a compact process record would capture the learner's first answer, the timing and content of AI feedback, any revision, and the final explanation produced without help — separating students who check an independently formed hypothesis from those who copy a solution before forming one. Repeated observations would permit analysis of change rather than interpolation of unobserved forgetting curves, and item-level reporting with comparable forms would stop a repeated-item advantage from passing as retention. For teaching, the arrangement is narrower than a ban: allow AI during guided preparation, reserve supervised opportunities for unaided retrieval, and treat what emerges as formative until follow-up testing exists.

What this means for practice

• Instructors: Decide which of the three conditions each assessment is meant to report, and label it that way: a task completed with AI is evidence of supported performance, not of what students can do when the tool is gone.
• Assessment designers: If a no-AI test is meant to certify independent capability, write the access restriction and the compliance check into the procedure, and treat a paper-based or merely invigilated test as unverified until both exist.
• Faculty developers: Pair any AI-integrated unit with a scheduled unaided retrieval task later on, since the review's positive examples combined structured support with a separate independent measure.
• Administrators: Do not read rising assisted-task scores as learning gains in program review; ask instead which assessments in the program produce unaided evidence and how test-time access was verified.
• Researchers: Report AI availability during learning and at each test, the construct assessed, and the learning activities between assessments, since these are the three omissions that make the twelve studies uncomparable.

Limitations

• The search was targeted rather than systematic: three Topic searches of Web of Science Core Collection, no exhaustive full-text eligibility screening, and supplementary leads and pre-2022 cognitive studies kept outside the 664-record pool.
• The evidence base is twelve purposively selected studies, and Wei states that none documented all three conditions with verified AI exclusion at both no-AI posttests, so cross-study synthesis of retention is not available.
• Baselines, follow-up intervals and comparators differ across studies: no immediate independent baseline in Barcaui, a remote unsupervised six-week test in Sezgunsay, and intact classes with different instructors in Qin.
• Test-time access is unverified across most of the corpus, so several contrasts cannot be read as unaided performance at all.
• Tasks range from vocabulary production to secondary mathematics to nursing procedures, and the review does not pool effect sizes across them.

Citation

Wei, J. (2026). Assessing learning beyond AI assistance: assisted performance and unaided retention in generative AI-supported learning. Frontiers in Psychology, 17, 2029407.