On this page

Synthesis: Kearney (2026) argues that generative AI has no uniform effect on learning: the same tool becomes an engagement amplifier or a cognitive substitute depending on how the assessment is configured, not on the technology or on student intent. The paper builds a two-dimensional framework — the distribution of learning-relevant cognitive responsibility and the visibility of student process — that yields four assessment conditions, and uses them to show why the common countermeasures fail: a visible process that documents substitution is still substitution, and responsibility without evidentiary capture leaves genuine competence indistinguishable from substituted performance. It is a conceptual paper with no primary data; its value is in reframing the question from whether AI was used to what the assessment evidence actually warrants.

Key Findings

  • GenAI has no uniform effect on learning. The same tool amplifies engagement or substitutes for cognition depending on how the assessment is configured, not on the technology or the student's intent.
  • Two dimensions generate four conditions: how much learning-relevant cognitive responsibility the student retains, and how much of their process the assessor can see.
  • Visible process is not sufficient. A complete reasoning trace that documents uncritical acceptance of model output is evidence of substitution, not of competence.
  • Responsibility without evidentiary capture fails the assessment's purpose even when the learning is real, because genuine competence becomes indistinguishable from substituted performance.
  • Three design orientations follow — process-oriented assessment, authentic and participatory tasks, and dialogic formats such as oral defense — each a context-dependent trade-off rather than a solution.

The argument

Generative AI is not like earlier tools because of what it offloads. A search engine offloads memory and a calculator offloads computation; GenAI offloads reasoning, synthesis, evaluation and generation — the activities that constitute higher-order cognition. That is why assessment design, rather than detection or prohibition, is the lever: the same model that produces an unreadable essay can produce a competent-looking artifact with no competence behind it.

The paper's central distinction is between supported performance and durable capability. Validity, in Kane's (2013) and Messick's (1989) argument-based sense, is about whether the evidence warrants the inference being claimed. If the claim is "this student can analyse a case independently", a submission produced with a model in the loop does not warrant it — not because the work is dishonest, but because the inference is unsupported.

Two dimensions, four conditions

The framework crosses two questions:

  • Cognitive responsibility — who is doing the learning-relevant thinking: the student, or the tool?
  • Process visibility — can the assessor see how the artifact came to be, or only the artifact?

That yields four conditions, each with a different validity profile:

  • Engagement amplifier — high responsibility, high visibility. The student retains judgment and the assessor can see the reasoning. This is where AI plausibly helps.
  • Cognitive substitute — low responsibility, low visibility. The artifact is produced, the capability is absent, and nothing in the evidence reveals it: the worst case, and the one that produces "performance without capability".
  • Constrained engagement — low responsibility, high visibility. A student submits a complete reasoning trace showing uncritical acceptance of GenAI output at each step. The process is transparent, but it documents substitution rather than competence, so processing evidence alone does not rescue validity.
  • Misaligned effort — high responsibility, low visibility. The student does the thinking and the assessment cannot see it. Educationally sound, evidentially costly: the design fails "not the student, but the evidentiary purpose of the assessment".

The worked example applies one task — a 2,000-word case analysis in a second-year undergraduate course, requiring a theoretical framework to be applied to an authentic professional scenario — across all four conditions. Marking only the final submission puts it in cognitive substitute; adding a reflective appendix moves it toward constrained engagement; adding an annotated GenAI exchange or a short oral defense moves responsibility and visibility together.

Why the obvious countermeasures fall short

Three implications follow, and each cuts against a common institutional reflex:

  • Process evidence is necessary but not sufficient. Drafts, reasoning traces and annotated decisions help only if they would differ between a student who understood the material and one who did not. A rubric-satisfying reflection on tool use is itself generatable.
  • Restriction is not automatically the authentic option. Where AI-supported practice is the capability being certified, excluding GenAI reduces authenticity rather than protecting it. The pivotal question is whether the outcome is what the student knows, what they can do unaided, or what they can do competently with the tool.
  • AI intensifies existing patterns rather than disrupting them. Students with stronger prior knowledge and better self-regulation interrogate the tool and keep responsibility; less-proficient students more readily accept generated answers. But this is not simply advantage accumulating to the strong: GenAI can lift the immediate performance of lower-performing learners without those gains surviving the removal of support — which is precisely the performance-without-capability pattern.

Three design orientations

The paper proposes three orientations, presented as context-dependent options rather than solutions, each to be weighed against disciplinary purpose, cohort size, resources and the capability being certified:

  • Process-oriented assessment — capture the trajectory, not just the product.
  • Authentic and participatory tasks — require judgment, contextual application and transfer, raising the threshold at which GenAI can complete the work without the student's understanding.
  • Dialogic and interactive formats — oral defenses, vivas, in-class development from unseen material.

What this means for practice

  • Assessment designers. Name the capability first. Before choosing a format, specify which cognitive responsibilities students must retain, not how much effort they should expend.
  • Assessment designers. Require process evidence that discriminates: drafts, annotated decisions and accounts of tool use that would look different for a student who had not understood the material.
  • Assessment designers. Do not accept a reflective appendix on its own. If the reflection can be generated or rubric-satisfied, the task has become constrained engagement with visible shallow work.
  • Assessment designers. Pair the product with something only understanding can produce — an annotated GenAI exchange, a short oral defense of interpretive choices, or in-class development of the analysis from unseen material.
  • Quality assurance staff. Sample performance across tasks, time points or conditions of AI support wherever independent capability is part of the certified claim.

Limitations

  • Conceptual, not empirical. The paper presents no primary data and does not test whether the four conditions predict learning outcomes; the framework is an argument, not a validated instrument.
  • Two dimensions are not a validity theory. Cognitive responsibility and process visibility do not by themselves establish validity, nor do they resolve competing demands among assessment purposes (certification, feedback, gatekeeping, equity).
  • Visibility is a means, not a sufficient condition — the paper says so itself, which leaves open how much process evidence is enough in any given discipline.
  • No universal solution is offered. Assessment design with GenAI is situated, involving trade-offs in workload, scalability, accessibility, authenticity and equity that the framework names but does not settle.
  • Boundary and disconfirming cases are unexamined — prior knowledge, self-regulation differences, tool configuration, cohort size, disciplinary epistemology and institutional constraint are all flagged as future research rather than modeled.

Citation

Kearney, S. (2026). Performance without capability: generative AI and the validity of assessment in higher education. Assessment & Evaluation in Higher Education.

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.