Research Article
Assessing the process, not the product: the PACE framework for generative AI in higher education
Synthesis: Francis, Rutherford and Smith argue that generative AI has broken the inference most summative assessment rests on: that a good product was made by a student who did the learning. Detection cannot restore it, they argue, because detectors are unreliable and inequitable, and permission frameworks specify what use is allowed without saying how well it was done. Their alternative is PACE, Process-Anchored Competency Evaluation: a five-domain pathway (critical sensemaking, strategic verification, directed co-creation, iterative refinement, metacognitive engagement) crossed with a four-level scale of human agency (deferring, checking, directing, critiquing), and all of it aimed at the student's evaluative judgment. Process and product are marked separately, and integrity rests on cognitive friction, a design in which faking the evidence costs more than doing the work. The paper reports no student data; its four validation claims are stated as an agenda.
Key Findings
- The object of assessment becomes evaluative judgment. PACE grades the quality of a student's decisions in directing GenAI, questioning, verifying, steering and refining, rather than the polish of the submitted artifact.
- Five domains locate where judgment happens. Critical sensemaking, strategic verification, directed co-creation, iterative refinement that retains author integrity, and metacognitive engagement form an iterative pathway rather than a fixed sequence of stages.
- A four-level agency scale grades how well it happens. Deferring, checking, directing and critiquing run from uncritical acceptance to interrogating the tool's methods and limits, with directing set as the benchmark for competent practice.
- Process and product are marked apart. A mediocre artifact with strong verification decisions and a polished artifact built on uncritical reliance receive different profiles, which the authors argue stops one mark conflating two constructs.
- Integrity comes from friction, not surveillance. A rejection log paired with a synchronous conversation is designed so that fabricating the process costs more than doing it, and the log alone never counts as sufficient evidence.
- The paper claims no outcome data. Reliability, marker agreement, resistance to fabrication and gains in evaluative judgment are presented as four testable propositions, not as findings.
Why the product stopped being evidence
The validity of most summative work depends on an inference from artifact to capability: a well-argued essay is taken to show a student who can argue. The authors' prevalence figures make the inference untenable, with 88% of UK undergraduates reporting GenAI use in assessment in 2025 and 94% in 2026. Their case against detection has three parts. Detectors produce high false-positive rates and are defeated by light paraphrasing or by a second tool asked to humanize the text. They over-flag students writing in an additional language and neurodivergent students. And even a perfect detector would settle nothing, because establishing that a student did not use GenAI says nothing about what they can do with it. Permission instruments such as the Artificial Intelligence Assessment Scale and structural models such as the University of Sydney's two-lane approach govern where GenAI may be used, but a marker still has no way to judge the quality of the thinking inside a permitted use, which is the gap process-based assessment has to close.
The five domains and the agency axis
PACE separates two questions that most rubrics merge: what the student does, and how well they do it. The pathway covers critical sensemaking (framing the problem and deciding whether GenAI helps), strategic verification (testing claims and citations against credible evidence), directed co-creation (holding the purpose, structure and argument while using the tool to generate and challenge), iterative refinement that retains author integrity, and metacognitive engagement, which makes decisions and revisions visible rather than naming a stage of production. It rests on self-regulated learning and Metacognition, the processes a student must run rather than hand to the tool. The second dimension grades each domain on four levels of human agency, and the same axis runs across all five rows, so students and markers share one vocabulary for improvement. A task need not reach the same level everywhere: the result is a profile across the domains that matter, not uniform attainment. Directing is the benchmark for sound practice, and sustained deferring or checking in key domains indicates limited critical control.
Making the process hard to fake
If the product can be faked, the process evidence can be too, and the framework answers with friction rather than surveillance, treating integrity as a design problem. A short rejection log recording the prompt, the rejected response and the reason for rejecting it is designed so that fabricating it convincingly requires exactly the judgment being assessed, an idea the authors connect to desirable difficulties in the learning sciences. Three qualifications limit the claim. No free-standing text artifact is trustworthy, since GenAI can produce reflective writing that experienced markers cannot distinguish from a student's own, so rationales are marked on their specificity to the student's own argument and generic justifications score at the checking level. The log is never the sole evidence, because a student must also speak to the decisions in a synchronous conversation. And what the artifacts evidence is warranted judgment, the capacity to defend a decision, not the fluency with which it is presented.
The equity commitments
Process assessment can reproduce the biases that discredited detection, so the framework carries four commitments. It marks decisions rather than linguistic fluency, which keeps attainment apart from register and avoids penalizing students whose natural style reads as robotic. It favors low-surveillance artifacts such as rejection logs and annotated drafts over screen recordings and keystroke histories, which treat composing on paper, working offline or sharing a device as anomalous. It treats consent, data protection and reasonable adjustments as design constraints alongside universal design for learning. And it confronts unequal tool access: only 38% of UK students report being provided with GenAI tools by their institution, so programs should supply a common toolset, should not assume students will use GenAI, and should offer equivalent alternatives where tasks require it.
What this means for practice
- Instructors: Pair every process artifact with a short synchronous conversation about the decisions inside it, because a log on its own is the kind of talk that changes nothing, and treat directing a tool as itself authentic assessment of professional practice.
- Assessment designers: Report process and product as separate outcomes and grade a profile across the domains that matter for the task, rather than demanding the same level in all five; process-based rubrics show how the mechanics can work.
- Faculty developers: Calibrate markers on the four agency levels with exemplars before adopting the rubric; the authors concede that descriptors alone do not produce consistent judgments.
- Program directors: Adopt PACE across a program rather than in one module, so students meet the same expectations repeatedly, and decide in advance whether GenAI tasks are compulsory or carry an equivalent alternative.
Limitations
- The paper is a conceptual contribution with no empirical validation, and the authors state plainly that the matrix has face validity rather than demonstrated reliability.
- Whether markers can apply four levels consistently is untested, and the paper cites evidence that shared criteria alone do not produce agreement, which makes calibration a non-trivial precondition.
- All four validation propositions are untested: reduced attainment penalty for additional-language writers, higher inter-marker agreement, stronger evaluative judgment, and resistance of friction artifacts to fabrication.
- The prevalence figures are secondary citations (88% in 2025, 94% in 2026, 38% given institutional tools), and the worked examples of converting the matrix to marks sit in supplementary material not included in the preprint text.
- PACE develops the process and performance proxies of the 4Ps framework but not the practice proxy, and the extension to agentic AI is signposted rather than designed, leaving multi-step delegation unaddressed.
Citation
Francis, N. J., Rutherford, S. M., & Smith, D. P. (2026). Assessing the process, not the product: the PACE framework for generative AI in higher education. EdArXiv preprint. https://doi.org/10.35542/osf.io/h982u_v1