Black Box Assessment: Rethinking Integrity and Learning for a Time of Generative AI

Synthesis: Winstone, Gravett and Elkington treat the GenAI assessment problem as a wicked problem in Rittel and Webber's sense — no correct solution, no stable problem space, trade-offs everywhere — and argue that the sector's dominant responses are discursive rather than structural: traffic-light systems, declarations and permitted-use rules tell students what is allowed without changing the task. Their alternative, Black Box Assessment, keeps the task and changes what is assessed, requiring students to submit "windows" into the process — drafts, logs, reflections, records of GenAI use — and rewriting learning outcomes and rubrics so that error, revision and cognitive change earn credit. The paper is a conceptual blueprint: it reports no data, no pilot and no outcome, and it says so. Its contribution is the design argument and the four principles behind it.

Why the usual answers do not hold

The authors begin from Corbin and colleagues' framing of GenAI assessment as a wicked problem, and they take the practical consequence seriously: if there is no single solution, then principles are more useful than prescriptions, because principles can be adapted across contexts and can guide trade-offs. They add the distinction that does most of the work — between discursive responses, which communicate what is permitted, and structural responses, which change the nature of the task so that permitted use is built in rather than policed.

Discursive responses fail on their own terms. They are hard to enforce, they place the burden of interpretation on students, and institutions cannot tell whether students understood or followed them. Corbin's interview work with students and staff at an Australian university found both groups doing "boundary work" to guess where the line was, and some educators responding to that uncertainty with paralysis rather than redesign. Black Box Assessment is offered as the structural alternative: the assessment itself determines the boundaries, so there is nothing to guess.

Four principles, one inversion

The model rests on four principles. First, assess process to capture evidence of learning: a polished product does not show the cognitive work that produced it. Second, value errors as evidence of progress: drafts, wrong turns and abandoned arguments are traces of learning, not deficiencies to conceal. Third, encourage transparency and agency: students make their thinking visible through reflections, drafts and feedback encounters, which builds metacognitive awareness and makes outsourcing less attractive. Fourth, reward the learning journey rather than the endpoint, crediting the evolution of understanding alongside the quality of the final artifact.

The name comes from two directions at once. Black and Wiliam's Inside the Black Box supplies the formative-assessment lineage: inputs and outputs are visible, learning is not, and assessment can open the box. Syed's Black Box Thinking supplies the error logic: aviation recovers the flight recorder after failure in order to learn, not to punish, and assessment can treat drafts and revision the same way. The inversion is deliberate — traditional assessment rewards opacity and polish, while this model treats transparency as the source of integrity.

What a window is, and how many

A "Black Box Window" is structured process evidence submitted alongside the product: draft histories, process screencasts, research logs, reflective commentaries, annotated bibliographies, lab notebooks, code commits, debugging logs, records of GenAI interaction. The authors recommend two to three well-justified windows per task rather than a portfolio, and suggest combining natural process evidence (what the work leaves behind) with reflective evidence (what the student says about it).

That combination matters because it addresses the obvious objection. If process evidence is itself a written artifact, a student can generate it too. The authors' answer is not detection but design: ask for evidence that is costly to fake and revealing to produce, make the reflection specific to decisions the student actually made, and treat GenAI use as a transparent and assessable element rather than a hidden shortcut. Their test is a design question, not a forensic one: does this design make outsourcing harder, riskier and more effortful than genuine engagement?

The four components

Implementation needs four things: a task; at least one learning outcome that explicitly concerns how students engage with and develop their work rather than only the product; one or more windows; and rubric language that distinguishes levels of process evidence. The authors present this as a conceptual toolkit rather than a template, and note that existing tasks can often be kept — only the assessment of them changes.

They also address the workload objection with a concrete arithmetic: a 2,500-word essay could become a 1,500-word essay plus 1,000 words of process evidence, so the total read is unchanged while the evidence is richer. Staged submissions and peer feedback provide engagement points without formal assessment at every stage.

Key Findings

1. GenAI assessment is framed as a wicked problem, which makes principles more useful than prescriptions and legitimises iteration, divergence and compromise in design.
2. The decisive distinction is between discursive responses (rules, declarations, traffic lights) and structural responses that change the task, and only the latter removes the boundary work from students.
3. Black Box Assessment keeps the product but adds process evidence: at least one process-focused learning outcome, two to three windows, and rubric criteria that credit process.
4. Errors, revisions and false starts are treated as assessable evidence of learning rather than deficiencies, drawing on failure-driven memory and negative knowledge.
5. The model inverts the usual logic of integrity: it seeks integrity through visibility of process rather than through policing products, which cannot be done reliably.
6. Process evidence can itself be generated by AI, so the defense is design-based — make genuine engagement the least effortful path — rather than detection-based.
7. The same four principles are proposed as an evaluative frame for institutions redesigning assessment, not only for individual tasks.

What this means for practice

• Pick one high-stakes task and add a process outcome. Keep the product, add a learning outcome about how the work developed, and write the rubric so that evidence of iteration and cognitive change earns credit at each level.
• Ask for two or three windows, not a portfolio. Combine something the work leaves behind (draft history, commit log, lab notebook) with a short reflection tied to a specific decision, and drop the essay length to keep marking flat.
• Write reflection prompts that cannot be answered generically. "What did you believe at the start that you later changed your mind about, and why?" is harder to fabricate than "reflect on your process".
• Scaffold the unfamiliarity. Students socialised into polished-output assessment need exemplars, assessment dialogue and formative feedback before process evidence is a fair ask.

Limitations

• No empirical test. The paper is a conceptual blueprint; it reports no pilot, no comparison and no outcome data, so claims about workload, integrity or learning gains are design arguments rather than findings.
• The workload arithmetic is illustrative. Replacing a long essay with a shorter one plus process evidence assumes markers read the two in comparable time, which the paper asserts rather than measures.
• Process evidence is itself text. The design argument raises the cost of faking but does not establish that it prevents it, and the paper concedes the approach cannot make assessment AI-proof.
• Failure is not equally distributed. The authors acknowledge that the opportunity to fail and recover is shaped by power and privilege, and that "failure" can stick to some students more than others, but they do not resolve how a rubric should handle that asymmetry.
• Institutional conditions are assumed. Two to three windows per task across a program, with staged submissions, is a substantial design and coordination load that the paper does not cost.
• Discipline examples are brief. STEM labs, code repositories and clinical logs are named as natural process evidence, but the paper does not examine how well rubric language for process transfers across those epistemologies.

Citation

Winstone, N. E., Gravett, K., & Elkington, S. (2026). Black box assessment: Rethinking integrity and learning for a time of generative AI. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2026.2661365