Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics

Synthesis: Quince and Faulconer (2026) track how students in a second-year physics unit used generative AI across three assessments after an institutional policy change forced the assessments to be redesigned. One assessment ran under a purpose-specific use policy and two under open use, and every submission carried a mandatory GenAI declaration. Coding 120+ reported use cases into learning, formatting, doing-task, and feedback functions and mapping them to SAMR shows the task, not the policy tier, shaping what students did: feedback use was absent in the scaffolded presentation, appeared in the laboratory workbook, and dominated the take-home quiz at 66.7%. Students who declared GenAI use scored slightly higher on all three assessments, but no difference was statistically significant.

Key Findings

1. Use followed the task, not the policy tier. Under the same open-use policy in a university course, the laboratory workbook drew 83.3% learning-oriented uses while the take-home quiz drew 66.7% feedback uses — the authors conclude that assessment design, not permission level, determines which GenAI functions students reach for.
2. Feedback became the dominant use as the term progressed. It accounted for 0.0% of uses in the scaffolded presentation, 11.1% in the workbook, and 66.7% (n = 18) in the quiz, where students reported checking calculations, verifying quiz answers, and clarifying concepts.
3. Formatting was heaviest early and then declined. Presentation work ran on Learning (54.5%) and Formatting (40.9%), matching GenAI's substitution role for structure, references, and grammar; Formatting fell to 17.6% in the workbook and 14.8% in the quiz.
4. Doing-task uses were rare but reached modification and redefinition. Only 4.5%, 22.2%, and 14.8% of reported uses across the three assessments involved performing calculations or procedures — the category the authors map from modification to redefinition when GenAI generates complete solutions to unfamiliar problems.
5. Declared GenAI use was not associated with weaker performance. Users scored 15.76 vs. 14.80 (/20) on the presentation (d = 0.63), 41.18 vs. 40.60 (/50) on the workbook (d = 0.12), and 23.44 vs. 22.77 (/30) on the quiz (d = 0.19); none of the differences was statistically significant.
6. ChatGPT dominated tool choice, and one tool was banned. Students reported ChatGPT most often (A1 n = 16, A2 n = 13, A3 n = 21) and Copilot next (n = 5 each time), while DeepSeek was prohibited by the institution and Copilot was the only licensed, data-protected option.
7. Feedback use spanned all four SAMR levels. The authors' mapping puts learning at substitution–augmentation, formatting at substitution, doing-task at modification–redefinition, and feedback across every level, because checking an answer and interrogating an explanation can look identical in a declaration.

What the three assessments did differently

The unit was revised before term because the 2025 policy replaced a five-tier GenAI model with three tiers — no use, purpose-specific use, and open use — and none of the existing three assessments permitted any GenAI use. The first assessment, an oral presentation worth 20%, moved to purpose-specific use that allowed background research, idea generation, slide generation, and scripting while prohibiting GenAI generation of the final slide content or narrative. The laboratory workbook (50%) and the final take-home quiz (30%) ran under open use, with the quiz converted from an in-class multiple-choice test to a take-home format whose tasks were reframed around real-world scenarios.

Because the three assessments differed in format, weighting, and cognitive demand as well as in policy, the authors describe the design as an observational time series and explicitly decline to claim that policy caused the differences they observe. Common learning progress and growing tool familiarity are named as confounders. The comparison they do make is functional: each task made a different kind of support useful, so the observed shift from formatting toward feedback is read as a response to what the work required.

Coding the declarations, and where the coding is weak

Declarations were mandatory and submitted through the LMS; the paper reproduces the exact statement students completed, including the tool used and the purpose, and those purposes became the qualitative data. Two academics coded the reported uses collaboratively into four functions, and the paper is candid that because the coding was collaborative rather than independent, no intercoder reliability coefficient was calculated. GenAI was used only to suggest possible category labels and descriptions, with all categorisation decisions left to the researchers.

The authors also flag that the alignment between declarations and submitted work was judged by the lead author, who coordinated the unit and marked all three assessments. Performance data came from that single marker, so no inter-rater reliability can be claimed, and the ethics exemption was granted after delivery because the redesign responded to a policy change rather than a research plan. Thirty-six students were enrolled; incomplete submissions or missing declarations reduced the analysis to 31, 31, and 36 submissions, and students could not be tracked across assessments.

Why the feedback finding matters

The shift toward feedback is the paper's most consequential result because it is where feedback literacy and evaluative judgment meet. Students asking GenAI to check a calculation may be comparing GenAI's explanation against their own reasoning — the behavior the paper treats as the theoretical marker of deeper engagement — or they may be accepting an authoritative-looking answer. The declarations cannot tell those apart, which is why the authors' first recommendation is procedural: require students to highlight the feedback they received and explain whether and how they revised in response, so evaluative judgment becomes visible rather than inferred.

That recommendation is also the paper's answer to integrity policy built on restriction and suspicion. The declarations documented preparation, clarification, refinement, and verification rather than only answer production, and where GenAI could perform the intended cognitive work, greater restriction is the authors' own recommendation. Their framing is that a policy tier sets a boundary, while the assessment determines the demand — the design principle that authentic assessment, constructive alignment, and taking account of GenAI together make actionable.

What this means for practice

• Instructors. Treat the declaration as evidence, not paperwork: ask students to name the tool, the purpose, and what changed in their work because of it, then grade the explanation of that change.
• Instructors. Design the check into the task. Verification, clarification, and correction are the uses students reach for when a task rewards getting the reasoning right, so put those uses on the critical path rather than leaving them optional.
• Assessment designers. Assume policy tiers are weak levers. The same open-use policy produced learning-heavy use in one assessment and feedback-heavy use in another, so redesign the task when you want a different pattern of use.
• Administrators. Expect declarations to under-report how much the task is doing and to miss whether a check was shallow; pair the declaration with a revision record before using GenAI-use data as a quality indicator.
• Researchers. Single-marker grading, collaborative coding without a reliability coefficient, and self-reported use all bound what a design like this can claim; budget for independent marking and a second coder when the study is planned rather than retrospective.

Limitations

• One unit, one institution, 31–36 submissions. A second-year physics unit at one regional Australian university cannot establish how the pattern generalises to other disciplines, cohorts, or policy environments.
• Declared use, not observed use. GenAI use came from student-written declarations that the authors describe as potentially incomplete, strategically framed, or open to interpretation; declarations generally matched the submitted work, but that judgment was made by the marker rather than an independent rater.
• Students could not be tracked across assessments. No individual change can be measured, so the progression from formatting to feedback is a change in the distribution of uses rather than a within-student shift.
• Performance differences are not statistically significant at any assessment and the design cannot separate policy, task format, weighting, and growing tool familiarity; the paper presents them as a tentative trend that does not support a claim of improved performance.
• Table 4's percentages and counts do not reconcile. The workbook row reports 17.6% from n = 11 while its other three rows are computed against a denominator of 18, and the four category counts for that assessment sum to 32 across 31 submissions, so the reported per-assessment composition cannot be reconstructed from the table.
• The redesign was not planned as research. The ethics exemption was retroactive, marking and coding were done by the unit's own coordinator, and no follow-up measures of learning or of evaluative judgment were collected.

Citation

Quince, Z., & Faulconer, E. (2026). Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics. International Journal of STEM Education, 13(61).