Using Generative AI to Promote Psychological, Feedback, and Artificial Intelligence Literacies in Undergraduate Psychology

Synthesis: Richmond & Nicholls (2025) describe integrating generative AI (ChatGPT) into a large second-year psychology assessment — the "Media Assignment," in which students translate a published research article into a media release and video for a lay audience. They modified Phase 1 so that instead of peer assessment, students used ChatGPT to generate a draft media release and then critiqued that output against the course's marking rubric, grading six criteria and identifying three strengths, three weaknesses, and three priorities for revision. In Phase 2 students revised the AI text (tracked changes), and in Phase 3 they produced a final video. Students accurately assessed the ChatGPT output: they rated it stylistically strong but lacking accurate coverage of the research's aims, methods, and results. Working with the rubric and genAI early had a small but significant benefit for script-revision grades relative to the prior peer-review cohort, though final video grades did not differ. The design is presented as building three literacies at once — psychological, feedback, and AI — and as an example of shifting Assessment emphasis from product to learning process.

Key Findings

• Rubric-based critique of genAI output: Students graded a ChatGPT-drafted media release against the marking rubric; 92% rated the output at pass or above on "Style," but 94% rated it at credit level or lower on "Aims/Methods/Results" — recognizing strong style alongside inaccurate coverage of aims, methods, and results (χ²(4) = 317.48, p < .001; Cramer's V = 0.66).
• Early rubric engagement helped later performance: The 2023 AI-critique cohort scored higher than the 2022 peer-review cohort on the script revision task (t(733.05) = 4.93, p < .001, d = 0.36); no significant difference on the final video (t(732.48) = 1.52, p = .129). The authors caution the gain could stem from starting from stylistically strong ChatGPT output rather than rubric engagement.
• Scaffolding gap: A small number of students failed to catch coverage inaccuracies (awarding ChatGPT High Distinction/Distinction on "Aims/Methods/Results"), indicating some need more Scaffolding of the Critical Thinking process for evaluating genAI veracity.
• Three literacies at once: Psychological literacy (translating research for a lay audience), feedback literacy (early rubric engagement, candid critique without peer-reaction concerns, self-evaluation), and AI literacy (understanding, working with, and critically evaluating AI) are promoted by one integrated assessment.
• Process over artifact: The authors argue genAI's ability to mimic human artifacts pushes educators to assess the learning process, not the final product, and to use AI to outsource initial creation so students practice higher-order evaluation and analysis.

What this means for practice

• Instructors. Schedule rubric-based critique of the AI draft before students revise it — the 2023 cohort that graded ChatGPT against the rubric outperformed the 2022 peer-review cohort on the script-revision task (d = 0.36, t(733.05) = 4.93), so the critique belongs early in the assessment sequence.
• Instructors. Use a stylistically strong but factually unreliable ChatGPT draft as a "teachable artifact" instead of banning the tool, so students build AI literacy by detecting hallucination and appraising evidence quality.
• Instructors. Model the verification step explicitly for the minority who awarded ChatGPT High Distinction or Distinction on "Aims/Methods/Results": have students check the draft's claims against the source article, because recognizing strong style did not guarantee catching coverage errors.
• Designers. Collect process evidence — the rubric critique and the tracked-changes revision — alongside the final video, since the measurable gain appeared on the revision task and not on the final artifact.

Limitations

• The evidence is a quasi-experimental comparison of two year cohorts in one second-year psychology course at a single university (2022 peer review vs. 2023 AI critique), not a randomized trial.
• The authors cannot separate rubric engagement from the head start of beginning with stylistically strong ChatGPT output, so the source of the revision-task gain is unresolved.
• The benefit did not extend to the final video (t(732.48) = 1.52, p = .129), so it is limited to one task within a three-phase assignment.
• Outcomes are course grades rather than direct measures of feedback or AI literacy.

Citation

Richmond, J. L., & Nicholls, K. (2025). Using generative AI to promote psychological, feedback, and artificial intelligence literacies in undergraduate psychology. Teaching of Psychology, 52(3), 291–297. https://doi.org/10.1177/00986283241287203