Asynchronous Oral Assessments: Enhancing Integrity, Engagement, and Communication in the AI Era

Synthesis: Pentland, Lowenthal & Krier (2026) evaluate Asynchronous Oral Assessments (AOAs) — web-based assessments in which prompts are delivered just-in-time, students record brief, time-limited webcam responses that cannot be revisited, and instructors grade against embedded rubrics with auto-generated transcripts. Across two studies (intermediate accounting pilot; data analytics course), students scored higher on AOAs than on in-person multiple-choice exams (Study 2, significant; Study 1, positive but non-significant trends), with moderate cross-format correlations supporting convergent validity. Students reported preparing differently for AOAs, using more active study strategies, and perceiving AOAs as professionally relevant and cognitively engaging. The paper positions AOAs as an administratively scalable complement to traditional Assessment that preserves the authenticity and communication value of oral exams while addressing integrity concerns in the AI era.

Core Finding

AOAs offer a scalable, authentic alternative that performs comparably to — and in Study 2 significantly better than — in-person multiple-choice exams, while shifting students toward more active, deeper preparation and perceived professional relevance. Students performed significantly better on AOAs than on in-person MCQs (midterm AOA Md = 92.5 vs MC Md = 70, p < .001; final AOA Md = 94.2 vs MC Md = 86.4, p = .002), with moderate correlations between formats (midterm τ = .44; final τ = .25, non-significant). Importantly, these are assessment-format score differences, not necessarily evidence of learning gains per se.

Asynchronous oral assessments as a scalable assessment format

Traditional oral assessments offer authentic measurement, deeper engagement, and professional communication development, but are underutilized because of logistical demands (scheduling, instructor time, space). Authentic AOAs remove these constraints by delivering questions just-in-time and collecting time-limited, non-revisitable recorded responses via standard webcams, graded against embedded rubrics with auto-generated transcripts. Both studies were administered to all enrolled students by a single instructor without scheduling constraints. The delivery mirrors asynchronous video interviews (AVIs) used in personnel selection, drawing on high-structure interviewing principles (randomized questions, short prep windows ~30 s, brief 2–3 min responses) that support reliability and reduce bias.

Authenticity, professional communication, and engagement

A core theme is that AOAs function as authentic assessment that builds transferable competencies. A large majority of students reported that AOAs are more closely aligned with workplace communication skills than traditional written exams (90.91%), that AOAs improved their ability to communicate clearly and concisely (90.91%), and that AOAs helped them engage more actively with content (81.82%). The format is framed within Universal Design for Learning (UDL), providing alternative modalities for expression and multiple pathways to success. Students reported shifting toward more active, cognitively demanding preparation strategies — speaking aloud, self-quizzing, organizing ideas — suggesting AOAs stimulate deeper engagement than passive review for fixed-response exams.

Academic integrity in the AI era

AOAs require real-time verbal reasoning and on-the-spot thinking, emphasizing the learner's own voice — a format that is harder to cheat on than written exams and more resistant to generative AI substitution, since recorded, time-limited spoken responses cannot easily be generated by AI tools designed for text. The paper frames AOAs as a promising integrity-protective supplement in an era of AI-assisted text, though it notes the studies did not directly measure cheating behavior — the integrity argument is inferential rather than measured.

Performance, validity, and the learning-gains caveat

Study 1 found positive but non-significant associations between AOA participation/performance and MCQ performance (trends directionally consistent; small effect sizes). Study 2 found significantly higher AOA scores and moderate cross-format correlations. The authors interpret this as convergent validity comparable to prior oral-vs-written research (r = .66, .41, .53) and position AOAs as a valid alternative. Caveat: the higher AOA scores reflect assessment performance on the AOA format (open-ended oral, moderate correlation with MCQ) rather than demonstrated learning gains; higher scores on one format do not establish improved learning or achievement. This paper should not be read as evidence of Learning Gains from AOAs, only that students performed better on the AOA format itself.

Scalability, grading, and AI-assisted scoring

Administratively, AOAs scale well, but grading remains labor-intensive — the instructor still reviewed each response individually despite auto-generated transcripts and embedded rubrics. As a reliability check, the authors re-scored Study 2 responses with an LLM (Anthropic Claude 3.5 Sonnet), finding instructor scores systematically higher but moderate-to-good agreement (ICC = 0.73 midterm, 0.60 final) and consistent relative ordering. This is framed solely as a reliability check, not an endorsement of automated grading; transparency, bias, and fairness in AI-assisted scoring are flagged as beyond scope.

Relevance to the knowledge base

This paper is a significant empirical contribution to the knowledge base's Assessment, Authentic Assessment, and Academic Integrity threads — it offers concrete evidence and a working design for an AI-era assessment format that is scalable, authentic, and integrity-protective. It extends the knowledge base's move from AI detection toward assessment design that recognizes AI's role, providing a practical alternative to fixed-response exams. Its Student Engagement findings (shifted preparation strategies, perceived professional relevance) connect assessment design to engagement and professional skill development, and its LLM re-scoring data speak to Assessment Validity and AI-assisted evaluation. Note the learning-gains caveat: score differences across formats should not be over-interpreted as learning improvement.

Connected Concepts

Connected Articles

Beyond Detection Authentic Assessment AI 2025 — Beyond detection: authentic assessment in an AI-mediated world
Roe Assessment Twins 2026 — Assessment twins: strengthening assessment validity in the age of generative AI
Bassett AI Detectors Education 2026 — AI detectors in education and the case against detection-based integrity
Llms Do Not Grade Essays Like Humans 2026 — LLM-based grading and the limits of automated assessment

Citation

Pentland, S. J., Lowenthal, P. R., & Krier, K. (2026). Asynchronous oral assessments: Enhancing integrity, engagement, and communication in the AI era. Educational Technology Research and Development.

Source History Edit Chat