Asynchronous Oral Assessments: Enhancing Integrity, Engagement, and Communication in the AI Era

Synthesis: Pentland, Lowenthal & Krier (2026) evaluate Asynchronous Oral Assessments (AOAs) — web-based assessments in which prompts are delivered just-in-time, students record brief, time-limited webcam responses that cannot be revisited, and instructors grade against embedded rubrics with auto-generated transcripts. Across two studies (intermediate accounting pilot; data analytics course), students scored higher on AOAs than on in-person multiple-choice exams (Study 2, significant; Study 1, positive but non-significant trends), with moderate cross-format correlations supporting convergent validity. Students reported preparing differently for AOAs, using more active study strategies, and perceiving AOAs as professionally relevant and cognitively engaging. The paper positions AOAs as an administratively scalable complement to traditional Assessment that preserves the authenticity and communication value of oral exams while addressing integrity concerns in the AI era.

Core Finding

AOAs offer a scalable, authentic alternative that performs comparably to — and in Study 2 significantly better than — in-person multiple-choice exams, while shifting students toward more active, deeper preparation and perceived professional relevance. Students performed significantly better on AOAs than on in-person MCQs (midterm AOA Md = 92.5 vs MC Md = 70, p < .001; final AOA Md = 94.2 vs MC Md = 86.4, p = .002), with moderate correlations between formats (midterm τ = .44; final τ = .25, non-significant). Importantly, these are assessment-format score differences, not necessarily evidence of learning gains per se.

Asynchronous oral assessments as a scalable assessment format

Traditional oral assessments offer authentic measurement, deeper engagement, and professional communication development, but are underutilized because of logistical demands (scheduling, instructor time, space). Authentic AOAs remove these constraints by delivering questions just-in-time and collecting time-limited, non-revisitable recorded responses via standard webcams, graded against embedded rubrics with auto-generated transcripts. Both studies were administered to all enrolled students by a single instructor without scheduling constraints. The delivery mirrors asynchronous video interviews (AVIs) used in personnel selection, drawing on high-structure interviewing principles (randomized questions, short prep windows ~30 s, brief 2–3 min responses) that support reliability and reduce bias.

Authenticity, professional communication, and engagement

A core theme is that AOAs function as authentic assessment that builds transferable competencies. A large majority of students reported that AOAs are more closely aligned with workplace communication skills than traditional written exams (90.91%), that AOAs improved their ability to communicate clearly and concisely (90.91%), and that AOAs helped them engage more actively with content (81.82%). The format is framed within Universal Design for Learning (UDL), providing alternative modalities for expression and multiple pathways to success. Students reported shifting toward more active, cognitively demanding preparation strategies — speaking aloud, self-quizzing, organizing ideas — suggesting AOAs stimulate deeper engagement than passive review for fixed-response exams.

Academic integrity in the AI era

AOAs require real-time verbal reasoning and on-the-spot thinking, emphasizing the learner's own voice — a format that is harder to cheat on than written exams and more resistant to generative AI substitution, since recorded, time-limited spoken responses cannot easily be generated by AI tools designed for text. The paper frames AOAs as a promising integrity-protective supplement in an era of AI-assisted text, though it notes the studies did not directly measure cheating behavior — the integrity argument is inferential rather than measured.

Performance, validity, and the learning-gains caveat

Study 1 found positive but non-significant associations between AOA participation/performance and MCQ performance (trends directionally consistent; small effect sizes). Study 2 found significantly higher AOA scores and moderate cross-format correlations. The authors interpret this as convergent validity comparable to prior oral-vs-written research (r = .66, .41, .53) and position AOAs as a valid alternative. Caveat: the higher AOA scores reflect assessment performance on the AOA format (open-ended oral, moderate correlation with MCQ) rather than demonstrated learning gains; higher scores on one format do not establish improved learning or achievement. This paper should not be read as evidence of Learning Gains from AOAs, only that students performed better on the AOA format itself.

Scalability, grading, and AI-assisted scoring

Administratively, AOAs scale well, but grading remains labor-intensive — the instructor still reviewed each response individually despite auto-generated transcripts and embedded rubrics. As a reliability check, the authors re-scored Study 2 responses with an Large Language Models (LLMs) (Anthropic Claude 3.5 Sonnet), finding instructor scores systematically higher but moderate-to-good agreement (ICC = 0.73 midterm, 0.60 final) and consistent relative ordering. This is framed solely as a reliability check, not an endorsement of automated grading; transparency, bias, and fairness in AI-assisted scoring are flagged as beyond scope.

Relevance to the knowledge base

This paper is a significant empirical contribution to the knowledge base's Assessment, Authentic Assessment, and Academic Integrity threads — it offers concrete evidence and a working design for an AI-era assessment format that is scalable, authentic, and integrity-protective. It extends the knowledge base's move from AI detection toward assessment design that recognizes AI's role, providing a practical alternative to fixed-response exams. Its Student Engagement findings (shifted preparation strategies, perceived professional relevance) connect assessment design to engagement and professional skill development, and its LLM re-scoring data speak to Assessment Validity and AI-assisted evaluation. Note the learning-gains caveat: score differences across formats should not be over-interpreted as learning improvement.

What this means for practice

• Instructors. Offer an optional, ungraded practice AOA with feedback before the graded version; practice participation was moderate (42–57%) but appeared to build familiarity and reduce anxiety when the format was new.
• Instructors. Randomize prompt selection and hold the timing to roughly 30 s of preparation and 2–3 min of response, aligning prompts with exam content — in Study 2 AOAs ran one week before each exam and counted 7.5% of the course grade, serving as graded assessment and retrieval practice at once.
• Assessment designers. Grade with criterion-referenced rubrics that specify performance levels and descriptive anchors; instructor and LLM scores showed ICC = 0.73 (midterm) and 0.60 (final) despite the instructor scoring systematically higher.
• Assessment designers. Build in accessibility features such as audio presentation of question prompts, since time-limited text displays can disadvantage students with reading-related disabilities.
• Instructors. Position AOAs as a complement to written work rather than a replacement, and do not treat the higher AOA scores as evidence of learning gains.

Limitations

• Sample sizes were N = 34 in Study 1 (26 of 62 students completed the practice AOA, and 35 of 62 the extra-credit AOA) and N = 30 in Study 2; the authors state that these small samples constrain statistical power and generalizability; study 1 participation was voluntary and incentivized with extra credit, so self-selection cannot be ruled out and the link between AOA completion and later exam performance should not be read causally.

• Study 1 prompts were partly misaligned with exam items, and only four multiple-choice cash-flow questions measured exam performance, limiting how much preparatory benefit the design could detect.

• Neither study measured cheating behavior; the integrity advantage of AOAs is inferred from the format's recorded, time-limited spoken responses rather than demonstrated.

• Both studies ran in two courses taught by a single instructor of the authors' own courses, with in-person comparisons limited to multiple-choice exams.

Citation

Pentland, S. J., Lowenthal, P. R., & Krier, K. (2026). Asynchronous oral assessments: Enhancing integrity, engagement, and communication in the AI era. Educational Technology Research and Development.