To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks

Created: 2026-06-30 | Tags: formative-assessmenthigher-edllmstem-educationstudent-experience

Hutchison, Applebaum, Angelikas, Patel et al. (2026) โ€” cs.HC (ITiCSE 2026) ๐Ÿ“„ Full text (arXiv)

Hutchison et al. (2026) develop and validate a method for measuring critical engagement with AI code completion tools in educational settings. Using behavioral signals (time-to-accept, edit distance from suggestion) and embedded attention checks, they find that the majority of students accept AI code suggestions passively, without critically evaluating correctness or appropriateness. This 'tab-and-go' behavior directly threatens the development of programming skills, as students bypass the cognitive effort required for ai-learning-transfer. The work provides a methodological toolkit for formative-assessment in AI-augmented programming courses, enabling instructors to detect when students are over-reliant on AI. The findings connect to student-experience research in stem-education by showing that the mere availability of AI tools does not lead to productive learning โ€” structured pedagogical interventions are required to ensure students engage critically rather than deferring to AI output.

Related Pages

Citation

APA: Jessica Hutchison, Ian Tyler Applebaum, Kenneth Angelikas, Kush Rakesh Patel, Phuoc Nguyen, Antonio Lazaro, Nicholas Rucinski, Rahad Arman Nabid, Stephen MacNeil (2026). To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks. arXiv:2606.30549. cs.HC (ITiCSE 2026).