Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks

Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks Park, Orozco Vasquez, & Conati (2026) — University of British Columbia. Accepted at EDM 2026.

Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks

Summary

This study extends prior work on student LLM use by analyzing data from two offerings of a research-oriented course where students learn to read, reason about, and critique academic papers — a setting that moves beyond the problem-solving domains that dominate existing research. Crucially, students had no restrictions on LLM usage, providing ecological validity.

Key contributions:

1. Refined bottom-up categorization of LLM usage types in academic critical thinking, cross-labeled by the extent of student initiative — from passive (copy-pasting text for summaries) to active (using LLM as a Socratic dialogue partner for Critical Thinking GenAI Scaffolding).

2. Learning outcome analysis linking usage frequency and type to performance on three midterm exams. This addresses the core question raised by GenAI Performance Vs Learning: do specific LLM usage patterns help or hinder actual learning?

3. The student initiative dimension is particularly valuable for understanding AI Literacy development — it maps onto the distinction between using AI as a crutch vs. as a cognitive tool, directly relevant to Scaffolding design.

This work complements GenAI Tutor Engagement Patterns by shifting focus from tutoring to student-initiated LLM use in authentic academic tasks. The EDM 2026 acceptance places it within the Learning Analytics community's growing interest in modeling AI-augmented learning behaviors. Findings also inform Faculty Development strategies for guiding student AI use.

Connected Concepts

Connected Articles

Citation

Park, M., Orozco Vasquez, I., & Conati, C. (2026). Characterizing students' LLM usage behaviors and their association with learning in critical thinking tasks. In Proceedings of EDM 2026. arXiv:2605.04534.