Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks Park, Orozco Vasquez, & Conati (2026) — University of British Columbia. Accepted at EDM 2026. 📄 Full text (arXiv)
Summary
This study extends prior work on student LLM use by analyzing data from two offerings of a research-oriented course where students learn to read, reason about, and critique academic papers — a setting that moves beyond the problem-solving domains that dominate existing research. Crucially, students had no restrictions on LLM usage, providing ecological validity.
Key contributions:
1. Refined bottom-up categorization of LLM usage types in academic critical thinking, cross-labeled by the extent of student initiative — from passive (copy-pasting text for summaries) to active (using LLM as a Socratic dialogue partner for critical-thinking-genai-scaffolding).
2. Learning outcome analysis linking usage frequency and type to performance on three midterm exams. This addresses the core question raised by genai-performance-vs-learning: do specific LLM usage patterns help or hinder actual learning?
3. The student initiative dimension is particularly valuable for understanding ai-literacy development — it maps onto the distinction between using AI as a crutch vs. as a cognitive tool, directly relevant to scaffolding design.
This work complements genai-tutor-engagement-patterns by shifting focus from tutoring to student-initiated LLM use in authentic academic tasks. The EDM 2026 acceptance places it within the learning-analytics community's growing interest in modeling AI-augmented learning behaviors. Findings also inform faculty-development-genai strategies for guiding student AI use.
Related Pages
- genai-performance-vs-learning — Performance vs. learning with AI
- critical-thinking-genai-scaffolding — Scaffolding critical thinking
- genai-tutor-engagement-patterns — Student engagement patterns with AI tutors
- ai-literacy — AI literacy competencies
- learning-analytics — Learning analytics frameworks
- faculty-development-genai — Faculty development for GenAI
- scaffolding — Instructional scaffolding