Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use

Synthesis: Large-scale analysis (N=11,406 students, 200 classes, 10 institutions) of GenAI tutor engagement identifies four session-level engagement types — Deep, Shallow, Routine-Learning, and Exam-Driven — with 10.4% of sessions being shallow copy-paste use and deeper engagement more common at selective institutions.

Summary

Large-scale analysis (N=11,406, 200 classes, 10 institutions) of GenAI tutor engagement. Four engagement types; 10.4% shallow with copy-paste. Students at selective institutions more likely to engage deeply.

This study analyzes de-identified interaction logs from a commercial LMS with an integrated GenAI Tutor used across ten post-secondary institutions during the Spring 2025 semester. Among 11,406 students, 6,932 (60.8%) engaged with the tutor at least once, generating a median of 5 conversation sessions per user across 113,255 segmented conversation sessions. Using clustering on behavioral, cognitive, and temporal session features, the authors identified four engagement types, then used process mining at the student level to examine how learners transitioned between them over time.

Key Contributions

• A two-stage learning-analytics pipeline that identifies conversation-session-level engagement types via clustering and aggregates them into student-level engagement patterns via process mining (First-Order Markov Model), applicable to other human-AI interaction data.
• Four session-level engagement types — Deep, Shallow, Routine-Learning, and Exam-Driven — grounded in behavioral, cognitive, and temporal features of student-GenAI conversations.
• Evidence on contextual variation across institution selectivity and course discipline, reflecting equity concerns about GenAI use in educational settings.
• Insights intended to inform learning-analytics interventions and institutional guidance for supporting student learning with GenAI Tutors in more effective and equitable ways.

The Four Engagement Types

• Deep Engagement (14.0% of sessions): multiple conversation turns, longer durations, more words, and a high prevalence of understanding-oriented queries; distributed relatively evenly across the semester.
• Shallow Engagement (10.4%): few turns, short durations, few words, with copy-pasting behaviors and direct answer-seeking requests prevalent; somewhat more likely to occur during class.
• Routine-Learning Engagement (44.5%): concentrated in daytime and the first half of the semester alongside coursework; few turns but reasonable word counts, with more understanding-oriented questions and fewer direct answer requests.
• Exam-Driven Engagement (31.2%): concentrated in the final weeks as exams approached; few, short turns with the least copy-pasting, suggesting fewer needs for formalistic homework-style answers.

Contextual Variation & Implications

Context mattered. At highly selective universities, the proportion of deep engagement (19.36% vs. 12.42%) and routine-learning engagement (58.88% vs. 40.41%) was significantly higher, while exam-driven engagement was lower (10.65% vs. 36.98%). STEM courses showed significantly more shallow (15.41% vs. 8.56%) and routine-learning engagement and less exam-driven engagement than non-STEM courses, while STEM students were overall less likely to adopt the tutor (51% vs. 71% adoption). Adoption and usage intensity were not significantly associated with institutional selectivity. For Learning Analytics and Equity in Higher Education, the takeaway is that "engagement" is not a single behavior: interventions and regulations should target specific patterns — such as shallow, copy-paste-heavy use — rather than treating all GenAI Tutor use alike, especially since students who engaged shallowly were more likely to remain in that mode over time, pointing to a risk group for Over-Reliance without deep processing.

What this means for practice

• Learners. Have at least one understanding-oriented exchange per session instead of pasting a prompt and copying the answer: shallow sessions were a minority (10.4%) but students who used the tutor that way seldom moved out of it later in the semester.
• Instructors. Design tasks so shallow use is not the path of least resistance. STEM courses, with their weekly closed-ended problems, showed significantly more shallow engagement (15.41% vs. 8.56%) than non-STEM courses.
• Administrators. Monitor engagement quality rather than adoption or usage volume — selectivity was not significantly associated with adoption or usage intensity, but it was with the mix of engagement types (deep engagement 19.36% vs. 12.42% at highly selective institutions).
• Researchers. Pair these interaction-log patterns with outcome measures. The authors state that identifying shallow engagement from copy-paste and answer-seeking indicators still requires validation against learning outcomes and other process measures.

Limitations

• The evidence is one semester (Spring 2025) of interaction logs from a single commercial LMS's GenAI Tutor, labeled from behavioral, cognitive, and temporal session features rather than from any learning outcome.
• The analyses cover classes and institutions with high GenAI Tutor adoption — 11,406 students in 200 classes across ten post-secondary institutions — so the estimates may not hold in settings where the tool is rarely used.
• The study has no data on students' use of general-purpose GenAI tools, and students who adopted the tutor may be more motivated than those who did not, which limits attribution of the patterns to the tutor itself.
• Shallow engagement was inferred from indicators such as copy-pasting and direct answer requests; the authors describe that identification as needing further validation.

Citation

Youjie Chen et al. (2026). Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use.