Engagement in LLM Chatbot-Supported Learning: The Pivotal Roles of GenAI Competency and Emotion

Synthesis: Huang and Zhang (2026) propose and test a theoretically grounded model of student engagement in Large Language Models (LLMs) chatbot-supported learning, guided by appraisal theory. Unlike prior studies that focus on isolated antecedents, the model integrates learners' capacity appraisals (GenAI competency and academic self-efficacy), in-situ cognitive appraisals (perceived usefulness and ease of use of the chatbot), and affective appraisals (emotions experienced when using the chatbot). Based on 234 responses analyzed with structural equation modeling and mediation analyses, the study finds that GenAI competency had the strongest relationship with student engagement, emotion fully mediated the effect of perceived usefulness on engagement, and academic self-efficacy showed no significant direct association after accounting for other variables.

Key Findings

• GenAI competency had the strongest relationship with student engagement in chatbot-supported learning.
• Emotion fully mediated the effect of perceived usefulness on engagement.
• Perceived usefulness partially mediated the effect of perceived ease of use.
• Academic self-efficacy showed no significant direct association with engagement after accounting for other variables.
• The findings highlight the critical role of GenAI competency and positive emotions in chatbot-supported learning.

What this means for practice

• Learning designers. Build GenAI competency training into chatbot-supported courses as a prerequisite rather than an add-on: competency had the strongest relationship with engagement in the model.
• Learning designers. Design for emotional experience alongside perceived usefulness; emotion fully mediated the effect of usefulness on engagement, so enjoyment, confidence, and curiosity need deliberate cultivation rather than being left to chance.
• Learning designers. Do not assume academic self-efficacy will carry engagement — it showed no significant direct association once other appraisals were accounted for — and instead target the appraisals that do predict it.
• Researchers. Extend appraisal-theory models to include contextual factors and specific GenAI features, given that this study focused on individual-level factors.

Limitations

• The model was tested on 234 self-reported survey responses collected at a single time point, so no temporal dynamics or causal pathways between appraisals and engagement can be established.
• All measures are self-report and subject to social desirability and inaccurate Self-Assessment; the authors used Harman's single-factor test to check common method variance but did not add observational or multi-source data.
• The model covers individual-level factors only, leaving contextual conditions and chatbot design features outside the analysis.
• Task-embedded survey responses were analyzed as independent units rather than as data nested within individuals, so within- versus between-person effects remain untested.

Citation

Huang, X., & Zhang, S. (2026). Engagement in LLM chatbot-supported learning: The pivotal roles of GenAI competency and emotion. Computers and Education: Artificial Intelligence, 10, 100559.