🧠 AI Ed Wiki

πŸ“„ Research Article

Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement

● high Β· 2026-07-29self-regulated-learning, generative-ai, higher-ed, student-experience, engagement-metrics, efficacy-study, scaffolding, metacognition, llm

Summary

A randomized 2Γ—2 full-factorial field experiment (N = 179 German university students, 22 days of app use, 12-week follow-up) testing two design principles for a mobile chatbot-based learning journaling system aimed at keeping students motivated to maintain reflective learning journals β€” a known pain point (rapid decline in motivation/engagement after brief use). The two principles: (1) an example-based built-in course (7 days, one SRL topic per day, time-gated, modeled example responses) and (2) an LLM-based journaling assistant (GPT-3.5-turbo-1106) that scaffolds entries by summarizing drafts, asking clarifying follow-up questions, and generating alternative first-person formulations.

Design & method

  • Groups: Baseline (B, n=53) | Assistant (A, n=53) | Course (C, n=52) | Course+Assistant (CA, n=52); stratified randomization (gender, age, LIST-K SRL scales).
  • Measures: Intrinsic Motivation Inventory (IMI: enjoyment, perceived choice, pressure, competence, effort); LIST-K (SRL: cognition, metacognition, internal/external resource strategies) at pre/post/12-week follow-up; behavioral engagement = characters written per journal prompt (7,286 responses; 1,904 entries; mean 10.64 entries/student).
  • Engagement analyses: multiple regression + mixed-effects robustness check; outliers trimmed (top 1% and single-word messages removed β†’ 5,181 messages, M = 71.87 chars).

    • Key findings

    Intrinsic motivation (H1/H2)

  • Course β†’ motivation: SUPPORTED. Small significant effect on enjoyment (Ξ·Β² = 0.03, F(1,153) = 4.81, p < .05) and perceived competence (Ξ·Β² = 0.04, F(1,154) = 5.77, p < .05). No effect on perceived choice or pressure.
  • Assistant β†’ motivation: NOT SUPPORTED. No significant effect on enjoyment (p = .67) or competence (p = .95); no interaction between features. Even usage-days analysis found no competence effect (t = 1.95, p = .054).

    • Behavioral engagement (H3/H4)

  • Both features SUPPORTED. Regression on characters written: Assistant B = 18.32, Course B = 22.95, with a significant negative interaction (CourseΓ—Assistant B = βˆ’15.21*) β€” mean message lengths B 51.21 β†’ A 71.25 / C 79.26 / CA 83.04.
  • Distinct mechanisms: the course's effect was constant (unrelated to course days completed), while the assistant formed a positive feedback loop β€” more assistant use predicted longer messages over time (B = 6.20 per assistant day, p < .001), echoing the social-cognitive modeling account (Schunk): the assistant's model adapts, the course's static examples do not.

    • Course benefit is temporary

  • Course users were far more likely to be "early" writers (OR = 3.45, p < .001) β€” less reliant on the 9 PM notification β€” but significantly fewer reached 10+ journal days (OR = 0.38, p < .001): most completed the 7-day course, journaled one more day, then stopped. Static one-off scaffolding stimulates early activity but does not sustain it.

    • SRL development

  • All groups (including baseline) significantly increased cognitive and metacognitive strategy use from pre to 12-week follow-up (p < .05) β€” the structured prompting concept itself supported SRL, unlike earlier structured-journal studies.
  • 32 of 97 users reported the auto-generated summaries helped them reflect on prior entries (an unprompted purpose).

    • Implications

  • For reflective-learning-tool design: pair one-off course-style scaffolding with recurring/adaptive support (follow-up prompts, phase-specific guidance, timely interventions) to sustain engagement; the LLM assistant is the more promising candidate for durable engagement dynamics, but needs to be used (only 55.9% used it post-onboarding β€” a self-selection caveat).
  • Engagement measured as text length only; cognitive engagement/reflection quality untested. Implementation is a single instantiation; seasonal/semester effects possible; long-term effects beyond 3 weeks unverified.

    • Connected Concepts

  • Generative AI
  • Higher Ed
  • Metacognition
  • Scaffolding
  • Self Regulated Learning
  • Student Experience
  • LLM

    • Connected Articles

  • A4l Analytics Pipeline β€” Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D...
  • Aaai2026 Prompting Literacy K12 β€” Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module
  • Academiclaw Student Agent Benchmark β€” AcademiClaw: When Students Set Challenges for AI Agents
  • Access Not Enough AI Tutoring 2026 β€” Access is Not Enough: Human Support Improves Engagement with AI Tutoring
  • Adapt Adaptive Lesson Plan Transformer β€” AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction
  • Adaptive Pretesting Retention β€” Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study
  • Affective Text Wearable Student Health β€” A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring
  • Agency Gap AI Writing β€” The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning
  • Agent Voice Accents K12 Group Learning β€” Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning
  • Agentic AI Education Scoping Review β€” Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm
  • Agentic AI Pedagogical Best Practice 2026 β€” Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning
  • Agentic Education Coding β€” Agentic Education with AI Coding Assistants
  • Agentic Literacy Debt β€” Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named
  • Agents That Teach Incidental Learning β€” Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development
  • Agreement Not Quality LLM Coding Verification β€” Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G...
  • AI Adoption Training Public Sector β€” The Main Barrier to AI Adoption in the Public Sector is Lack of Training
  • AI Adult Learning Guidelines Dis2026 β€” Guidelines for Designing AI Technologies to Support Adult Learning
  • AI Agents Constructive Conflict Design Education 2026 β€” Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers
  • AI Agents Peer Learning Discourse β€” When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community
  • AI Assessment Human Tutors β€” AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice
  • AI Assessment Scale Reform β€” A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform
  • AI Assistance Discretionary Feedback β€” AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education
  • AI Assisted Learning Modes Eeg β€” An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig...
  • AI Assisted Se Curriculum Syllabus Analysis 2026 β€” Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis
  • AI Assisted Writing Research Teams β€” Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams

    • Citation

    Scheu, S., Loeffler, S. N., & Maedche, A. (2026). Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement. International Journal of Educational Technology in Higher Education, 23, 15