Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System

Synthesis: Recent work in Technology-Enhanced Learning and HumanComputer Interaction highlights the importance of transparency and trust calibration in AI-supported learning environments as they pose a risk of hallucinations. In this study, we investigate whether a simple transparency intervention that warns s

Abstract

Recent work in Technology-Enhanced Learning and HumanComputer Interaction highlights the importance of transparency and trust calibration in AI-supported learning environments as they pose a risk of hallucinations. In this study, we investigate whether a simple transparency intervention that warns students that a pedagogical agent may make mistakes affects learner behavior in a math intelligent tutoring system. We conducted a classroom experiment with 252 school students using two system versions: one including a warning message about potential system errors, and one that does not mention potential errors. Using log data, we analyzed students’ problem-solving performance data, including help-seeking behavior, error rate, and time-on-task. Results show that students who were warned about po

Connected Concepts

Help Seeking

Affective Tutoring

Knowledge Tracing

Teacher AI Competency

Socratic AI Dialogue

Pedagogical Agent

Automated Question Generation

Pedagogical LLM Training

Connected Articles

LLM Student Simulation Misconception Faithfulness — Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators

Rethinking Scaffolding LLM Tutors — Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments

Prompt Injection Defenses Educational LLM Tutors — Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs

Favero Critical AI Tutors Empower Enslave 2025 — Critical AI Tutors: Empower or Enslave?

Persistent AI Agents Academic Research — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study

GenAI Tutor Engagement Patterns — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use

Citation

Nagashima, T., Hladký, M., & Rief, V. (2026). Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System. arXiv:2606.03822.