📄 Full text: arXiv:2604.01114 · local
Pitts, Rani & Mildort (2026, AIED) show with 432 undergraduates that higher trust in an AI assistant is associated with lower appropriate reliance: students who trusted the assistant more were worse at discriminating correct from misleading AI suggestions during Python problem-solving. The relationship is non-linear and moderated by AI literacy and need for cognition — trust is not a safe proxy for appropriate use.
The experiment
- 432 undergraduates solved Python output-prediction problems with recommendations + explanations from an AI chatbot that included accurate and intentionally misleading suggestions
- Appropriate reliance measured behaviourally: accepting correct suggestions, rejecting incorrect ones
- Surveys captured trust, AI literacy, need for cognition, programming self-efficacy, programming literacy
Findings
- Non-linear trust→reliance relationship: higher trust → lower appropriate reliance (weaker discrimination between correct/incorrect recommendations)
- Moderators: AI literacy and need for cognition significantly shaped how trust translated into reliance behaviour
- Implication: interventions should target calibration — instructional and system supports that encourage reflective evaluation of AI assistance during problem-solving
Connections to the wiki
- Behavioural evidence for the over-reliance mechanism: trust without verification is the failure mode
- Links ai-literacy to actual reliance behaviour — literacy is a moderator, not just a stated competency
- Complements learner-ai-interaction-patterns-oop (how learners actually interact with AI) with a controlled causal-mediation design
- The reflective-evaluation recommendation matches the critical-thinking goals of critical-thinking and the verification-gated designs in tool-invariant-framework-agentic-ai
- Same calibration problem as measuring-llm-tutors-teach-vs-solve: learners must judge when AI output deserves adoption
- Extends student-ai-interaction with trust dynamics; ties to metacognition via the monitoring needed to reject misleading output
Related Pages
- ai-literacy — the moderating competency
- over-reliance — the failure mode being measured
- cs-education — Python problem-solving context
- student-ai-interaction — trust dynamics in learner-AI use
- metacognition — reflective evaluation of AI output
- learner-ai-interaction-patterns-oop — real-world usage patterns
- critical-thinking — discriminating correct from misleading output
- measuring-llm-tutors-teach-vs-solve — judging when AI output deserves adoption
- tool-invariant-framework-agentic-ai — verification-gated designs
- trust-calibration — the design goal for AI assistance
Sources
- Pitts, G., Rani, N., & Mildort, W. (2026). Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators. arXiv:2604.01114. AIED 2026.