AI Advice Suppresses People's Willingness to Say "I Don't Know", Even When the Advice Is Wrong and Accuracy Is Incentivized

Synthesis: In five experiments (N = 3,132; four preregistered, one direct replication), merely having access to AI advice nearly eliminated people's willingness to suspend judgment — to say "I don't know" — even when the AI advice was systematically wrong and accuracy was rewarded. Participants answered more questions but were correct about a third as often as when Generative AI was unavailable, while their confidence nearly doubled. The effect persisted whether advice was actively requested or automatically displayed, and monetary incentives only partly corrected it. AI access doesn't just change what people answer; it changes the metacognitive threshold at which they decide they know enough to answer.

Core Finding

Key Findings

• Availability of AI collapses judgment suspension. In Studies 1a–1b, participants who could seek AI advice suspended judgment far less than baseline (0.06 vs. 0.36; 0.03 vs. 0.44). The replication ruled out tool malfunction as an explanation.
• Stakes don't restore suspension. Monetary incentives for accuracy did not moderate the AI effect on suspension in Studies 2–4 (the pre-registered AI × stakes interaction was non-significant: p = .312, .506, .784). AI availability and stakes acted largely independently.
• But incentives improve accuracy when AI is present. Rewarding accuracy made participants seek AI advice less (e.g., 5.44→4.93 and 5.27→4.53 of 6 questions) and improved correctness specifically in the AI-advice conditions — by encouraging people to override AI rather than by restoring "I don't know."
• Confidence nearly doubled while accuracy fell. Without incentives, pooled correctness was 27.5% (no AI) vs. 9.2% (AI), while mean confidence jumped ~29.6 → 75.9. The AI advice converted some would-be-correct responses into errors, and divergent-but-wrong advice raised confidence instead of lowering it.
• The effect survives unsolicited AI. Study 4 presented AI advice automatically (mirroring search summaries, writing assistants, autocomplete) and found the same near-elimination of suspension — no active consultation required.
• It's a change in willingness, not capacity. Because incentives partly restored accuracy, people can still recruit their own judgment; AI lowers the threshold to respond rather than removing the ability to reason.

Why this matters for education

Although the study is framed around general human judgment (fine visual details in films), its mechanism operates on the availability and fluency of an answer, not its subject matter — so it applies directly wherever AI supplies a confident response, including Assessment, homework, tutoring, and information-seeking in learning. The authors explicitly note this bears on Cognitive Offloading, the erosion of human Learner Agency, and "Cognitive Surrender." For education specifically, the findings suggest that the mere presence of AI may suppress learners' willingness to withhold judgment and acknowledge uncertainty — a core metacognitive and epistemic skill (Metacognition, Critical Thinking) — while inflating their confidence in wrong answers (Trust Calibration) and eroding Self-Efficacy. It also implies that the goal of AI-literacy instruction may be less to teach tool skill than to preserve learners' readiness to recognize and act on the limits of what they know (AI Literacy, Reducing AI Misuse), a challenge that sits at the heart of Human AI Collaboration and AI in Education.

What this means for practice

• Researchers. Score judgment suspension itself rather than only correctness: assessments and experiments that reward any output miss the near-elimination of "I don't know" that AI availability produced (0.06 vs. 0.36 in Study 1a).
• Researchers. Build structured opportunities to acknowledge uncertainty — process documentation, calibrated-confidence prompts, metacognitive reflection — instead of only rewarding final answers, and treat unsolicited AI defaults (search summaries, autocomplete, writing assistants) as threats to that practice, since automatically displayed advice caused the same collapse.
• Researchers. Target instruction at verification and calibration — monitoring confidence, recognizing when to withhold, overriding AI output — rather than at tool skill, as the results imply technical AI literacy does not protect learners' readiness to suspend judgment.
• Researchers. Instrument verification behavior directly and test the boundary conditions: whether suspension collapses when AI advice is usually correct, and whether real-consequence stakes differ from the modest $0.10 accuracy incentives used here.

Limitations

• All five experiments (N = 3,132) used a single class of stimuli — fine visual details in films — deliberately chosen because they reliably induce hallucinations, so the AI was almost always wrong; whether judgment suspension collapses to the same degree when advice is usually correct is left open by the authors.
• Participants were US-based Prolific respondents paid a £0.30 base rate and $0.10 per correct answer in the stakes conditions; whether errors with real-world rather than monetary consequences behave the same is untested.
• The design did not record whether participants who overrode the AI consulted other sources such as the open Internet or another model, so successful overrides cannot be attributed to reduced AI reliance rather than increased use of alternatives.
• The incentives, though effective, were modest, and the authors state that whether larger or reputational stakes would close more of the gap to baseline remains unknown.

Citation

Marcoccia, C., Quattrociocchi, W., & Capraro, V. (2026). AI advice suppresses people's willingness to say "I don't know", even when the advice is wrong and accuracy is incentivized.