AIEd Knowledge BaseAIEd KB
On this page

Research Article

AI Advice Suppresses People's Willingness to Say "I Don't Know", Even When the Advice Is Wrong and Accuracy Is Incentivized

created 2026-08-21 · updated 2026-08-21 · cognitive offloading, metacognition, critical thinking, ai literacy, trust calibration, reducing ai misuse, bias mitigation, assessment, human ai collaboration, generative ai

Core Finding

In five experiments (N = 3,132; four preregistered, one direct replication), merely having access to AI advice nearly eliminated people's willingness to suspend judgment — to say "I don't know" — even when the AI advice was systematically wrong and accuracy was rewarded. Participants answered more questions but were correct about a third as often as when Generative AI was unavailable, while their confidence nearly doubled. The effect persisted whether advice was actively requested or automatically displayed, and monetary incentives only partly corrected it. AI access doesn't just change what people answer; it changes the metacognitive threshold at which they decide they know enough to answer.

Key Findings

  • Availability of AI collapses judgment suspension. In Studies 1a–1b, participants who could seek AI advice suspended judgment far less than baseline (0.06 vs. 0.36; 0.03 vs. 0.44). The replication ruled out tool malfunction as an explanation.
  • Stakes don't restore suspension. Monetary incentives for accuracy did not moderate the AI effect on suspension in Studies 2–4 (the pre-registered AI × stakes interaction was non-significant: p = .312, .506, .784). AI availability and stakes acted largely independently.
  • But incentives improve accuracy when AI is present. Rewarding accuracy made participants seek AI advice less (e.g., 5.44→4.93 and 5.27→4.53 of 6 questions) and improved correctness specifically in the AI-advice conditions — by encouraging people to override AI rather than by restoring "I don't know."
  • Confidence nearly doubled while accuracy fell. Without incentives, pooled correctness was 27.5% (no AI) vs. 9.2% (AI), while mean confidence jumped ~29.6 → 75.9. The AI advice converted some would-be-correct responses into errors, and divergent-but-wrong advice raised confidence instead of lowering it.
  • The effect survives unsolicited AI. Study 4 presented AI advice automatically (mirroring search summaries, writing assistants, autocomplete) and found the same near-elimination of suspension — no active consultation required.
  • It's a change in willingness, not capacity. Because incentives partly restored accuracy, people can still recruit their own judgment; AI lowers the threshold to respond rather than removing the ability to reason.

Why this matters for education

Although the study is framed around general human judgment (fine visual details in films), its mechanism operates on the availability and fluency of an answer, not its subject matter — so it applies directly wherever AI supplies a confident response, including Assessment, homework, tutoring, and information-seeking in learning. The authors explicitly note this bears on Cognitive Offloading, the erosion of human Agency, and "cognitive surrender." For education specifically, the findings suggest that the mere presence of AI may suppress learners' willingness to withhold judgment and acknowledge uncertainty — a core metacognitive and epistemic skill (Metacognition, Critical Thinking) — while inflating their confidence in wrong answers (Trust Calibration) and eroding Self Efficacy. It also implies that the goal of AI-literacy instruction may be less to teach tool skill than to preserve learners' readiness to recognize and act on the limits of what they know (AI Literacy, Reducing AI Misuse), a challenge that sits at the heart of Human AI Collaboration and AI Education.

Practical Implications

  • Design for "I don't know," not just correct answers. Assessments that reward any output, or environments where AI answers are always at hand, may discourage learners from withholding judgment under uncertainty. Build in structured opportunities and incentives for acknowledging uncertainty and suspending judgment (process documentation, calibrated-confidence prompts, metacognitive reflection) rather than only rewarding final answers.
  • Make consequences of errors salient. Monetary-style stakes (or real grade/accuracy consequences) reduced reliance on AI and improved accuracy when AI was present — though they did not restore suspension. Use assessment and feedback designs that reward accuracy over fluent completion.
  • Reduce unsolicited AI defaults. Since automatic AI suggestions produced the same collapse of judgment suspension, educators should be mindful of search-summary, autocomplete, and writing-assistant defaults that surface an answer before learners form their own judgment.
  • Teach verification and calibration, not just prompting. Instruction should target monitoring confidence, recognizing when to withhold, and overriding AI output — aligning with automation-bias and Trust Calibration research — rather than assuming technical AI literacy protects learners.
  • Extend AI-literacy toward epistemic judgment. The authors frame a research direction: whether teaching users how and when LLMs fail can restore judgment suspension and guard against cognitive surrender — a directly testable goal for AI-literacy curricula.

Connected Concepts

Connected Articles

Citation

Marcoccia, C., Quattrociocchi, W., & Capraro, V. (2026). AI advice suppresses people's willingness to say "I don't know", even when the advice is wrong and accuracy is incentivized.

SourceHistoryEditChat

Embed this page

Copy the code below to embed a chromeless version of this page in a learning management system or other website. The embedded view hides the site header, navigation, and footer.