---
source_url: https://arxiv.org/abs/2604.14807
ingested: 2026-05-07
sha256: bddf61f82ca37fc1133a1d50a0ea1a7fc2afad37b8c1b22ec09e7d72adfeaa55
---

# The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows

**arXiv:** 2604.14807v2 [cs.AI] | **Date:** 28 Apr 2026  
**Authors:** Hyunwoo Kim, Harin Yu, Hanau Yi | **Affiliation:** ddai Inc.

---

## Abstract

The rapid integration of large language models (LLMs) into everyday workflows has transformed how individuals perform cognitive tasks such as writing, programming, analysis, and multilingual communication. This paper introduces the **LLM fallacy**, a cognitive attribution error in which individuals misinterpret LLM-assisted outputs as evidence of their own independent competence, producing a systematic divergence between perceived and actual capability. The opacity, fluency, and low-friction interaction patterns of LLMs obscure the boundary between human and machine contribution. We propose a conceptual framework and a typology of manifestations across computational, linguistic, analytical, and creative domains. We examine implications for education, hiring, and AI literacy.

---

## Core Definition

> "The LLM fallacy is defined as a cognitive attribution error in which individuals misinterpret LLM-assisted outputs as evidence of their own independent competence."

Produces systematic **divergence between perceived and actual capability** (∆C). Independent of output correctness — persists whether LLM output is accurate or erroneous.

## Distinctions from Related Phenomena

| Concept | Focus | LLM Fallacy Focus |
|---|---|---|
| **Hallucination** | System output failure | How outputs are *cognitively interpreted* by the user |
| **Automation Bias** | Over-reliance on system outputs | Self-perception of *personal competence* derived from outputs |
| **Cognitive Offloading** | Delegating mental effort to tools | Integration of outputs into user's *identity and capability attribution* |

## Mechanisms of Emergence

**System-Level Properties:**
- **Opacity:** Hidden retrieval, pattern matching, and synthesis obscure division of labor
- **Fluency:** Grammatically correct, coherent outputs act as metacognitive cues; users infer competence from surface ease
- **Interactional Immediacy:** Rapid response cycles bias toward fast, intuitive judgments

**Cognitive Mediation:**
- **Attribution Ambiguity:** The boundary between user contribution and system generation becomes difficult to delineate; authorship inferred from outcomes
- **Cognitive Outsourcing:** As system assumes more cognitive workload, users engage less with underlying reasoning, weakening ability to assess own understanding

**Formal Relationship:**
> "capability divergence (∆C) emerges from the interaction of system-level properties (opacity, fluency, immediacy), mediated by attribution ambiguity and cognitive outsourcing."

## Necessary Conditions
1. Task involves LLM-mediated output generation requiring domain expertise
2. Interaction is sufficiently seamless that human/system boundary is not salient
3. Output exhibits fluency associated with skilled human performance

## Typology of Manifestations

| Domain | Description |
|---|---|
| **Computational** | Users produce functional code without understanding architecture; execution misinterpreted as technical competence |
| **Linguistic** | Fluent text generated in languages user doesn't independently command |
| **Analytical** | Structured explanations create impression of reasoning skill when reasoning is externally generated |
| **Creative** | Narratives misattributed as personal creativity despite substantial system contribution |
| **Epistemic** | Summaries lead users to equate *access to information* with *conceptual mastery* (illusion of explanatory depth) |
| **Professional Signaling** | Resumes, interviews, portfolios reflect ability to produce LLM-assisted outputs rather than independent expertise |

## Institutional Implications

**Education:** Assessment validity compromised; completed assignments may reflect LLM scaffolding rather than learning. AI assistance can improve short-term performance while weakening relationship between performance and competence.

**Hiring:** Observable outputs are unreliable proxies for competence when AI mediation is invisible. Evaluators (human and automated) conflate system-assisted performance with independently grounded skill.

**AI Literacy:** Need for interventions that help users correctly attribute contribution and calibrate self-assessment.