Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic (2026) — Monash University / Tsinghua University. arXiv preprint (cs.HC, cs.AI).
This study embedded undisclosed AI agents as teammates in synchronous text-based group interactions across analytical, creative, and ethical tasks with 786 participants making 1,572 identity judgments. The central finding is striking: humans cannot distinguish AI from human teammates above chance levels. This failure is not due to a lack of identity-relevant information — computational models could accurately classify AI vs. human from conversational behavior — but because participants relied on flawed suspicion heuristics (response speed, fluency, perceived scriptedness) that were only weakly correlated with actual identity.
The implications for education are significant. As AI agents increasingly participate in student group work, online discussions, and peer learning environments, students may interact with AI without awareness. This creates vulnerabilities explored in eduframetrap-llm-sycophancy-educational-safety — if students cannot detect AI teammates, sycophantic AI could reinforce misconceptions unchallenged. The findings also complicate hybrid-human-ai-tutoring-differentiated models that rely on transparent role differentiation. The work underscores the urgent need for ai-literacy curricula that teach not just how to use AI, but how to recognize when one is interacting with it. The dissociation between behavioral truth and human perception suggests that over-reliance on AI may be exacerbated when source identity is concealed.
Related Pages
- ai-literacy — Humans fail to detect AI teammates at above-chance rates
- over-reliance — Social fluency creates new manipulation vulnerabilities
- student-experience — Students in online groups may unknowingly interact with AI agents
- eduframetrap-llm-sycophancy-educational-safety — AI identity concealment compounds sycophancy risks
- llm — Socially fluent LLMs decouple conversational signals from identity
- hybrid-human-ai-tutoring-differentiated — Implications for human-AI teaming in educational contexts
- llm-detecting-llm-generated-content-education — Distinguishing Artificial from Authentic: Evaluating LLMs fo
Citation
APA: Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic (2026). Socially fluent AI decouples conversational signals from source identity in online interaction. arXiv:2605.23426. arXiv preprint (cs.HC, cs.AI).
- humanlike-ai-collaborative-writing — AI identity detection gap compounds costs of humanlike interface design