You've Got AI Friend in Me: LLMs as Collaborative Learning Partners

You've Got AI Friend in Me: LLMs as Collaborative Learning Partners — This exploratory classroom study (n = 154) tested whether Large Language Models can serve as collaborative learning partners. Across one semester, introductory social science students wrote argumentative essays, had them critiqued by LLMs such as ChatGPT, Gemini, or Claude, and either incorporated or rebutted the critiques. Students engaged deeply with the LLMs, enjoyed the work, and showed gains in argumentative writing, prompt engineering, response-to-feedback quality, and Self Efficacy for working with generative AI.

Key Findings

• Students improved on all three performance dimensions from the first to the fifth iteration of the assignment: quality of initial argument (2.58 → 2.90, t(109) = 6.22), prompt engineering (2.34 → 2.89, t(109) = 8.80), and response to AI Feedback (2.41 → 2.88, t(109) = 5.44); all p < .001, with improvement roughly a full standard deviation per dimension.
• Gains appeared even in essays written without LLM support. Students' initial-draft argument quality improved over the semester, suggesting skill development rather than mere tool dependency on the LLM — though causal attribution is limited by the lack of a control condition.
• Students engaged deeply rather than passively. Blind coders found evidence of reflection in 92.7%, rebuttal of LLM claims in 87.8%, and acceptance of feedback in 93.6% of responses (inter-rater κs = 0.81–0.89), indicating students acted as critical consumers who actively decided what to accept or refute.
• Self-efficacy for generative AI grew across the semester. Current self-efficacy was significantly higher after the fifth iteration than the first (t(177) = 4.25, p = .006), with the strongest within-assignment growth on the final iteration (t(97) = −5.65, p < .001).
• Affective responses were positive and durable. Students rated the assignment useful, engaging, enjoyable, and appropriately challenging at both the first and fifth iterations (all significantly above the neutral midpoint, ps < .002), with engagement and enjoyment stable across the semester.
• A qualitative sentiment analysis (run via GPT-4) classified 92.9% of general open-ended responses as positively valenced, reinforcing the quantitative affective results.

Connected Concepts

• Collaborative Learning
• LLM
• Human AI Collaboration
• Student AI Interaction
• Conversational AI
• Learning Gains

Connected Articles

• Song GenAI Learning Partner SRL Over Time 2026
• Student Perception AI Use Collaboration
• GenAI Counter Learner Groupthink 2025
• AI Feedback Critical Thinking Writing 2026
• Multi Agent LLM Social Learning

Citation

Oppenheimer, D. M., Cash, T. N., & Connell Pensky, A. E. (2025). You've Got AI Friend in Me: LLMs as Collaborative Learning Partners. International Journal of Artificial Intelligence in Education, 35, 3896–3921.