Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing

Identifies interaction signatures of LLM literacy using Epistemic Network Analysis (ENA) on logs from 162 students. High-literacy students exhibit iterative, strategic refinement and dense cognitive networking, while low-literacy students rely on direct, linear commands. This work emphasizes that AI Literacy is a developmental capacity requiring structured Scaffolding and Prompt Engineering discipline. It connects to the need for Curriculum Design that targets Metacognition and Agentic AI rather than just syntax mastery.

Key Findings

Interaction logs from 162 university students engaged in a GenAI-assisted abstract writing task were analyzed using Epistemic Network Analysis (ENA).

High-literacy students exhibit iterative refinement and strategic questioning, producing distinct interaction signatures in the process data.

Low-literacy students rely on direct generation commands, characterized by transactional, generation-oriented dependence on the tool.

GenAI literacy is not just a static score but a dynamic behavioral capability that shapes the human-AI collaboration process.

The work paves the way for data-driven literacy assessment and real-time interventions based on observed interaction behavior rather than self-report.

Study Design & Method

A total of 162 university students (M_age = 20.1) participated in a GenAI-assisted abstract writing task. Prior to the task, students completed a validated GenAI Literacy Test assessing technical understanding, interaction skills, and related constructs. Using Epistemic Network Analysis, the researchers modeled and compared the questioning strategies of students with varying GenAI literacy levels, examining how literacy manifests in the structure of actual human-AI collaboration. The study was presented at the First International Workshop on Advancing AI Literacy with Learning Analytics (AI-LIT) at LAK26.

Implications for AI in Education

The study demonstrates that process data can characterize GenAI literacy in ways that self-reported questionnaires cannot: whether a student actually prompts iteratively, refines outputs, and manages hallucinations in real time is observable in interaction logs. For Learning Analytics practice, this suggests building dashboards and automated assessments around behavioral markers of literate use — for example, detecting the shift from direct generation commands toward iterative refinement. The authors propose data-driven interventions that help learners move from transactional use to epistemic collaboration, such as prompting students to draft their own summary first and asking the AI for critique to improve learning depth. For Writing Education and AI Literacy instruction, the findings argue for curricula that treat literacy as a developmental, behaviorally observable capacity shaped by structured Scaffolding and strategic prompting, rather than a fixed trait measured once.

Connected Concepts

Connected Articles

Ase 26 Agentic Software Engineering Curriculum — ASE-26: A Curriculum for Agentic Software Engineering as a Discipline

Guided LLM Scaffolding Independent Learning — Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics

AI Adoption Training Public Sector — The Main Barrier to AI Adoption in the Public Sector is Lack of Training

Agency Gap AI Writing — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning

Finkelstein Principled AI Education 2025 — Principled AI Education Framework

Prober AI Inquiry Writing — Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing

Citation

Angxuan Chen & Jiyou Jia (2026). Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing. arXiv:2606.00040.