A cultural-historical activity theory-anchored learning analytics pipeline for early detection and social-epistemic

Synthesis: Moon (2026) develops a Cultural-Historical Activity Theory (CHAT)-anchored learning analytics pipeline that couples three facets — temporal participation, discourse quality, and concept sophistication — each mapped to CHAT activity-system elements, to support early detection and social-epistemic integration in small discussion-based AI literacy classes. Deployed in a five-week AI literacy course (n = 25; 438 posts), DTW-based k-medoids clustering identified two stable participation archetypes (validated via bootstrap silhouette analysis); discourse quality showed modest lexical gains while reasoning depth stagnated and overall concept sophistication declined (Δz = –1.24). An Isolation-Forest detector flagged Week 2 volatility two weeks before participation decline, and automated discourse-quality scores converged with human-coded ICAP levels. The study demonstrates how a theory-aware, effect-size-centered workflow delivers interpretable insights and early warnings in a single small class without network-reconstruction overhead.

Key Findings

• A CHAT-mapped analytics design. The pipeline maps three analytic facets to CHAT elements: temporal participation (Subject/Community), discourse quality (Division of Labor/Rules), and concept sophistication (Object/Tools). This grounds learning-analytics features in the activity system rather than treating them as generic engagement metrics.
• Two stable participation archetypes. DTW-based k-medoids clustering (with bootstrap silhouette validation) identified two archetypes in the 25-student, 438-post course. "Reserved Observers" showed persistent behavior–episteme mismatches, signaling a Subject–Community tension.
• Shallow gains, deep stagnation. Discourse quality showed modest lexical-polish improvements, but reasoning depth stagnated and overall concept sophistication declined (Δz = –1.24) — evidence that conversational activity did not translate into conceptual growth.
• Early-warning detection. An Isolation-Forest detector flagged Week 2 volatility two weeks before a participation decline, robust across contamination settings 0.05–0.25 — a practical early-warning signal for instructors.
• Validity via ICAP alignment. Automated discourse-quality indices converged with human-coded ICAP levels, supporting substantive alignment between the CHAT-anchored analytics and an established engagement taxonomy.

What this means for practice

• Instructors. Track concept sophistication alongside participation, not instead of it: volume and depth were only loosely coupled, and discourse showed modest lexical improvement while reasoning depth stagnated and concept sophistication declined (Δz = –1.24).
• Instructors. Treat the Week 2 volatility flag as a trigger to intervene, since the Isolation-Forest detector signaled the drop two weeks before participation fell — enough lead time to add just-in-time scaffolding.
• Instructors. Re-read "Reserved Observers" as a Subject–Community tension rather than disengagement: this archetype posted consistently but showed persistent behavior–episteme mismatches.
• Researchers. Adopt the CHAT-anchored, effect-size-centered workflow for small courses — it yields interpretable indices without large-sample statistics or full network reconstruction — but treat it as proof-of-concept rather than a generalizable model.
• Researchers. Re-weight discourse-quality indices toward reasoning depth over surface lexis if AI-assisted polish is suspected, rather than accepting lexical gains as evidence of learning.

Limitations

• The pipeline was deployed in a single five-week AI literacy course with 25 undergraduates; the authors present it as a proof-of-concept and call for replication with different noise models and platforms.
• All data are text and platform logs (438 posts, 7,033 time-stamped events); the authors note that multimodal evidence could yield richer cognitive-engagement indicators than text alone.
• Students were not directed to use AI writing assistants, so LLM-augmented surface polish remains an uncontrolled confound, and only 30% of posts (n = 132) were hand-coded for ICAP to check automated indices.
• The pipeline operates only at the descriptive-diagnostic level; the early-warning signal was not coupled to any automated or instructor scaffold, so the study measured no effect of the analytics on learning.

Citation

Moon, J. (2026). A cultural-historical activity theory-anchored learning analytics pipeline for early detection and social-epistemic integration in AI literacy education. Interactive Learning Environments (in press).