# AI in Education Wiki — Full Content > Complete text of 594 articles and 133 concepts. # Articles ## [A framework for characterising and capturing the quality of digital interactions and experiences in early childhood education](https://edtechdev.github.io/aied/articles/a-framework-for-characterising-and-capturing-the-quality-of-digital-interactions/) > **Synthesis:** This study introduces a Digital Interactions Quality (DigIQ) framework and scale as a protocol to observe and index the quality of interactions and experiences involving digital technologies with children in Early Childhood Education and Care (ECEC) settings. Observations in 50 pre-school classrooms validated the framework, revealing that quality ratings were higher when learning intentions drove activity and digital technologies enhanced or transformed the learning experience rather than merely replicating non-digital activities. ## Key Findings This study introduces a Digital Interactions Quality (DigIQ) framework and scale as a protocol to observe and index the quality of interactions and experiences involving digital technologies with children in Early Childhood Education and Care (ECEC) settings. Observations in 50 pre-school classrooms validated the framework, revealing that quality ratings were higher when learning intentions drove activity and digital technologies enhanced or transformed the learning experience rather than merely replicating non-digital activities. ## Connected Concepts - [[rag]] - [[intelligent-tutoring]] - [[adult-learning]] - [[socratic-method]] - [[math-education]] - [[higher-ed]] - [[accessible-learning]] - [[human-in-the-loop-ai]] ## Connected Articles - [[ai-learning-assistants-higher-ed-large-scale]] - [[xai-education-framework]] - [[tech-enhanced-tabletop-cybersecurity-education]] - [[precision-education-student-digital-twins-2026]] ## Citation Howard, S. J., Lewis, K., Day, N., Peach, L., & Kervin, L. K. (2026). [*A framework for characterising and capturing the quality of digital interactions and experiences in early childhood education*](https://doi.org/10.1111/bjet.70063). *British Journal of Educational Technology.* --- ## [Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains](https://edtechdev.github.io/aied/articles/a4l-analytics-pipeline/) Artificial intelligence assistants deployed in online learning environments create new opportunities to collect large volumes of learner interaction data and generate insights to improve student outcomes. Architecture for AI-Augmented Learning (A4L) is a modular data architecture that enables the collection, integration, and analysis of learner interaction data from educational AI systems, supporting the generation of instructional insights that facilitate personalized learning and reinforce the bidirectional feedback loop between instructors and learners. This study examines the modular design of the A4L Data Analytics Pipeline, an extensible data infrastructure that enables the ingestion, processing, and analysis of heterogeneous datasets generated by educational AI assistants. We describe the design principles and development process used to extend the pipeline's analytical capabilities while preserving flexibility across domains. We evaluate the pipeline through case studies spanning three research domains corresponding to three educational AI assistants deployed in online learning environments at Georgia Tech. - **Reusable analytics infrastructure:** Bai et al. present the A4L Data Analytics Pipeline as modular, domain-agnostic infrastructure for analyzing learner interaction data from educational AI assistants. The pipeline is designed to ingest heterogeneous datasets across different courses and tutoring contexts without rebuilding analytics from scratch. - **Cross-domain validation:** The pipeline was evaluated through three case studies at Georgia Tech, each involving a different educational AI assistant deployed in real online learning environments. Results showed that a common set of statistical methods could be consistently applied across datasets with varying structures and instructional contexts. - **Extensibility demonstrated:** Analytical capabilities initially developed for one domain were successfully extended to support richer analyses in another domain, proving the pipeline's extensibility. This positions the A4L pipeline as reusable infrastructure for future [[learning-analytics]] systems. - **Bidirectional feedback loop:** The architecture supports a feedback loop between instructors and learners, enabling [[personalized-learning]] insights derived from AI-augmented learning data. This connects to broader conversations in [[edtech-platform]] design about how analytics infrastructure should scale across domains. - **EDULEARN26 publication** suggests growing academic interest in systematizing analytics for educational AI, complementing work on [[ai-assisted-writing-research-teams]] and [[llm-sentiment-analysis-education-research]] which explore different facets of AI-augmented education research. ## Connected Concepts - [[learning-analytics]] - [[personalized-learning]] - [[edtech-platform]] ## Connected Articles - [[ai-assisted-writing-research-teams]] - [[llm-sentiment-analysis-education-research]] ## Citation Yallen Bai, Ploy Thajchayapong, & Ashok Goel (2026). [Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains](https://arxiv.org/abs/2605.30303). arXiv:2605.30303. EDULEARN26. --- ## [Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module](https://edtechdev.github.io/aied/articles/aaai2026-prompting-literacy-k12/) > **Synthesis:** An LLM-based interactive module teaches K-12 students prompting literacy through scenario-based deliberate practice with an AI auto-grader providing immediate, detailed feedback. Deployed across 11 secondary classrooms in two iterations, the module improved students' prompting skills (particularly embedding background context) and confidence in using AI for learning. The study also validates an AI-based auto-grader achieving 0.92 average accuracy and identifies True/False + open-ended questions as more effective than MCQs for assessing prompting literacy. ## Study Design Xiao et al. designed and deployed a web-based interactive instructional system to teach **prompting literacy** to secondary-education students. The module was grounded in two learning sciences principles: **learning-by-doing** and **elaborated immediate feedback**. Students practiced prompt writing in three hypothetical learning scenarios (biology, geography, math), each paired with a unique instructional activity (extending knowledge, quiz preparation, homework struggle). After a student writes a prompt, an **LLM-based auto-grader** (GPT-4o) evaluates it across preset dimensions and delivers immediate, detailed feedback. The pipeline mirrors authentic AI chatbot interaction: write prompt → receive AI response → get graded feedback. Two iterations were conducted across 11 secondary classrooms: - **Study 1** (June 2024): 111 students, 6 classrooms in East Asia - **Study 2**: Assessment iteration follow-up with similar population ## The Auto-Grader The AI auto-grader achieved **0.92 average accuracy** across dimensions when grading student-written prompts, using human labels as ground truth: | Dimension | Accuracy | |---|---| | Relevance | 0.98 | | Background/Context | 0.96 | | Conciseness | 0.93 | | Elaboration | 0.90 | | No Direct Answer | 0.88 | | Clarity of Purpose | 0.85 | The lowest accuracy (Purpose, 0.85) stemmed from the auto-grader over-generating keywords or conflating Purpose with No Direct Answer criteria. The auto-grader tended to weigh heavily on some keywords while ignoring others — a known limitation of LLM-based grading. ## Key Findings ### Prompting Skill Improvement - Students improved significantly at **embedding background/context information** in prompts (McNemar test, p = .039 from Q1 to Q3) - Students performed well on Relevance, Conciseness, and Purpose even in the first question (ceiling effects) - **Prior AI usage frequency was positively correlated with initial prompt quality** (r = 0.27, p = .017), suggesting an equity concern ### Confidence and Perception - Self-reported confidence in using AI for learning **increased by 10.4%** (p < .001) - **87% of students** reported learning AI-related knowledge (how to use AI for learning, how to ask effective questions, AI's capabilities) - Students valued: direct AI interaction, scenario-based design, immediate comprehensive feedback, and visual elements ### Assessment Design Lessons - **MCQs suffered from ceiling effects** — students could identify good prompts conceptually but couldn't write them effectively - **True/False + open-ended questions** demonstrated better item difficulty and discrimination than MCQs - None of the original MCQ items fell into the desired difficulty range [0.3, 0.7], while 60% of OE and 30% of TF questions did ## Challenges Identified - **Productive struggles**: difficulty writing effective prompts (the core skill being taught) - **Extraneous load**: slow AI response times, login issues, limited typing skills (22 students reported this) - **Scenario variety**: some students wanted non-STEM scenarios - LLM response latency disrupted the practice flow ## Design Implications The study demonstrates that [[prompt-engineering]] can be taught effectively to K-12 students through structured practice with automated feedback. Key design principles: 1. **Scenario-based deliberate practice** with authentic AI interaction 2. **Immediate, dimension-level feedback** powered by LLM auto-grading 3. **Assessment aligned to competency** — open-ended + T/F outperform MCQs for higher-order prompting skills 4. **Addressing the digital divide** — prior AI access correlates with initial performance, underscoring the need for in-school prompting literacy instruction ## Connected Concepts - [[ai-education]] - [[ai-literacy]] - [[automated-grading]] - [[k-12]] - [[llm]] - [[prompt-engineering]] - [[student-experience]] - [[rag]] ## Connected Articles - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation Xiao, R., Hou, X., Tseng, Y.-J., Nieu, H., Liao, G., Stamper, J., & Koedinger, K. R. (2026). [*Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module*](https://doi.org/10.1609/aaai.v40i28.36924). AAAI. --- ## [The Absent Cognitive Baseline: Theorizing a Structural Gap in AI-Native College Students' Academic Self-Assessment](https://edtechdev.github.io/aied/articles/absent-cognitive-baseline-2026/) > **Synthesis:** This paper proposes the Absent Cognitive Baseline (ACB) as a conceptual framework describing how pervasive generative AI use during secondary schooling may reduce the independent cognitive encounters on which academic self-assessment depends. Drawing on metacognitive theory, self-regulated learning, and epistemic development scholarship, the ACB framework describes a structural gap across three dimensions: inability to locate one's own cognitive boundary, miscalibrated sense of understanding from AI-generated fluency, and erosion of cognitive struggle as a recognizable part of learning. A conceptual model identifies three groups of moderating variables shaping ACB emergence. ## Key Findings This paper proposes the Absent Cognitive Baseline (ACB) as a conceptual framework describing how pervasive generative AI use during secondary schooling may reduce the independent cognitive encounters on which academic self-assessment depends. Drawing on metacognitive theory, self-regulated learning, and epistemic development scholarship, the ACB framework describes a structural gap across three dimensions: inability to locate one's own cognitive boundary, miscalibrated sense of understanding from AI-generated fluency, and erosion of cognitive struggle as a recognizable part of learning. A conceptual model identifies three groups of moderating variables shaping ACB emergence. The work contributes to understanding of [[metacognition]] in educational contexts, with implications for [[cognitive-offloading]], [[self-regulated-learning]]. ## Connected Concepts - [[metacognition]] - [[cognitive-offloading]] - [[self-regulated-learning]] - [[student-experience]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[cognitive-offloading-speedup-illusion]] - [[digital-literacy-illusion]] - [[ai-making-us-stupid]] ## Citation Jia, Y., & Xu, J. (2026). [*The Absent Cognitive Baseline: Theorizing a Structural Gap in AI-Native College Students' Academic Self-Assessment*](https://doi.org/10.35542/osf.io/4cr8j_v5). EdArXiv Preprint. --- ## [AcademiClaw: When Students Set Challenges for AI Agents](https://edtechdev.github.io/aied/articles/academiclaw-student-agent-benchmark/) > **Yu, Lu, Si et al. (77 authors, 2026)** — Shanghai Jiao Tong University, SII, GAIR. Open-source benchmark. # AcademiClaw: Student-Sourced AI Agent Benchmark **Yu, Lu, Si et al. (77 authors, 2026)** — Shanghai Jiao Tong University, SII, GAIR. Open-source benchmark. ## Core Contribution **AcademiClaw** is a bilingual benchmark of **80 complex, long-horizon tasks** sourced from university students' real academic workflows — homework, research projects, competitions, and personal projects — that they found current AI agents unable to solve. It extends the OpenClaw ecosystem beyond assistant-level tasks into academic-level agent evaluation. ## Benchmark Design ### Task Sourcing & Curation - **230 student-submitted candidates** → rigorous expert review → 80 final tasks - Tasks are *authentic*: real problems students couldn't solve with current AI agents - **25+ professional domains**: olympiad-level math, linguistics, GPU-intensive RL, full-stack system debugging - **16 tasks require CUDA GPU execution** — testing hardware-accelerated AI capabilities ### Execution & Scoring - Each task runs in an **isolated Docker sandbox** - Scored by **multi-dimensional rubrics** using 6 complementary techniques - **Independent 5-category safety audit** provides behavioral analysis beyond task completion - Bilingual (Chinese/English) ### Key Results - **6 frontier models** tested - **Best pass rate: 55%** — no model exceeds this - Sharp capability boundaries across task domains - Divergent behavioral strategies between models - **Disconnect between token consumption and output quality** — more tokens ≠ better results ## Why This Matters for AI in Education AcademiClaw flips the evaluation paradigm: instead of researchers designing artificial benchmarks, **students define what AI agents should be able to do**. This aligns evaluation with real educational needs: 1. **Authentic task validity**: Tasks reflect genuine academic workflows, not synthetic proxies 2. **Capability gap diagnosis**: The 55% ceiling reveals where AI agents still fail students 3. **Token-output disconnect**: Challenges the assumption that more compute solves academic problems — relevant to AI tutoring cost/benefit analysis 4. **Safety in academic contexts**: The 5-category safety audit surfaces risks specific to educational AI deployment ## Limitations - Contributor pool concentrated at a single institution (SJTU) — limits cultural and disciplinary diversity - GPU-intensive tasks (16/80) require specialized hardware, limiting reproducibility - Bilingual but primarily Chinese university context - Safety audit releases aggregate statistics only, not full violation traces ## Open Questions - How would the 55% pass rate change with iterative refinement or multi-agent collaboration? - Would results differ at non-Chinese universities with different academic workflows? - Can the benchmark be adapted for K-12 or professional training contexts? - What does the token-output disconnect imply for AI tutoring systems that bill by token usage? ## Connected Concepts - [[automated-question-generation]] - [[pedagogical-llm-training]] - [[socratic-method]] - [[math-education]] - [[prompt-engineering]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[automated-essay-scoring]] ## Connected Articles - [[codify-socratic-tutoring-programming]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[ai-tutor-behavioral-evaluation]] — The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness - [[lets-chat-chatbot-outreach-2026]] — Let''s Chat: Leveraging Chatbot Outreach for Improved Course Performance - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[scheu-mobile-chatbot-journaling-motivation-2026]] — Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement ## Citation Yu, J., Lu, P., Si, W., Lu, H., Wu, J., Tao, K., et al. (2026). [AcademiClaw: When Students Set Challenges for AI Agents](https://arxiv.org/abs/2605.02661). arXiv:2605.02661. --- ## [Acceptance of AI-Assisted English Language Learning Tools in Higher Education: Psychological Correlates Across Disciplinary and Proficiency Groups](https://edtechdev.github.io/aied/articles/acceptance-ai-english-tools-2026/) > **Synthesis:** Wu et al. (2026) examined how learning motivation, self-efficacy, anxiety, and risk perception relate to acceptance of AI-assisted English language learning in a Chinese higher-education context, building on the Technology Acceptance Model (TAM). Drawing on survey data from 210 undergraduates (STEM = 91, Humanities = 119; English proficiency Low = 77, Intermediate = 103, High = 30), they found that learning motivation and self-efficacy were consistently and positively associated with acceptance-related indicators (perceived usefulness, perceived ease of use, behavioral intention, and satisfaction), while anxiety and risk perception showed more nuanced, generally weaker patterns that must be interpreted cautiously. The study provides a learner-centered account emphasizing psychological correlates and descriptive group-level differences rather than causal claims. ## Key Findings - Learning motivation and self-efficacy were consistently and positively associated with acceptance-related indicators across the sample. - Anxiety did not show a uniformly negative pattern — its positive associations with acceptance outcomes were exploratory concurrent patterns and should not be read as evidence that anxiety is beneficial. - Risk perception was positively but generally more weakly associated with acceptance outcomes, and the authors caution against over-interpreting this association. - Mean-level comparisons showed descriptive differences in several acceptance outcomes across academic discipline and English proficiency groups, with exploratory subgroup-specific regressions describing within-group patterns (not formal between-group tests). - Analyses used SPSS 27.0 and Stata 18.0 with robust-standard-error adjusted linear regressions, mean-level comparisons, post hoc tests, and exploratory subgroup regressions; findings are associative, not causal, given the cross-sectional design. ## Study Design & Method The study surveyed 210 undergraduates from a Chinese university, with balanced disciplinary (STEM vs. Humanities) and proficiency (Low/Intermediate/High English) groups. Instruments measured AI-assisted English learning acceptance outcomes (perceived usefulness, perceived ease of use, behavioral intention, satisfaction) alongside learning motivation, self-efficacy, anxiety, and risk perception. Analyses included descriptive statistics, reliability and convergent validity, full-sample adjusted linear regressions with robust standard errors, mean-level group comparisons, post hoc comparisons, and exploratory subgroup-specific regressions. The authors were explicit that subgroup regressions describe within-group patterns rather than testing between-group coefficient differences, and that the cross-sectional design supports associative rather than causal interpretation. ## Implications for AI in Education The findings argue that acceptance of AI-assisted [[language-learning]] tools is not purely a matter of perceived usefulness or ease of use — it is shaped by learners' psychological resources ([[motivation]], [[self-regulated-learning|self-efficacy]]), affective states (anxiety), and evaluative judgments (risk perception), which vary descriptively across disciplines and proficiency levels. For practitioners, this supports designing AI-assisted English learning that builds learner confidence and motivation rather than merely optimizing usability, and tailoring support to disciplinary norms and proficiency groups. It also cautions against simplistic assumptions that anxiety always blocks or risk perception always deters AI adoption. The study grounds a learner-centered extension of TAM for [[generative-ai]] English learning, complementing system-centered technology acceptance accounts. ## Limitations The cross-sectional design precludes causal inference, and all constructs rely on self-report. The sample (N = 210) is a single-institution Chinese higher-education sample, bounding generalizability to other contexts and populations. Subgroup analyses were exploratory and descriptive, not formal tests of between-group differences. The counterintuitive associations of anxiety and risk perception with acceptance underscore the need for replication and for cautious interpretation of the descriptive patterns observed. ## Connected Concepts - [[language-learning]] - [[generative-ai]] - [[motivation]] - [[self-regulated-learning]] - [[higher-ed]] - [[personalized-learning]] - [[teacher-role]] ## Connected Articles - [[genai-pronunciation-feedback-wtc-2026]] — Associations Between Generative AI-Based Pronunciation Feedback and Willingness to Communicate - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition - [[genai-feedback-design-multisite-experiment]] — Human-Centered GenAI Feedback Design in Higher Education - [[ai-tools-arab-english-classrooms]] — AI Tools in Arab University English Classrooms - [[will-skill-not-tool-chinese-university-students-acceptance-of-generative-ai-for-]] — Will, Skill, Not Tool: Chinese University Students' Acceptance of Generative AI ## Citation Wu, J., Wang, Y., He, Y., Yin, X., Chen, F., & Wan, B. (2026). [*Acceptance of AI-assisted English language learning tools in higher education: Psychological correlates across disciplinary and proficiency groups*](https://doi.org/10.3389/fpsyg.2026.1806457). --- ## [Access is Not Enough: Human Support Improves Engagement with AI Tutoring](https://edtechdev.github.io/aied/articles/access-not-enough-ai-tutoring-2026/) > Robinson, Gormley, Ribeiro & Loeb (2026) ran two RCTs showing that AI tutoring's binding constraint is **take-up, not capability**: despite dedicated session time, nearly half of students never used the platform and users averaged only 2–5 minutes per week. An in-person engagement tutor (not direct instruction) raised usage by 1–4 minutes/week and engagement by 71–80% — but dosage stayed far below the level needed for reading gains, and achievement did not improve. ## The two RCTs - Districts A and B, elementary students, AI literacy (reading) platform; treatment = in-person tutor focused on *engagement* (10-min check-in + 15 min platform + 2–5 min reflection), control = platform alone with the full ~30 min - Pre-registered outcomes: minutes/week (usage) and stories read/week (engagement); spring ELA as achievement - Interventions lasted 14–31 weeks ## Take-up is the story - Only **60.7% / 53.3% of control students ever used the platform** - Average weekly usage: **2.18 / 5.23 minutes**; even users-only averages were 13.2 / 25.8 minutes - Students used the platform in only **4–5 of 14–31 weeks** - Platform users skewed higher-achieving and less likely to receive special-education services — an equity flag: the students who need adaptive support most are least likely to engage ## What human support did - Usage: **+1 min/week (A), +4.4 min/week (B)** — significant, but a drop against the ~30 min/week the provider recommends for measurable reading gains - Engagement: **+0.20 stories/week (A, +71%)** and **+0.92 stories/week (B, +80%)** - Total added dosage: ~22 minutes (A) and ~98 minutes (B) across the *entire* intervention - **No reading achievement gains** in either district (negative, non-significant) — consistent with the achieved dosage - Strong site-level variation: some sites saw little effect, others meaningful gains — local implementation conditions matter ## Connected Concepts - [[ai-literacy]] - [[k-12]] - [[student-experience]] - [[rag]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries ## Citation Robinson, C. D., Gormley, D., Trindade Ribeiro, A., & Loeb, S. (2026). [Access is Not Enough: Human Support Improves Engagement with AI Tutoring. EdWorkingPaper No. 26-1451, Annenberg Institute at Brown University. DOI](https://doi.org/10.26300/pz7p-p388) --- ## [AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction](https://edtechdev.github.io/aied/articles/adapt-adaptive-lesson-plan-transformer/) AdaPT uses transformers to adapt lesson plans across regional and differentiated instruction contexts; improves teacher efficiency while maintaining pedagogical alignment with local curricula. ## Synthesis AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction investigates adapt uses transformers to adapt lesson plans across regional and differentiated instruction contexts; improves teacher efficiency while maintaining pedagogical alignment with local curricula. This work connects to existing research on adaptive-learning-systems by demonstrating that Abstract:Due to educational inequality, high-quality lesson plans often mismatch the needs of disparate educational contexts. Teachers typically modify existing lesson plans to fit new contexts, but current tools instead focus on generating content from scratch, creating additional workload. Moreove.... ## Connected Concepts - [[affective-tutoring]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[affective-computing]] - [[lifelong-learning]] - [[computational-thinking]] - [[k-12-ai-education]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[civic-education-ai-lesson-plans]] — AI-Generated Lesson Plans in Civic Education ## Citation Yanjie Zhang, Jiajun Zhu, Minyu Wu, Huamin Qu, Sicheng Song (2026). [AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction](https://arxiv.org/abs/2606.17633). arXiv:2606.17633. --- ## [Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study](https://edtechdev.github.io/aied/articles/adaptive-pretesting-retention/) Akgun and Toker (2026) examine whether learning gains from GenAI-enabled adaptive pretesting persist over a seven-week retention period. Undergraduate participants completed adaptive AI-assisted pretesting, received instruction, took a baseline assessment, and were randomly assigned to three follow-up conditions: adaptive spaced retrieval practice, fixed spaced retrieval practice, or learner-directed AI-supported study. Multivariate analyses showed significant effects of condition on posttest performance and practice effort, with both retrieval-based conditions significantly outperforming learner-directed study. The findings indicate that while AI-adaptive pretesting can elevate initial understanding — especially for higher-order reasoning — sustained learning critically depends on how subsequent AI-supported practice is structured. This [[rct]] contributes to the [[adaptive-learning]] literature by showing that initial gains are fragile without structured retrieval, and has practical implications for designing [[personalized-learning]] sequences that maximize learning-gains and promote [[transfer-of-learning]]. ## Connected Concepts - [[rct]] - [[adaptive-learning]] - [[personalized-learning]] ## Connected Articles - [[transfer-of-learning]] ## Citation Mahir Akgun, Sacip Toker (2026). [Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study](https://arxiv.org/abs/2606.22328). arXiv:2606.22328. 27th International Conference on AI in Education --- ## [The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training](https://edtechdev.github.io/aied/articles/adaptive-virtual-patient-psychotherapy-training/) **Angela Chen, Siwei Jin, Catherine Bao, Canwen Wang, Robert E. Kraut, Tongshuang Wu, Haiyi Zhu** — cs.CY, cs.HC The Adaptive Virtual Patient (AVP) is an LLM-driven simulated patient for psychotherapy training that adapts its disclosure behavior — from guarded through moderate openness to full disclosure — in response to trainee skill. Grounded in a structural equation model fit to nearly 2,000 hours of real-world psychotherapy transcripts, the AVP quantifies how therapist empathy and exploration shift patient openness. An LLM generates utterances conditioned on a disclosure level updated each turn by the dynamics module. In evaluation with 20 clinicians and trainees over 80 sessions (1,033 turns), the AVP's disclosure rises in response to therapist empathy and exploration, while a prompt-only baseline stays flat; ablations confirm that the empirically motivated parameterization outperforms alternatives. This represents a significant advance in [[intelligent-tutoring]] for professional training domains where authentic practice opportunities are scarce and where existing role-play systems lack grounded, adaptive behavior. ## Connected Concepts - [[intelligent-tutoring]] ## Connected Articles - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy ## Citation Chen, A., Jin, S., Bao, C., Wang, C., Kraut, R. E., Wu, T., & Zhu, H. (2026). [*The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training: Disclosure That Responds to Therapist Micro-Skills*](https://arxiv.org/abs/2606.10051). arXiv:2606.10051. --- ## [Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education](https://edtechdev.github.io/aied/articles/adhd-video-segmentation-computing-education/) Pimenova, Begel and colleagues evaluate a post-hoc video processing intervention that segments instructional videos into single-instruction chunks with fixed pauses, reducing extraneous cognitive load for learners with ADHD. In a within-participants study (17 ADHD, 10 non-ADHD), the intervention improved everyone but had an equalizing effect: ADHD participants' errors and hesitations fell to parity with non-ADHD peers. This is strong evidence for Universal Design for Learning via automated content transformation — a task well-suited to AI pipelines. It extends [[neurodivergent-computing-students]] and [[accessible-learning]], complements simulation work like [[llm-educational-simulation-adhd]], informs video-based learning design in [[engagement-assessment-video]] and [[instructional-design]], within [[cs-education]]. ## Connected Concepts - [[accessible-learning]] - [[cs-education]] - [[instructional-design]] - [[neurodiversity]] - [[llm]] ## Connected Articles - [[neurodivergent-computing-students]] - [[llm-educational-simulation-adhd]] - [[engagement-assessment-video]] ## Citation Veronica Pimenova, Chris Lee, Baramee Bhakdibhumi, Simon Chu, Andrew Begel (2026). [Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education](https://arxiv.org/abs/2607.24612). arXiv:2607.24612. --- ## [Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation](https://edtechdev.github.io/aied/articles/adversarial-stress-testing-role-playing-agents/) > **Synthesis:** This paper presents a modular multi-agent platform for adversarially stress-testing [[agentic-ai|role-playing language agents]] through structured multi-turn dialogue. With three coordinated agents — Interrogator (applying six progressive adversarial strategies), Target, and Judge — the system reveals failure modes invisible to single-strategy testing, reducing robustness scores by 0.17-0.20 points. The framework is directly relevant to [[ai-education|educational AI agents]], where persona consistency and ethical constraints are critical for [[pedagogical-safety|safe deployment]] with learners. ## Platform Architecture The evaluation platform coordinates three specialized agents: **1. Interrogator Agent:** - Applies six progressive adversarial strategies including Authority Challenge and Emotional Manipulation - Escalates pressure across multi-turn interactions to test cumulative robustness **2. Target Agent:** - The role-playing language agent (RPLA) under evaluation - Evaluated across diverse personas including educational tutoring roles **3. Judging Agent:** - Automated scoring across four dimensions: role fidelity, behavioral drift, ethical deviation, and consistency - Achieves strong human alignment (r = 0.82, Fleiss' κ = 0.71) ## Key Findings | Finding | Result | |---------|--------| | Multi-strategy vs. single-strategy | 0.17-0.20 point robustness reduction | | Most effective attack | Authority Challenge + Emotional Manipulation | | Cross-model consistency | Consistent degradation across Llama-3.3-70B, GPT-4o-mini, Claude-3.5-Haiku | | Automated judging quality | r = 0.82 correlation with human judges | - **Failure mode discovery:** Multi-strategy adversarial testing reveals behavioral failures invisible to standard single-turn benchmarks - **Strategy effectiveness:** Authority Challenge and Emotional Manipulation emerge as the most effective attack vectors - **Cross-model validation:** Degradation patterns are consistent across three major LLM families, suggesting fundamental vulnerabilities rather than model-specific weaknesses ## Relevance to Educational AI The framework's relevance to education is twofold: - **[[ai-tutoring|AI tutors]] and [[pedagogical-agent|pedagogical agents]]** are role-playing agents that must maintain consistent instructional personas, making them candidates for this evaluation methodology - **Student interaction patterns** can be adversarial (testing boundaries, emotional appeals, authority challenges), and educational agents must be robust to these behaviors - The open-source release provides infrastructure for the AIED community to evaluate safety and robustness of educational language agents ## Connected Concepts - [[agentic-ai]] - [[ai-education]] - [[pedagogical-safety]] - [[ai-tutoring]] - [[pedagogical-agent]] ## Connected Articles - [[detecting-llm-generated-text-latent-prompt]] — Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[jeon-isd-agent-bench-2026]] — ISD Agent Benchmark - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[agentic-workflows-education]] — Agentic Workflows in Education ## Citation Shouqi, S., Nazly, A., Wanniarachchi, J., & De Alwis, R. (2026). [*Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation*](https://arxiv.org/abs/2608.03166v1). arXiv:2608.03166v1. --- ## [A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring](https://edtechdev.github.io/aied/articles/affective-text-wearable-student-health/) # Brief Affective Text and Wearable Sensing for Student Health Monitoring ## Key Findings In a year-long study of 458 university students (3,610 person-waves) using Oura rings for passive physiological sensing, researchers examined whether **ultra-brief affective text prompts** (median 3-word responses to "what concerns you most?") could enrich the interpretation of wearable data. Using NLP methods spanning dictionary-based (LIWC), general pretrained embeddings, and domain-adapted models: - **Academic concern framing** was associated with lower physical activity - **Emotional exhaustion language** was associated with poorer sleep quality and lower heart rate variability (HRV) - **General pretrained embeddings outperformed** domain-adapted models for most health outcomes - **Domain adaptation** showed relative advantage only for autonomic nervous system measures - **Affective dimensions** (emotional register) were consistently associated with outcomes across all NLP methods — *how* students express concerns matters more than *what* they are concerned about ## Methodological Significance The finding that emotional register rather than topical content carries predictive signal has implications for [[engagement-assessment-video]], [[genai-tutor-engagement-patterns]], and other work that analyzes student language for learning signals. It suggests that **simple affective prompts at minimal burden** may be more scalable than complex topic classification for educational well-being systems. ## Connection to AI Campus Well-Being This study provides empirical grounding for the kind of affective monitoring infrastructure imagined in [[ai-campus-wellbeing-tools]]. While Tang's framework proposes integrated AI tools (TigerGPT, AURA, PsychoGPT) for campus well-being, Harry et al. demonstrate that even ultra-brief, low-burden text prompts — analyzed with standard NLP — can surface meaningful psychological signals tied to physiological outcomes. ## Implications for Learning Analytics The dissociation between topic and affect aligns with [[multimodal-ai-feedback-learning]] research showing that *how* students interact with AI systems often matters more than *what* they produce. For [[learning-analytics]] dashboards and early-warning systems, this suggests tracking emotional tone in student communications may be more predictive than categorizing concern topics. ## Connected Concepts - [[learning-analytics]] ## Connected Articles - [[engagement-assessment-video]] - [[genai-tutor-engagement-patterns]] - [[ai-campus-wellbeing-tools]] - [[multimodal-ai-feedback-learning]] ## Citation Harry, T., Hidalgo, J., Price, M., Feng, Y., Stanton, K., Tompkins, C., Dodds, P. S., Fudolig, M. I., Bloomfield, L., & Danforth, C. (2026). [A formative study of brief affective text as a complement to wearable sensing for longitudinal student health monitoring](https://arxiv.org/abs/2605.14360). arXiv:2605.14360 [cs.HC]. Submitted to ACM IMWUT. --- ## [The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning](https://edtechdev.github.io/aied/articles/agency-gap-ai-writing/) ## Summary A randomized experiment (n = 79 medical/nursing students) examining how the **initiative design** of an AI writing agent shapes reasoning, agency, and immediate independent performance. Students completed two multimodal analytical writing tasks (interpreting healthcare-simulation data visualisations: bar chart, network diagram, ward heatmap) with either a **reactive agent** (responds only when prompted, n = 39) or a **proactive agent** (initiates sequenced questions and feedback, n = 40). GenAI literacy was measured with the validated 20-item **GLAT**. The study introduces the **agency gap**: a relational mismatch between the initiative an AI agent demands and the learner's capacity to initiate, monitor, evaluate, and internalise AI-supported reasoning — neither an individual deficit nor a fixed property of the system. ## Key findings ### RQ1 — Epistemic network structure differs strongly by design - ENA (explaining 38.3%/22.7% and 27.5%/26.4% of variance) separated conditions with **large effects** (Cliff's δ = −0.56, −0.78; both p < .001). - **Proactive** dialogues: stronger links between conceptual reasoning, adequate reasoning, and constructive engagement (EP-CS–EP-CP-Adeq, EP-PS–EP-CP-Adeq, I-CON–EP-CP-Adeq) — more integrated epistemic elaboration. - **Reactive** dialogues: more factual/procedural/off-task pairings (EP-PS–EP-OFF, EP-OFF–I-ACT) — the learner's own regulation is more visible but discourse stays descriptive. - The difference is in *how ideas are connected*, not how often categories appear and not the final score. ### RQ2 — GenAI literacy predicts immediate independent performance - After AI support was removed, GLAT predicted **Visual Data Integration** (OR 1.14, p = .039), **Critical Thinking** (OR 1.15, p = .029), and the **Composite score** (OR 1.11, p = .032) — modest, higher-order effects; not significant for insightfulness, organisation, or linguistic quality. - AI-supported performance strongly predicted AI-removal performance on all dimensions (all p ≤ .001) — continuity, but cannot distinguish learning from stable competence. - No significant condition effect and **no significant literacy-by-design interaction**. ### RQ3 — Mediation patterns are suggestive, not confirmatory - Reactive condition: significant total literacy→performance association (β = 0.172, p = .020) with direct path remaining (β = 0.140); indirect effect non-significant (95% CI [−0.022, 0.108]). - Proactive condition: total and direct coefficients near zero; indirect non-significant. - Pattern is *consistent with* smaller literacy-related performance differences under proactive scaffolding, but does **not establish** compensation, mediation, or moderation — hypothesis-generating for future adequately powered tests. ### RQ4 — Three design heuristics from learner reflections 1. **Sustain autonomy** through contextual and confirmatory feedback (reactive strength: confirms interpretations, lowers barrier, but redundant for proficient learners). 2. **Promote integrative reasoning and immediate independent application** through dialogic scaffolding (proactive strength: connects evidence across visuals, prompts self-correction; risks over-scaffolding easy tasks). 3. **Ensure equity through adaptive alignment** of initiative with learner expertise and task complexity — a uniform interaction style may under-support some learners while over-directing others. ## Interpretation - **Process ≠ outcome**: agent design produced large differences in the relational organisation of dialogue but no significant direct effect on immediate writing scores — the mechanism is how epistemic work is distributed, not output quality. - The agency gap frames the failure modes: **under-support** (low literacy × strongly reactive design) and **over-direction** (high capability × rigidly proactive design), echoing [[scaffolding]]'s expertise-reversal effect and adaptive-scaffolding accounts. - Practice: make initiative **visible and adjustable** (request/skip/pause prompting), structure proactive prompts to orient–interpret–connect–synthesise rather than supply answers, and fade prompts as learners demonstrate independence; teach GenAI literacy as part of academic writing ([[ai-literacy]], [[agentic-ai]]). - Limitations: n = 79 underpowered for mediation; medical/nursing sample; immediate AI-removal task measures near transfer, not durable learning; agency gap theorised, not directly measured; no manipulation-check coding of agent turns. ## Connected Concepts - [[agentic-ai]] - [[ai-literacy]] - [[higher-ed]] - [[scaffolding]] - [[student-experience]] - [[writing-education]] - [[generative-ai]] - [[rag]] - [[regulation]] ## Connected Articles - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative a... - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers' practices and perceptions - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... ## Citation Jin, Y., Yang, K., Martinez-Maldonado, R., Gašević, D., & Yan, L. (2026). The agency gap in AI-supported writing: How reactive and proactive agent designs shape multimodal reasoning. [*Computers and Education: Artificial Intelligence*](https://doi.org/10.1016/j.caeai.2026.100655). Advance online publication --- ## [Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning](https://edtechdev.github.io/aied/articles/agent-voice-accents-k12-group-learning/) **Ravi, Stevens, Hurt, Hanks, Lin & Anderson (2026)**. Ravi et al. investigate how the voice accent of a [[generative-ai]] conversational peer agent shapes learners' perceptions, trust, and interactional dynamics in [[k-12]] group learning. While prior work examined agent accent effects in one-to-one settings, little is known about how these effects manifest in multi-party group contexts — a critical gap as [[intelligent-tutoring|AI peer agents]] enter collaborative classrooms. The between-subjects mixed-methods study involved 33 teachers interacting with a GenAI voice agent in three accent conditions — British, Indian, and African American. Key findings: 1. **The British-accented agent** was largely treated as a **tool** and engaged with in detached, utility-based ways — less anthropomorphized, more like an external resource. 2. **Indian- and African American-accented agents** were more readily **anthropomorphized** and **integrated as peers** — building stronger trust, engagement, and reliance over time. 3. These role expectations influenced collaboration dynamics: turn-taking, questioning patterns, and perceived social presence all shifted based on accent condition. The findings advance understanding of how GenAI's sociolinguistic design features shape group dynamics in CSCL (computer-supported collaborative learning), with implications for designing [[equity|culturally inclusive AI partners]]. The work connects to [[human-ai-collaboration]] debates about [[teacher-role|teacher and AI roles]] in collaborative classrooms, and raises important questions about [[ai-literacy|how accent may inadvertently reinforce or disrupt power dynamics]] in educational AI. ## Connected Concepts - [[generative-ai]] - [[k-12]] - [[intelligent-tutoring]] - [[equity]] - [[human-ai-collaboration]] - [[teacher-role]] - [[ai-literacy]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[civic-education-ai-lesson-plans]] — AI-Generated Lesson Plans in Civic Education - [[gaide-vibe-coding-k12-teachers]] — A Guiding Framework for K-12 Teachers in Creating AI-powered Learning Technologies through Vibe Coding ## Citation Ravi, P., Stevens, C., Hurt, B., Hanks, B., Lin, G., & Anderson, E. (2026). [Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning](https://arxiv.org/abs/2606.12805). arXiv:2606.12805. --- ## [Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm](https://edtechdev.github.io/aied/articles/agentic-ai-education-scoping-review/) # Agentic AI in Education: Scoping Review (474 Studies, 2020–2026) Published in *Computers and Education: Artificial Intelligence*, accepted 27 July 2026. 📄 doi:10.1016/j.caeai.2026.100653 ## Summary This scoping review systematically maps **474 studies** (January 2020 – May 2026) on generative AI-powered agentic AI in education, providing the most comprehensive synthesis of the field to date. The authors analyze publication characteristics, study designs, agent roles, AI models/architectures, six dimensions of agentic capability, and the extent of educational theory integration. ## Key Findings ### 1. Rapid Expansion Since 2025 The field has grown explosively, but the literature is dominated by **conference papers** concentrated in **higher education**, **STEM disciplines**, and **text-based tutoring** scenarios. This mirrors the general trajectory of [[ai-education]] research, but with a specific agentic inflection point in 2025. ### 2. Technology Stack: GPT + LangChain Dominate **GPT-series models** and **LangChain** are the most widely adopted technologies. Notably, **OpenClaw** and other frontier agent paradigms (governed tool orchestration, persistent memory, long-horizon planning, multi-agent coordination) remain **largely absent** from educational research — revealing a significant technology–application gap. This stands in contrast to the vision articulated in [[agentic-ai]]. ### 3. Agentic Capabilities Remain Modest Across the six capability dimensions analyzed: - **Single-task autonomy** — commonly demonstrated ✓ - **Sequential planning** — increasingly present ✓ - **Multi-agent collaboration** — growing ✓ - **Strong tool orchestration** — rarely exhibited ✗ - **Robust embedded governance** — rarely exhibited ✗ - **Persistent memory / long-horizon planning** — largely absent ✗ This maps closely to the four-paradigm framework in [[agentic-workflows-education]] (reflection, planning, tool use, multi-agent collaboration), where the reviewed systems tend to cluster in the first two paradigms while falling short on the more advanced ones. ### 4. Theoretical Grounding is Limited Only **138 of 474 studies** (29%) explicitly drew on educational theory, revealing a **clear disciplinary divide** between technically oriented research (CS/engineering) and pedagogically oriented work (education/learning sciences). This echoes broader concerns in [[principled-ai-education]] about the gap between technological capability and pedagogical intentionality. ### 5. Methodological Limitations Most studies rely on **small-scale, short-term designs**. Longitudinal and real-world validation studies are rare, limiting the evidence base for claims about effectiveness. The review calls for more rigorous efficacy-study designs and attention to [[student-experience]] beyond immediate performance metrics. ### 6. Six Dimensions of Agentic Capability (the Review's Analytical Framework) | Dimension | Description | Status in Reviewed Studies | |---|---|---| | Task Autonomy | Independent task initiation, planning, completion | Common | | Goal-Directed Reasoning | Strategy selection and adaptation to context | Emerging | | Memory & Context Awareness | Using interaction history and learner profiles | Limited | | Planning & Sequencing | Multi-step plan formulation and execution | Growing | | Tool Orchestration | Invoking and coordinating external tools/resources | Rare | | Governance & Oversight | Auditable action, safety constraints, human-in-the-loop | Rare | The governance gap is particularly concerning given frameworks like [[human-in-the-loop-ai]], which emphasize that educational AI systems require robust oversight mechanisms — not just technical capability. ## Priority Research Directions The review identifies several converging priorities: 1. **Longitudinal and real-world validation** — moving beyond short-term lab studies 2. **Stronger pedagogical grounding** — bridging the CS/education disciplinary divide 3. **Governed adoption of emerging agent infrastructures** — particularly tool orchestration and multi-agent coordination 4. **Systematic integration of ethics and human oversight** — connecting to [[equity]] and [[academic-integrity]] concerns 5. **Expanding beyond STEM and higher education** — into K-12, language learning, special education, and professional training contexts ## OpenClaw as an Analytical Lens The review uses **OpenClaw** (Steinberger, 2026) — the fastest-growing [[open-source]] AI project in early 2026 — as an illustrative reference point for the "frontier agent paradigm": systems that feature governed tool orchestration via MCP, persistent memory, long-horizon planning, multi-agent coordination, and auditable action. The finding that these capabilities are largely absent from educational agentic systems is the review's most striking technology–application gap. While the authors are careful not to position OpenClaw as a normative target, its feature set serves as a useful benchmark for assessing how far educational systems lag behind general-purpose agentic infrastructure. ## Connected Concepts - [[adaptive-learning]] - [[agentic-ai]] - [[human-in-the-loop-ai]] - [[ai-education]] - [[open-source]] - [[student-experience]] - [[agentic-ai]] - [[ai-literacy]] - [[generative-ai]] - [[higher-ed]] - [[llm]] - [[scaffolding]] ## Connected Articles - [[agentic-workflows-education]] — Agentic Workflows in Education - [[principled-ai-education]] — Principled AI in Education - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis ## Citation Wang, N., Zou, D., Xie, H., & Qin, S. J. (2026). [*A scoping review of generative AI-powered agentic AI in education: Research landscape, agentic capabilities, and insights from the frontier agent paradigm, exemplified by OpenClaw*](https://doi.org/10.1016/j.caeai.2026.100653). --- ## [Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning](https://edtechdev.github.io/aied/articles/agentic-ai-pedagogical-best-practice-2026/) > Education AI is shifting from passive chatbots to **proactive agents** that initiate and pursue goals. This offers personalisation but risks undermining **learner agency and cognitive effort**. The paper walks each of six pedagogical principles through what agentic initiative does to it: > Woollaston, Flanagan, Wijerathne & Ogata (2026, AIED HAI-Agency Workshop) review six established pedagogical principles through the lens of **proactive agentic AI** and articulate the central tension: the more an agent automates, the less cognitive work the learner does. Their design response — **intentional friction, dynamic scaffolding, human-in-the-loop oversight, and considered AI utilisation** — is a principled guardrail for the wiki's agentic-education literature. ## The tension: automation vs. learning Education AI is shifting from passive chatbots to **proactive agents** that initiate and pursue goals. This offers personalisation but risks undermining **learner agency and cognitive effort**. The paper walks each of six pedagogical principles through what agentic initiative does to it: | Principle | Agentic-AI risk | |---|---| | Prior knowledge activation | Agents pre-fetching content bypass the retrieval practice that activates prior knowledge | | Collaborative learning | Agent initiative crowds out peer negotiation and role-taking | | Problem-based learning | Goal-directed agents resolve problems before learners grapple with them | | Formative assessment | Agent-generated feedback pre-empts learners' own self-assessment loops | | Scaffolding | Automated scaffolds stay static instead of being dynamically withdrawn | | Metacognition | Agent initiation displaces the learner's own planning, monitoring, evaluation | ## Design recommendations 1. **Intentional friction** — deliberately preserve productive struggle instead of maximising smoothness (cf. [[desirable-difficulties]]: difficulty that supports learning) 2. **Dynamic scaffolding** — scaffolds that adapt and fade as competence grows 3. **Human-in-the-loop oversight** — learners and educators retain control over agent initiation 4. **Considered AI utilisation** — purposeful, pedagogically justified agent use rather than maximal automation ## Connected Concepts - [[agentic-ai]] - [[agentic-ai]] - [[desirable-difficulties]] - [[formative-assessment]] - [[metacognition]] - [[scaffolding]] - [[zone-of-proximal-development]] - [[llm]] ## Connected Articles - [[agentic-workflows-education]] — Agentic Workflows in Education - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation Woollaston, S., Flanagan, B., Wijerathne, I., & Ogata, H. (2026). [Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning](https://arxiv.org/abs/2606.04543). arXiv:2606.04543 --- ## [Agentic Education with AI Coding Assistants](https://edtechdev.github.io/aied/articles/agentic-education-coding/) > AI coding assistants proliferate rapidly, but pedagogical frameworks for learning them remain scarce — a paradox at the heart of agentic coding education. > Using agentic AI workflows (Claude Code) to teach AI coding tools — structured pedagogy for tool mastery. ## The Gap in AI Coding Education **Naboulsi (2026)** identifies a paradox: **AI coding assistants proliferate rapidly, but pedagogical frameworks for learning them remain scarce**. ### Current State Learners face: - **Fragmented resources:** Blog posts, video tutorials, trial-and-error - **Documentation-to-practice gap:** Tool docs ≠ pedagogical structure - **No progression scaffold:** Jump from "hello world" to "production use" without intermediate structure ## Agentic Workflow Pedagogy The paper presents a framework for teaching **Claude Code** (agentic coding assistant) using **Claude Code itself** — recursive/agentic education. ### Pedagogical Structure | Phase | Focus | Agentic Capability | |-------|-------|-------------------| | **Tool familiarization** | Basic commands, setup | Single-turn interactions | | **Task decomposition** | Breaking problems into agentic steps | Multi-step planning | | **Autonomous execution** | Letting agent run, monitoring | Self-correction, iteration | | **Meta-cognition** | When to use (and not use) agentic workflows | Strategic tool selection | ## Connection to Agentic Workflows in Education This operationalizes [[agentic-workflows-education]] for a specific tool: - **Reflection:** Agent reflects on its own code/approach - **Planning:** Decomposes tasks before execution - **Tool use:** Uses file I/O, shell, other tools - **Multi-agent:** Could extend to peer-agent collaboration (cf. [[collaborative-ai-tutoring]]) ## Implications ### For CS Education - **Agentic literacy:** Learning *how to direct* AI agents is a new learning objective - **Scaffolded autonomy:** Progressive release of agentic control (teacher-guided → student-directed) - **Meta-cognitive skill:** Knowing when agentic approaches help vs. hinder learning ### For ITS Design - **Teach tool + teach with tool:** Agentic systems can be *both* tutor and tutee - **Recursive pedagogy:** Advanced agents can teach novice users (and novice agents) - **Transfer concerns:** Does learning with Claude Code transfer to other agentic tools? ## Connected Concepts - [[agentic-ai]] - [[math-education]] - [[curriculum-design]] - [[administrator]] - [[socratic-ai-dialogue]] - [[physics-education]] - [[help-seeking]] - [[metacognition]] ## Connected Articles - [[agentic-workflows-education]] - [[collaborative-ai-tutoring]] ## Citation Naboulsi, A.Z. (2026). [*Agentic Education with AI Coding Assistants*](https://arxiv.org/abs/2604.17460) --- ## [Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named](https://edtechdev.github.io/aied/articles/agentic-literacy-debt/) **Agentic Literacy Debt** names a critical gap in the [[ai-literacy]] landscape that has become urgent with the rise of autonomous AI agents. Existing AI literacy frameworks assume humans evaluate AI outputs and then decide — they were built for a world of tools, not agents. But modern AI agents plan, decide, and act without step-by-step human approval, creating a structural deficit when deployed without corresponding literacy infrastructure. The debt compounds through three reinforcing channels: (1) normalization of opaque delegation, (2) multi-agent ecosystem complexity, and (3) institutional path dependence. Critically, the debt is incurred by deploying organizations but paid by users, patients, and citizens — a responsibility asymmetry that parallels arguments in [[genai-minoritized-knowledges-disability]] about who bears the costs of AI deployment. The paper reframes [[ai-literacy]] from an evaluative competency ("can you spot AI errors?") to a governance capability ("do you understand what you've delegated, and can you contest it?"). This connects to [[digital-literacy-illusion]] findings that students overestimate their AI readiness, and to [[over-reliance]] research showing that delegation without understanding produces dependency. Published in AI & Ethics, this paper has immediate implications for [[k-12]] and [[higher-ed]] AI literacy curricula, suggesting they must add agentic delegation concepts alongside traditional evaluation skills. ## Connected Concepts - [[ai-literacy]] - [[over-reliance]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[genai-minoritized-knowledges-disability]] - [[digital-literacy-illusion]] ## Citation Rohith Nama (2026). [Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named](https://arxiv.org/abs/2605.27396). arXiv:2605.27396. AI & Ethics. --- ## [Agentic Workflows in Education](https://edtechdev.github.io/aied/articles/agentic-workflows-education/) > A design framework for educational AI systems structured around four agentic paradigms: **reflection**, **planning**, **tool use**, and **multi-agent collaboration**. Proposed by Kamalov et al. (2026) as a taxonomy for analyzing how AI agents operate in learning environments. A design framework for educational AI systems structured around four agentic paradigms: **reflection**, **planning**, **tool use**, and **multi-agent collaboration**. Proposed by Kamalov et al. (2026) as a taxonomy for analyzing how AI agents operate in learning environments. ## Four Paradigms ### 1. Reflection Agents evaluate their own outputs against criteria before delivering feedback to learners. Reduces immediate error propagation but adds latency and requires internal evaluators. ### 2. Planning Agents decompose educational goals into sub-goals and sequence pedagogical actions. Enables structured tutoring but risks rigidity when learner states diverge from expected trajectories. ### 3. Tool Use Agents invoke external resources (calculators, code interpreters, knowledge bases) to ground responses in verified information. Critical for STEM domains where hallucination tolerance is low. ### 4. Multi-Agent Collaboration Multiple specialized agents (e.g., question generator, validator, explainer) coordinate to produce holistic tutoring experiences. Improves consistency over stand-alone LLMs but introduces orchestration complexity and interpretability challenges. ## Proof of Concept: MASS Kamalov et al. implemented a **multi-agent framework for automated essay scoring** (MASS) as a demonstration. Preliminary results suggest improved consistency compared to single-model approaches, though the authors flag the need for deeper research into interpretability and trustworthiness. ## Challenges - **Interpretability:** Multi-agent traces are harder to audit than single-model outputs. - **Trustworthiness:** Learners and teachers need transparency into which agent contributed what. - **Orchestration overhead:** Coordination cost scales non-linearly with agent count. - **Latency:** Reflection and multi-agent negotiation introduce response delays. ## Connected Concepts - [[adaptive-learning]] - [[ai-literacy]] - [[formative-assessment]] - [[human-in-the-loop-ai]] - [[pedagogical-llm-training]] - [[persistent-ai-agents-academic-research]] - [[agentic-ai]] - [[ai-education]] ## Connected Articles - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-powered-materials-discovery-ai-literacy]] — Preparing Students for AI-Powered Materials Discovery: A Workflow-Aligned Framework for AI Literacy, Equity, and Scie... - [[ai-tutor-authoring-promptdecipher]] — PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[authentic-assessment]] — Authentic Assessment - [[cognitive-agent-compilation]] — Cognitive Agent Compilation for Explicit Problem Solver Modeling - [[collaborative-ai-tutoring]] — Collaborative AI Tutoring - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[human-ai-co-mentorship]] — Human-AI Co-Mentorship in Project-Based Learning: A Case Study in Financial Forecasting - [[multi-agent-instructional-design]] — Multi-Agent Systems for Instructional Design - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[principled-ai-education]] — Principled AI in Education - [[programming-its]] — Programming Intelligent Tutoring Systems - [[teachbench-llm-teaching-evaluation]] — TeachBench - Evaluating LLM Teaching Ability - [[teacher-control-ai-generation-math-visuals]] — When Should Teachers Control AI Generation for Mathematics Visuals? - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning ## Citation Kamalov, F., Santandreu Calonge, D., Smail, L., Azizov, D., Thadani, D. R., Kwong, T., & Atif, A. (2026). [*Evolution of AI in Education: Agentic Workflows*](https://arxiv.org/abs/2504.20082v2). arXiv:2504.20082. --- ## [Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development](https://edtechdev.github.io/aied/articles/agents-that-teach-incidental-learning/) As AI coding agents take over substantial implementation work, developers increasingly lose the informal, effortful problem-solving through which software engineering expertise historically accumulated. The authors argue this "incidental learning" will not return spontaneously and that over-reliance on agentic coding lets unpracticed skills atrophy, accruing a developer-level analogue of Technical Debt they name **Knowledge Debt** — changes the agent executes that the developer cannot fully understand. They propose six design principles for learning-aware development and operationalize them in **SHIELD**, a multi-agent system that surfaces contextual, out-of-band learning moments drawn from the coding agent's own reasoning without disrupting developer flow. - **Incidental learning at risk:** Delegating coding to autonomous agents short-circuits the problem-solving pathway that traditionally built SE expertise, risking silent skill atrophy and accumulating [[over-reliance]]. - **Knowledge Debt concept:** A novel framing extending Technical Debt to the developer's understanding gap when agent-generated changes outpace comprehension. - **Design principles:** Six principles guide systems that consciously re-introduce learning into developer–agent interaction, relevant to [[agentic-education-coding]] and [[agentic-workflows-education]]. - **SHIELD prototype:** A "agents that teach" multi-agent system leverages the coding agent's own reasoning to surface learning moments, complementing [[ai-literacy]] goals for practitioners. - **Vision:** Learning-aware development environments where productivity and learning are complementary rather than competing — a theme echoed in [[self-regulated-learning]] and [[professional-training]]. ## Connected Concepts - [[over-reliance]] - [[ai-literacy]] - [[self-regulated-learning]] - [[professional-training]] ## Connected Articles - [[agentic-education-coding]] - [[agentic-workflows-education]] ## Citation Mehra, R., Suri, S., Tagadinamani, P. K., Singi, K., & Kaulgud, V. (2026). [Agents That Teach: Towards Designing Incidental Learning Back into AI-Assisted Software Development](https://arxiv.org/abs/2607.06101). arXiv:2607.06101. --- ## [AgentSchool: An LLM-Powered Multi-Agent Simulation for Education](https://edtechdev.github.io/aied/articles/agentschool-multi-agent-simulation-education-2026/) > Ye et al. (2026) introduce **AgentSchool**, an LLM-driven multi-agent [[simulating-students|simulator]] that models learning as **state transition rather than prompted behavior**. It couples cognitively growable student agents (weighted subject knowledge graphs, thinking-workflow pools, explicit misconceptions) with adaptive teacher agents that plan, scaffold, and reflect along the [[zone-of-proximal-development]], embedded in a configurable scenery generator and a multi-scale simulator. It produces more differentiated mastery and misconception traces than baseline simulators and generates plausible classroom social dynamics (peripheral participation, cliques, opinion-leader emergence). The paper argues that validating educational AI is uniquely hard: interventions act on developing learners whose trajectories are irreversibly shaped, while real-world trials are slow, ethically constrained, and institutionally locked. LLM-based simulators offer a remedy, but many collapse learning into persona-conditioned role-play and can structurally penalize institutional novelty. ## Architecture - **Cognitively growable student agents:** equipped with weighted subject knowledge graphs, thinking-workflow pools, and explicit misconceptions, so that their knowledge state changes as they "learn" (state transition, not just prompted persona). - **Adaptive teacher agents:** plan, scaffold, and reflect along the Zone of Proximal Development, adapting instruction to each simulated student. - **Configurable scenery generator:** situates instruction within both formal and informal learning fields. - **Multi-scale simulator:** decouples interaction scale, temporal granularity, and simulation duration. ## Findings - Structured student agents produce **more differentiated mastery and misconception traces** than a baseline simulator — i.e., more realistic variation across learners. - Teacher-agent comparisons show **backbone-dependent patterns consistent with ZPD-informed adaptation**. - The simulator generates **plausible social dynamics** — peripheral participation, clique formation, aggressor-induced cohesion, and opinion-leader emergence — consistent with classroom social theories. ## Implications AgentSchool reframes student simulation as **stateful learning rather than role-play**, addressing the validity concerns raised elsewhere in the [[simulating-students]] literature. It positions education as a testbed for long-horizon memory, multi-agent coordination, and institutional reasoning, while serving as a research instrument for validating educational AI and studying classroom dynamics. ## Connected Concepts - [[simulating-students]] - [[agentic-ai]] - [[zone-of-proximal-development]] - [[knowledge-graph]] - [[adaptive-learning]] - [[intelligent-tutoring]] - [[llm]] - [[collaborative-learning]] ## Connected Articles - [[simulating-students-diverse-cognitive-levels-2025]] — Embracing Imperfection: Simulating Diverse Cognitive Levels - [[simulating-students-llm-review-2026]] — Simulating Students with LLMs: A Review - [[valid-student-simulation-llm-2026]] — Towards Valid Student Simulation - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? - [[history-aware-student-simulation]] — History-Aware Profiles for Student Simulation - [[llm-student-simulation-teacher-insights]] — Can LLMs Simulate Human Learners? ## Citation Ye, Y., Li, W., Wen, Z., Huang, Y., Hu, Y., Wei, Z., Wang, Y., Xie, X., Yang, H., Huang, Y., Li, R., Qian, H., Song, Y., Jiang, B., Li, B., Li, L., Zhang, B., Cai, P., Xu, X., Chen, S., Hu, X., He, L., Zhou, A., Qu, J., Shao, J., & Wang, X. (2026). [*AgentSchool: An LLM-powered multi-agent simulation for education*](https://arxiv.org/abs/2605.30144). arXiv:2605.30144. --- ## [Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth](https://edtechdev.github.io/aied/articles/agreement-not-quality-llm-coding-verification/) > **Alex Liu, Lief Esbenshade, Michael Xiao, Victor Tian, Zachary Zhang, Kevin He, Min Sun** — arXiv preprint (2026). ## Synthesis This study challenges the standard practice of evaluating LLM qualitative coding by agreement with human coders, using data from a K-12 AI platform: five LLM systems and three trained human coders applied a 72-item hierarchical codebook to 2,560 educator messages. An independent domain expert judged 855 pairwise code-set comparisons blind to source, treating human and machine outputs symmetrically. Human-LLM agreement (mean Jaccard 0.30) fell well below human-human agreement (0.52), yet the blind verifier preferred human and LLM coding at indistinguishable rates (51.5% vs 48.5%, p = 0.537). A Bradley-Terry ranking placed two LLMs above two of three human coders, and for several substantive codes human consensus encoded shared bias that the verifier rejected in favor of the LLM interpretation — evidence that agreement metrics can mislead automation decisions. The study contributes a transferable blind-verification protocol for evaluating qualitative coding quality in AIED research, with implications for how LLM-assisted analysis of educator and student data should be validated. ## Connected Concepts - [[teacher-ai-competency]] - [[bias-mitigation]] - [[k-12-ai-education]] - [[student-experience]] - [[equity-in-ai-education]] - [[culturally-relevant-pedagogy]] - [[ai-education]] - [[formative-assessment]] ## Connected Articles - [[human-llm-collaborative-coding-k12-educator-ai]] — Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[civic-education-ai-lesson-plans]] — AI-Generated Lesson Plans in Civic Education - [[lodge-loble-cognitive-offloading-2026]] — Artificial intelligence, cognitive offloading and implications for education ## Citation Liu, A., Esbenshade, L., Xiao, M., Tian, V., Zhang, Z., He, K., & Sun, M. (2026). [Agreement is not quality: Blind expert verification of human and LLM qualitative coding](https://arxiv.org/abs/2607.28890). arXiv:2607.28890. --- ## [Perceptions and Acceptance of Artificial Intelligence in Science Education Programmes: Voices of Pre-Service Science Teachers](https://edtechdev.github.io/aied/articles/ai-acceptance-preservice-science-teachers-2026/) > **Synthesis:** This survey of 380 pre-service science teachers in Ghana, guided by UTAUT and the Theory of Planned Behaviour, finds generally positive perceptions of AI and strong intentions to use it, with ChatGPT the most frequently used tool for research, content explanation, and lesson planning. Positive attitudes and favorable effort expectancy were not fully matched by actual adoption, indicating an acceptance-to-use gap. ## Key Findings 1. **Positive perceptions.** Pre-service science teachers recognized AI's potential to improve teaching quality, instructional effectiveness, and student learning outcomes. 2. **ChatGPT dominance.** ChatGPT emerged as the most frequently used AI tool, with research, content explanation, and lesson planning as the primary purposes. 3. **Acceptance–use gap.** Despite positive attitudes, favorable effort expectancy, and strong behavioural intentions, actual adoption of AI remained uneven. ## Implications For [[teacher-ai-competency]] and [[teacher-role]] development, the findings point to a gap between teacher-candidates' enthusiasm for AI and their realized classroom use. The study's grounding in [[higher-ed]] and [[stem-education]] teacher preparation suggests that initial teacher education should move beyond awareness toward structured practice and [[faculty-development]] that closes the intention-to-use gap. Conducted in Ghana, the work also speaks to [[equity-in-ai-education]] and [[equity]] contexts where AI tools like ChatGPT are entering teacher education unevenly, with implications for [[educational-policy-ai]] and for building [[ai-literacy]] in institutions with limited infrastructure. ## Connected Concepts - [[teacher-ai-competency]] - [[teacher-role]] - [[ai-literacy]] - [[higher-ed]] - [[stem-education]] - [[equity-in-ai-education]] - [[faculty-development]] - [[educational-policy-ai]] ## Connected Articles - [[ai-adult-learning-design]] — AI in adult learning design - [[ai-education-global-capacity]] — Global capacity for AI education - [[genai-pd-ai-pck-learning-gain-2026]] — GenAI professional development and AI-PCK - [[ai-higher-ed-workforce-survey]] — AI in higher-ed workforce survey - [[elementary-writing-genai-systematic-review-2026]] — Elementary writing and GenAI review ## Citation Amponsah, K. D., Adu-Boahen, J., Commey-Mintah, P., Kumassah, E. K., Ayittey, R. F., & Nketsiah, J. (2026). [*Perceptions and acceptance of artificial intelligence in science education programmes: Voices of pre-service science teachers*](https://doi.org/10.31014/aior.1993.09.03.724). doi:10.31014/aior.1993.09.03.724. --- ## [The Main Barrier to AI Adoption in the Public Sector is Lack of Training](https://edtechdev.github.io/aied/articles/ai-adoption-training-public-sector/) Through Brazilian government case studies, demonstrates that a four-layer pedagogical methodology (Literacy, Protocol, Prompt Engineering, Audit) is the key to productivity gains (up to 50%), rather than premium models. This work emphasizes that [[ai-literacy]] is a developmental capacity requiring structured [[scaffolding]] and [[prompt-engineering]] discipline. It connects to the need for [[curriculum-design]] that targets [[metacognition]] and [[agentic-ai]] rather than just syntax mastery. ## Key Findings - Across two Brazilian federal-district government units with distinct mandates, official indicators recorded by the SEI-GDF system showed substantial productivity gains after the training method was applied: average case processing time fell by 18.2% at the Sectoral Internal Control Office of the Federal District Department of Health (SES/CONT) during 2024, and by 50% at the Internal Control Unit of the Federal District Department of Economic Development, Labor and Income (UCI/SEDET) during 2025. - UCI/SEDET's gains were not only temporal: technical-report production rose 92%, the unit issued 288 formal recommendations to public managers, and it analyzed cases totaling US$104.3 million in financial volume — an average of 2.27 recommendations per report, indicating increased analytical intensity rather than mere output volume. - In neither unit did internal control mechanisms identify any information-security incident, sensitive-data leakage, or formal compliance challenge from external oversight bodies during the period examined. - The determining barrier to adoption observed in these units was not technological but training-related: free, browser-accessible AI models were available to every public servant, yet were not being used productively until a structured pedagogical method was introduced. - The four-layer method — literacy, protocol, prompt engineering, and audit — was designed to comply with international and national data-protection law and with the principles of public administration, supporting its portability across agencies with distinct mandates. - A note-by-note review of the 288 recommendations (with classification by nature and materiality) identified material implications for US$48.1 million in payments and contracts analyzed; applying a probability matrix calibrated to public-audit literature, potential mitigation is estimated between US$1.2 million (conservative) and US$5.7 million (optimistic), with a central estimate of US$3 million. ## Study Design & Method The paper reports two auditable, third-party-verifiable cases rather than a controlled experiment. The method was applied throughout 2024 at SES/CONT — a multidisciplinary team with no homogeneous legal background that was manually processing a caseload whose average processing time the SEI-GDF recorded at 17 days, 22 hours, and 11 minutes — and throughout 2025 at UCI/SEDET. Outcomes were drawn from the official indicators of the Federal District Government's Electronic Information System (SEI-GDF). The two units differed in profile: at SES/CONT the method reduced processing time while keeping documentary output stable, whereas at UCI/SEDET the time reduction was accompanied by simultaneous growth in document volume and analytical depth. ## Implications for AI in Education The study is a concrete demonstration that [[ai-literacy]] behaves as a developmental capacity: the availability of capable models does not translate into adoption or productivity without structured [[scaffolding]]. For education, the four-layer sequence — literacy, protocol, prompt engineering, and audit — maps onto the kind of [[curriculum-design]] that pairs tool competence with disciplined workflow and verification. The absence of security incidents shows that productivity-oriented training can be compatible with rigorous governance, and the use of free models makes the approach accessible to organizations under budget constraints, a relevant consideration for resource-limited educational settings. The finding that gains differed across units reinforces the need to tailor training to institutional context rather than assuming one-size-fits-all transfer. ## Connected Concepts - [[ai-literacy]] - [[scaffolding]] - [[prompt-engineering]] - [[curriculum-design]] - [[metacognition]] - [[agentic-ai]] ## Connected Articles - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing - [[ase-26-agentic-software-engineering-curriculum]] — ASE-26: A Curriculum for Agentic Software Engineering as a Discipline - [[guided-llm-scaffolding-independent-learning]] — Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI ## Citation Vinicius Santana Gomes (2026). [The Main Barrier to AI Adoption in the Public Sector is Lack of Training](https://arxiv.org/abs/2606.01517). arXiv:2606.01517. --- ## [Guidelines for Designing AI Technologies to Support Adult Learning](https://edtechdev.github.io/aied/articles/ai-adult-learning-design/) > A set of 19 empirically-grounded design guidelines for AI-supported learning technologies tailored to adult learners, synthesized by Reddig et al. (2026) from longitudinal deployment data at a US national research institute focused on adult learning and online education. # AI Design Guidelines for Adult Learning ## Definition A set of 19 empirically-grounded design guidelines for AI-supported learning technologies tailored to adult learners, synthesized by Reddig et al. (2026) from longitudinal deployment data at a US national research institute focused on adult learning and online education. ## The Adult Learning Gap Most AI educational technology is designed for and evaluated in K-12 contexts. Adult learners have fundamentally different needs: time constraints from work/family, prior knowledge and experience, self-directed learning goals, and different motivation structures. This connects to [[personalized-learning]] but requires distinct design considerations beyond what works for younger learners. ## Design Guidelines Derived from reflexive thematic analysis of deployment data across multiple AI learning technologies. The 19 guidelines address recurring challenges including: - Accommodating variable time availability - Leveraging prior knowledge and experience - Supporting self-directed goal-setting - Providing just-in-time rather than just-in-case learning - Integrating with workplace contexts ## Validation A heuristic evaluation of the original deployed systems against the guidelines demonstrated their practical applicability. A companion exploration tool links guidelines to stakeholder statements, supporting designers in contextualizing recommendations. ## Connections to Wiki - Extends [[personalized-learning]] into the adult learning domain - Complements [[adaptive-learning]] with adult-specific design principles - Relevant to [[faculty-development]] as faculty themselves are adult learners - Addresses gaps in [[lifelong-learning]] technology design ## Connected Concepts - [[personalized-learning]] - [[adaptive-learning]] - [[faculty-development]] - [[lifelong-learning]] ## Connected Articles - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers'' practices and perceptions - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[golrang-propact-pair-programming-2026]] — ProPACT: Pair Programming with AI - [[cognitive-agent-compilation]] — Cognitive Agent Compilation for Explicit Problem Solver Modeling - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition Support Methods Using Deep Learning Audio-Video Processing Techniques - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components ## Citation Reddig, J.M., Jr, G.R.S., authors), S.A.S.E.A., & 2026, D. (2026). [*Guidelines for Designing AI Technologies to Support Adult Learning*](https://arxiv.org/abs/2605.04616) --- ## [Guidelines for Designing AI Technologies to Support Adult Learning](https://edtechdev.github.io/aied/articles/ai-adult-learning-guidelines-dis2026/) > **Synthesis:** Drawing on longitudinal deployment data from the National AI Institute for Adult Learning and Online Education (AI-ALOE), this DIS 2026 paper synthesizes 19 empirically grounded design guidelines for AI-powered adult learning technologies. The guidelines span cognitive, social, and teaching presence dimensions and are derived from reflexive thematic analysis of ~1,600 stakeholder statements across seven deployed systems. The work also provides a heuristic evaluation method and an interactive guideline exploration tool. ## Context and Motivation AI-powered educational technologies have demonstrated benefits but are predominantly designed and evaluated in K-12 contexts. Adult learners differ motivationally and contextually: they are often self-directed, goal-oriented (career advancement, reskilling), and must balance education with employment and family responsibilities. Existing systems inherit design patterns from K-12 that are poorly aligned with adult learning constraints. This paper examines seven AI-powered technologies deployed within **AI-ALOE**, a US National AI Institute: - **Apprentice Tutors** — web-based ITS for adult math/STEM practice - **iTELL** — Intelligent Texts for Enhanced Lifelong Learning - **Ivy** — interactive video-based AI coaching for procedural skills - **Jill Watson** — RAG-powered Q&A agent for online courses - **SAMI** — social agent for online discussion forums - **SMART** — Student Mental Model Analyzer for Research and Teaching - **VERA** — conceptual modeling tool for guided inquiry ## Methodology Using **reflexive thematic analysis**, the team analyzed approximately 1,600 stakeholder statements from focus groups with learners and instructors, technical artifacts, and progress reports. Statements were organized through affinity diagramming into sub-themes, needs statements, and finally 19 design guidelines framed as "AI tools should..." Each guideline is labeled by Community of Inquiry dimension (cognitive/social/teaching presence) and most-impacted stakeholder. ## The 19 Design Guidelines The guidelines span four categories: ### Cognitive Presence - **G2:** AI tools should be accessible and fit into the busy lives of adult learners (mobile-friendly, offline-capable, affordable) - **G3:** AI tools should be informed by learning science and learning theories - **G4:** AI tools should be easy to understand and frictionless to use ### Teaching Presence - **G7:** AI tools should support learner motivation and engagement - **G8:** AI tools should align with established instructional best practices - **G9:** AI tools should support diverse pedagogical strategies - **G11:** AI tools should personalize the learning experience - **G13:** AI tools should provide substantive educational support (not just surface-level assistance) - **G16:** AI tools should align with instructors' personal instructional approach - **G17:** AI tools should provide meaningful feedback and explanations ### Social Presence - **G18:** AI tools should scaffold and support learners in developing their social competencies - **G19:** AI tools should foster social connection and community ### Cross-Cutting - **G1:** AI tools should be transparent about data practices (collection, storage, access) - **G6:** AI tools should provide scaffolded support that adapts as learners progress - **G12:** AI tools should connect content to real-world problems that matter to adult learners - **G14:** AI tools should support learning engineering through actionable data - **G15:** AI tools should integrate easily with existing educational ecosystems ## Key Findings - **Stakeholder priorities diverge**: Instructors focused on usability (G4) and instructional alignment (G16); students emphasized educational support (G13) and community-building (G18, G19); researchers prioritized learning theories (G3) and best practices (G8-G10). - **No single technology satisfied all 19 guidelines** — but the broader AI-ALOE ecosystem collectively covered the full set. - **Personalization (G11)** had low satisfaction across systems: most deployed surface-level personalization (adapting examples, knowledge checks) rather than deeper adaptations like task sequencing or difficulty calibration. - **Data transparency (G1)** and **social/community features (G18, G19)** consistently scored lowest across the heuristic evaluation. ## Design Implications 1. **Design for adult life constraints** — mobile access, offline support, and asynchronous availability are not nice-to-haves; they are prerequisites for adult learners balancing work, family, and education. 2. **Ground AI tools in andragogy**, not just pedagogy — adult learners are self-directed, problem-oriented, and bring life experience that shapes how they engage with technology. 3. **Heuristic evaluation using these 19 guidelines** provides a structured method for assessing adult learning technologies against empirically grounded criteria. 4. **The guideline exploration tool** connects abstract design principles to concrete stakeholder quotes, supporting ideation and technology refinement. ## Connected Concepts - [[adult-learning]] - [[ai-education]] - [[adult-learning]] - design-guidelines - educational-technology - [[higher-ed]] - human-computer-interaction - [[professional-training]] - [[rag]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions ## Citation 13-17, V.D.J., Adult, G.F.D.A.T.T.S., Reddig∗, J., Computing, I., Georgia, A., & Morris, W.G. (2026). [*Guidelines for Designing AI Technologies to Support Adult Learning*](https://doi.org/10.1145/3800645.3813102). Designing Interactive Systems Conference (DIS ’26), June 13–17 --- ## [Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers](https://edtechdev.github.io/aied/articles/ai-agents-constructive-conflict-design-education-2026/) > **Synthesis:** Investigates adversarial AI design agents that enact constructive conflict to prompt reconsideration in novice designers. Between-subjects experiment (N=48) comparing adversarial vs. cooperative AI agent roles. Adversarial agent condition produced significantly more design iterations, broader exploration of alternatives, and higher-rated final designs. Participants reported the conflict agent was frustrating but ultimately helpful — a productive friction dynamic. Connects to adversarial design theory and Socratic tutoring. [[agentic-ai]], [[scaffolding]], [[student-experience]], [[design-thinking]], and [[socratic-method]]. Investigates adversarial AI design agents that enact constructive conflict to prompt reconsideration in novice designers. Between-subjects experiment (N=48) comparing adversarial vs. cooperative AI agent roles. Adversarial agent condition produced significantly more design iterations, broader exploration of alternatives, and higher-rated final designs. Participants reported the conflict agent was frustrating but ultimately helpful — a productive friction dynamic. Connects to adversarial design theory and Socratic tutoring. ## Connected Concepts - [[agentic-ai]] - [[scaffolding]] - [[student-experience]] - [[design-thinking]] - [[socratic-method]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[knowloop-confusion-to-consolidation-2026]] — From Confusion to Consolidation: A Staged Conversational Workflow for Post-Lecture Review - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review ## Citation Howard Ziyu Han, & Nikolas Martelaro (2026). [Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers](https://arxiv.org/abs/2608.04166). arXiv:2608.04166. Human-Agent Interaction (HAI) 2026. --- ## [When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community](https://edtechdev.github.io/aied/articles/ai-agents-peer-learning-discourse/) > **Authors:** Eason Chen, Ce Guan, A Elshafiey, Zhonghao Zhao, Joshua Zekeri, Afeez Edeifo Shaibu, Emmanuel Osadebe Prince **Year:** 2026 **Venue:** arXiv (cs.HC) > Mining discourse from Moltbook, a social network of over 2.4 million AI agents, reveals peer-learning-like dynamics (validation 22%, knowledge extension 18%) across 28,683 posts and yields six design hypotheses for educational AI. ## Key Findings - Educational data mining of Moltbook, a social network where over 2.4 million AI agents share skills, discoveries, and collaboratively discuss knowledge, identified discourse that structurally resembles human peer learning, in which participants alternate between teacher and learner roles. - Analysis of 28,683 posts (after filtering automated spam) and 138 comment threads, using statistical and qualitative methods, found responses distributed across a taxonomy: validation (22%), knowledge extension (18%), application (12%), and metacognitive responses (7%). - The discourse was heavily statement-driven, with a statement-to-question ratio of 11.4:1, indicating that agents predominantly asserted and shared knowledge rather than asking questions. - The paper derives six design hypotheses for educational AI from these observations, connecting agent-community dynamics to the design of AI systems for learning. - The observed patterns echo established peer-learning benefits — explaining benefits the "teacher" and personalized instruction benefits the "learner" — suggesting that agent discourse patterns can inform expectations for multi-agent educational systems. ## Study Design & Method The study applies educational data mining to Moltbook, a large-scale community of AI agents built as a social network. Researchers filtered automated spam, then analyzed 28,683 posts and 138 comment threads using a combination of statistical and qualitative methods. A response taxonomy was used to classify how agents respond to one another, and ratio analyses (statement-to-question) captured the overall shape of the discourse. The work is explicitly grounded in the peer-learning literature, where learners teach and learn from each other, share skills, and collaboratively construct understanding. ## Implications for AI in Education For AI in education, the Moltbook analysis suggests that multi-agent systems can exhibit peer-learning-like dynamics at scale, with implications for how [[agentic-ai]] systems might be designed to support [[collaborative-learning]] rather than isolated question-answering. The dominance of validation and knowledge-extension over metacognitive responses (only 7% of the taxonomy) highlights a gap: even well-organized agent discourse leans toward assertion, so platforms built on agent communities may need explicit design pressure toward questioning, explanation, and metacognitive engagement. The six design hypotheses provide a starting point for such design work, and the study demonstrates the value of [[learning-analytics]] methods for inspecting agent behavior at scale. ## Connected Concepts - [[agentic-ai]] - [[collaborative-learning]] - [[learning-analytics]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[administrator]] - [[help-seeking]] ## Connected Articles - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[engagement-forecasting-its]] — From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning - [[interactive-learning-dashboards-engagement]] — Interactive learning dashboards: rethinking learning visualisations as engagement tools - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents - [[scheu-mobile-chatbot-journaling-motivation-2026]] — Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement ## Citation Eason Chen et al. (2026). [When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community](https://arxiv.org/abs/2602.14477). arXiv:2602.14477. cs.HC. --- ## [AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice](https://edtechdev.github.io/aied/articles/ai-assessment-human-tutors/) AI-driven assessment of human tutor training performance correlates with real-life tutoring quality; bridges the gap between training metrics and classroom practice. ## Synthesis AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice investigates ai-driven assessment of human tutor training performance correlates with real-life tutoring quality; bridges the gap between training metrics and classroom practice. This work connects to existing research on intelligent-tutoring-systems by demonstrating that Abstract:There exist numerous tutor training platforms. However, few provide AI-driven training and evaluation for human tutors based on real-life performance. We present an AI-driven system that assesses both open responses during training and authentic real-life tutoring. Unlike platforms that onl.... ## Connected Concepts - [[ai-feedback-quality]] - [[automated-assessment]] - [[scaffolding]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[learning-analytics]] ## Connected Articles - [[tutoring-effectiveness-index]] — The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals - [[teaching-feedback-classification-benchmark]] — A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol - [[sequenced-ai-feedback-learning]] — Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI ## Citation Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Clara Brandt, Conrad Borchers, Kenneth R. Koedinger (2026). [AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice](https://arxiv.org/abs/2606.18617). arXiv:2606.18617. --- ## [A bit of chaos and madness: The AI Assessment Scale and the work of assessment reform](https://edtechdev.github.io/aied/articles/ai-assessment-scale-reform/) 📄 [PDF](https://arxiv.org/pdf/2606.26729) This study examines the implementation of the Artificial Intelligence Assessment Scale (AIAS), a structured framework for redesigning [[assessment|university assessment]] in response to [[generative-ai|generative AI]]. Surveying 80 academic staff, the researchers found that while the framework's transparency and guidance were valued, implementation was hampered by departmental inconsistencies, workload pressures, and uncertainty about appropriate AI use levels. The study frames [[assessment|assessment reform]] through the lens of 'assessment security', conceptualizing the work of redesign as a security practice in [[higher-ed|higher education]]. Staff described the process as 'a bit of chaos and madness', capturing both the disruptive potential and the lack of coordinated institutional support. This connects to broader challenges around [[academic-integrity|academic integrity]] in the age of generative AI. The findings have implications for [[teacher-role|faculty development]] and [[educational-policy-ai|institutional policy]], suggesting that successful AI assessment frameworks require not just clear guidelines but also adequate resourcing, departmental alignment, and ongoing professional support. The study contributes to the growing literature on implementation science for [[ai-literacy|AI literacy]] in higher education. ## Connected Concepts - [[assessment]] - [[generative-ai]] - [[higher-ed]] - [[academic-integrity]] - [[teacher-role]] - [[educational-policy-ai]] - [[ai-literacy]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata ## Citation Mike Perkins, Darius Postma, Jasper Roe, Susan Sisay, Craig Holdcroft (2026). ['A bit of chaos and madness': The AI Assessment Scale and the work of assessment reform](https://arxiv.org/abs/2606.26729). arXiv:2606.26729. arXiv cs.HC. --- ## [AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education](https://edtechdev.github.io/aied/articles/ai-assistance-discretionary-feedback/) This field experiment shows that AI-generated feedback drafts can measurably increase the rate and length of feedback that teaching assistants actually deliver to students, without sacrificing perceived usefulness or instructor time-efficiency. The design keeps humans firmly in the loop: TAs could edit or discard every draft, and the intervention still produced significant gains. The finding connects directly to [[automated-grading]] and [[ai-feedback-quality]] debates: AI does not replace the grader here, but reduces the activation energy for starting a feedback document. TAs treated drafts as editable scaffolds rather than authority, which aligns with [[teacher-role]] research on maintaining instructor agency. The +10.8pp provision effect and +39.8-char length increase are rare quantified benchmarks for discretionary AI assistance in education. Because the study measured usefulness ratings alongside quantity, it also informs the [[feedback-loop]] literature: more feedback is not automatically better feedback, yet the null result on student ratings suggests the drafts did not degrade quality. The mixed-methods design bridges efficacy-study and [[rct]] traditions in AIED evaluation. Practical implication: if deployed at scale, AI feedback scaffolding could be especially valuable in large-enrollment [[higher-ed]] courses where TA time is scarce but personalized feedback is pedagogically important. Future work should test whether gains persist across semesters and whether different subject domains moderate the effect. ## Connected Concepts - [[automated-grading]] - [[ai-feedback-quality]] - [[teacher-role]] - [[feedback-loop]] - [[rct]] - [[higher-ed]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[teaching-feedback-classification-benchmark]] — A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers'' practices and perceptions - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education ## Citation Romina Mahinpei, Victoria Dean, Ruth Fong, Lydia T. Liu, Manoel Horta Ribeiro (2026). [AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education](https://arxiv.org/abs/2606.03095). arXiv:2606.03095. arXiv. --- ## [Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education](https://edtechdev.github.io/aied/articles/ai-assisted-collaborative-learning-model-dbr/) > **Synthesis:** Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education ## Key Findings - Design-Based Research (DBR) study developing and evaluating an **AI-Assisted Collaborative Learning (AACL) Model** across a single implementation cycle: an **eight-week intervention** with undergraduate students at an Indonesian public university, structured around authentic problem-based tasks with students organized into groups of four to five. - Expert validation by three specialists (educational technology, instructional design, AI in education) rated the model **highly valid with an overall score of 92.4%** — Learning Design 94.1%, AI Integration 91.6%, Collaborative Learning 93.2%, and Activities 90.7% — with minor revisions focused on collaborative reflection, prompt-design guidelines, and ethical AI usage instructions. - **Critical thinking mean scores rose from 68.21 (SD = 7.34) to 84.63 (SD = 6.28), a 24.1% gain**, with the largest improvements in evidence evaluation (66.91 → 85.27, +27.4%) and analysis (67.84 → 84.11, +24.0%). - **Problem-solving performance rose from an initial project average of 70.14 to 88.72 after model refinement**; the strongest final criteria scores were team collaboration (90.02) and decision justification (89.41). - Student perceptions (five-point Likert questionnaire) were highly positive: **overall satisfaction 4.47**, learning motivation 4.54, collaborative learning support 4.51, problem-solving support 4.48, critical thinking enhancement 4.43, and ease of AI use 4.36. - Qualitative analysis surfaced three recurring themes: students viewed AI as an **effective brainstorming partner that stimulated idea generation rather than replacing independent thinking**; AI-generated responses made collaborative interaction more active; and students became increasingly aware of the importance of **verifying AI-generated information against academic literature** before group decisions. ## Study Design & Method The study followed a Design-Based Research methodology with four iterative phases — needs analysis, model design, classroom implementation, and model refinement — combining iterative design cycles with continuous evaluation. The model was built on constructivist learning principles, collaborative knowledge construction, and human–AI collaboration, positioning generative AI as an "intelligent learning partner" that supports idea generation, information analysis, reflection, and collaborative decision-making while students remain the primary decision-makers. The eight-week implementation used a four-stage learning cycle: (1) problem identification (exploring authentic cases and formulating research questions); (2) AI-assisted collaborative inquiry (groups gather information and compare viewpoints); (3) collaborative problem-solving (analyzing evidence, evaluating AI-generated responses, proposing solutions); and (4) reflection and presentation (peer feedback and reflection on AI-assisted collaboration). Instructors acted as facilitators, monitoring AI usage and encouraging critical assessment of AI content. Data sources included expert validation, pre- and post-intervention critical thinking and problem-solving assessments, structured questionnaires, classroom observations, reflective journals, focus-group discussions, and collaborative project reports. Qualitative data were analyzed thematically; quantitative data used descriptive statistics and paired-sample analysis (specific test statistics not reported in the paper), with findings triangulated across sources. ## Key Results - **Model development:** needs-analysis interviews found collaborative activities were often limited to information sharing and that generative AI use in instruction was largely unstructured and focused on content generation rather than collaborative knowledge construction — motivating the integrated AACL framework. - **Critical thinking gains:** pre/post assessments covering analytical reasoning, evidence evaluation, argument construction, and reflective judgment showed substantial improvement across all dimensions; classroom observations tracked a shift from accepting AI outputs uncritically to questioning accuracy, comparing alternative viewpoints, and supporting arguments with empirical evidence. - **Problem-solving gains:** students increasingly used AI to generate multiple perspectives rather than seek single answers, and group discussions became more analytical as learners debated AI-generated recommendations and selected solutions based on academic evidence and collaborative reasoning. - **Implementation challenges:** students initially showed varying levels of AI literacy, requiring additional instructional support in prompt engineering and information verification; some groups relied excessively on AI-generated content before gradually developing critical evaluation strategies. ## Implications for AI in Education The study demonstrates that **AI functions most effectively as a collaborative learning partner when embedded in structured pedagogical activities** rather than used as an isolated tool. The four-stage cycle offers a concrete, transferable instructional model for [[higher-ed]] institutions seeking responsible AI integration: authentic collaborative tasks that require evidence evaluation, decision justification, and reflection on AI-generated information. Rather than emphasizing technical AI proficiency alone, instructors should design structured prompt guidance, peer discussion, and reflective evaluation into collaborative activities to strengthen higher-order thinking — directly connecting to [[collaborative-learning]] and [[critical-thinking]] research. The 92.4% expert validity score and positive student perceptions support the feasibility of the approach, while the observed over-reliance early in the intervention suggests pairing [[generative-ai]] tools with explicit AI literacy training and collaborative assessment rubrics. ## Limitations - The study reports a single DBR implementation cycle within one course and institution; the participants section does not report a sample size, and the paired-sample statistical analysis is described without reporting test statistics. - Students' initially varying AI literacy required extra instructional support, and some groups over-relied on AI-generated content before developing critical evaluation strategies. - The authors call for future iterations with explicit AI literacy training, collaborative assessment rubrics, longitudinal implementation across multiple academic disciplines, and additional research with larger samples and comparative experimental designs to strengthen the empirical evidence. ## Connected Concepts - [[higher-ed]] - [[collaborative-learning]] - [[critical-thinking]] - [[generative-ai]] - [[socratic-method]] - [[math-education]] - [[creativity]] - [[prompt-engineering]] ## Connected Articles - [[ai-collaborative-learning-skills-impacts]] — Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving - [[llm-critical-thinking-teamwork-review]] — Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review - [[ai-chatbot-collective-efficacy-collaborative-learning]] — AI chatbot design principles to enhance the collective efficacy in collaborative learning - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[ai-collaborative-learning-systematic-review]] — A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade ## Citation Putra, A. D., Wijanarko, F., & Safitri, N. (2026). [*Design-based research for developing an AI-assisted collaborative learning model to enhance critical thinking and problem-solving skills in higher education*](https://educationresourcescenter.com/jel/article/view/59). --- ## [An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in high school students](https://edtechdev.github.io/aied/articles/ai-assisted-learning-modes-eeg/) 📄 [PDF](https://arxiv.org/pdf/2606.26579) This study investigates how different modes of AI interaction affect cognitive engagement and learning outcomes in high school students. Using a within-subjects design with 24 students, the researchers compared three conditions: **Auto mode** (AI solves problems independently), **Interactive mode** (student-AI collaboration with scaffolding), and **Manual mode** (no AI assistance). The Interactive mode produced the highest cognitive engagement and task accuracy, while the Auto mode led to reduced engagement and potential [[over-reliance]]. The use of [[affective-computing|electroencephalography (EEG)]] provides a neurophysiological dimension to understanding AI's impact on learning. Though EEG results did not reach statistical significance, descriptive patterns suggested differences in neural activity across the three AI interaction modes. This connects to broader debates about [[student-experience|student-AI interaction]] design in [[k-12]] classrooms. The finding that full automation reduces cognitive engagement echoes the [[over-reliance]] concerns documented in prior research on AI tutoring systems. The study's framework for categorizing AI interaction modes (Auto, Interactive, Manual) provides a replicable methodology for future human-AI interaction research in educational settings. ## Connected Concepts - [[over-reliance]] - [[affective-computing]] - [[student-experience]] - [[k-12]] ## Connected Articles - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[scheu-mobile-chatbot-journaling-motivation-2026]] — Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[brookings-ai-students-report]] — A New Direction for Students in an AI World: Prosper, Prepare, Protect - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Kashika Khurana, Ally Liew (2026). [An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in high school students](https://arxiv.org/abs/2606.26579). arXiv:2606.26579. arXiv cs.HC / cs.CY. --- ## [Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis](https://edtechdev.github.io/aied/articles/ai-assisted-se-curriculum-syllabus-analysis-2026/) > **Synthesis:** This paper analyses 23 publicly available syllabi from upper-division, credit-bearing university courses that teach AI-assisted software development. The study identifies common curricular themes — prompt engineering, code review with AI, AI-augmented testing, ethical considerations — and maps how different institutions are defining this emerging subject area. Key findings include: a strong emphasis on critical evaluation of AI-generated code over pure generation speed, widespread integration of human-AI collaboration workflows, and substantial variation in how ethics and professional responsibility are addressed. The authors derive design guidance for future AI-assisted SE curricula, emphasising the need to balance tool fluency with foundational software engineering knowledge. As Generative AI coding tools reshape professional software development, universities have begun designing courses to prepare students for AI-assisted development workflows. By analyzing the syllabi of these courses, we can gather empirical evidence about these courses, reveal how this emerging curricular area is being defined, and gain guidance for future curriculum design. We analyzed 23 publicly available syllabi and course materials of upper-division, credit-bearing courses that meet specific criteria, including explicitly addressing Generative AI in software engineering. Through iterative qualitative coding, we characterized courses' learning objectives, assessments, topics, and documented AI tools. Our analysis reveals commonalities and differences among these courses that allow rese analyses 23 publicly available syllabi from upper-division, credit-bearing university courses that teach AI-assisted software development. The study identifies common curricular themes — prompt engineering, code review with AI, AI-augmented testing, ethical considerations — and maps how different institutions are defining this emerging subject area. Key findings include: a strong emphasis on critical evaluation of AI-generated code over pure generation speed, widespread integration of human-AI collaboration workflows, and substantial variation in how ethics and professional responsibility are addressed. The authors derive design guidance for future AI-assisted SE curricula, emphasising the need to balance tool fluency with foundational software engineering knowledge. ## Connected Concepts - [[curriculum-design]] - [[instructional-design]] - [[higher-ed]] - [[accessible-learning]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[rct]] - [[active-learning]] ## Connected Articles - [[curriculum-as-code-instructional-design-2026]] — Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-marketing-education-roles-2026]] — When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education - [[learnity-graphs-lifelong-learning-framework-2026]] — Rethinking Higher Education: From Fixed Curricula to Learnity Graphs - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights ## Citation Geng, Francis et al. (2026). [Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis](https://arxiv.org/abs/2608.05898). arXiv:2608.05898. --- ## [Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams](https://edtechdev.github.io/aied/articles/ai-assisted-writing-research-teams/) **AI-Assisted Writing Transforms Research Teams** challenges the longstanding "Big Science" trend toward ever-larger teams, showing that AI writing tools enable smaller, younger research teams to produce highly impactful publications. Analyzing 147,074 full-text publications from PLoS and Nature portfolio journals since 2020, the study uses propensity score matching and multiple regression methods to demonstrate that AI-assisted writing is associated with more compact, junior-leaning teams — and counterintuitively, higher probability of producing impactful work. This has direct implications for [[higher-ed]] doctoral training and [[faculty-development]]: if junior researchers can produce frontier-quality work with smaller teams and AI assistance, the traditional apprenticeship model of large lab groups may need rethinking. The findings connect to [[ai-productivity-moderation]] research showing that AI productivity gains are not automatic, and to [[persistent-ai-agents-academic-research]] findings on how AI is reshaping research workflows. For [[writing-education]], the democratization of research writing through AI tools raises questions about how [[generative-ai]] reshapes the development of scholarly writing skills and the [[faculty-development]] needed to mentor AI-augmented researchers. ## Connected Concepts - [[higher-ed]] - [[faculty-development]] - [[writing-education]] - [[generative-ai]] ## Connected Articles - [[ai-productivity-moderation]] - [[persistent-ai-agents-academic-research]] ## Citation Haoyang Wang, Mingze Zhang, Yi Bu, Star Xing Zhao, Meijun Liu (2026). [Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams](https://arxiv.org/abs/2605.27404). arXiv:2605.27404. arXiv preprint. --- ## [AI-Assisted Autonomous Learning and Reduced Academic Accomplishment in Vocational Higher Education: The Mediating Role of Hardiness](https://edtechdev.github.io/aied/articles/ai-autonomous-learning-accomplishment-2026/) > **Synthesis:** Wang and Zhang (2026) examined how AI-assisted autonomous learning relates to reduced academic accomplishment among 1,264 vocational college students in China, focusing on the mediating role of hardiness (commitment, control, challenge). Using structural equation modeling, they found AI-assisted autonomous learning was negatively associated with hardiness and positively associated with reduced academic accomplishment, with hardiness partially mediating the relationship — the indirect effect accounted for a substantial proportion of the total effect. The study extends research on AI in higher education by suggesting AI-assisted learning has implications beyond efficiency and convenience for students' psychological resources and academic self-evaluation. ## Key Findings - AI-assisted autonomous learning was negatively associated with hardiness and positively associated with reduced academic accomplishment among vocational college students. - Hardiness was negatively associated with reduced academic accomplishment and partially mediated the AI-learning → reduced-accomplishment relationship, with the indirect effect accounting for a substantial proportion of the total effect. - The study frames AI-assisted autonomous learning as a double-edged condition: it can support efficiency, information access, and personalized assistance, but may also reduce students' opportunities for independent thinking, effortful problem solving, and the experience of self-generated mastery. - Reduced academic accomplishment — a core dimension of academic burnout reflecting negative self-evaluation of competence and learning effectiveness — was the focal outcome, particularly salient for vocational students whose confidence is tied to skill development and professional identity. - The authors call for studying AI beyond technology acceptance and learning efficiency, toward its associations with psychological resources and academic self-perception. ## Study Design & Method The study collected survey data from 1,264 students at a vocational college in China and analyzed the relationships among AI-assisted autonomous learning, hardiness, and reduced academic accomplishment using structural equation modeling (SEM). AI-assisted autonomous learning was operationalized as students' active use of generative AI tools to obtain information, complete tasks, solve problems, and regulate learning. Hardiness (Kobasa's commitment/control/challenge disposition) was treated as the proposed psychological mechanism, and reduced academic accomplishment (a Maslach-derived dimension of academic burnout) as the self-evaluative outcome. The mediation model tested both the direct path and the indirect path through hardiness. ## Implications for AI in Education The findings caution that heavy reliance on [[generative-ai]] for autonomous learning may come at the cost of learners' psychological resources and their sense of [[ai-misuse-learning-harm|meaningful, self-generated accomplishment]]. For vocational and skills-based education, where competence is built through repeated practice and active problem solving, using AI as a substitute for learning may reduce hardiness and the experience of mastery. The study supports designing AI integration that preserves effortful engagement, independent problem solving, and opportunities to face and overcome challenge, while connecting these concerns to academic burnout and [[over-reliance]]. It positions responsible AI integration as a matter not just of policy and efficiency but of students' academic adaptation and self-evaluation. ## Limitations The cross-sectional design precludes causal inference; the negative associations observed cannot establish that AI use causes reduced accomplishment. All constructs rely on self-report, and the sample is drawn from a single vocational college in China, bounding generalizability. Hardiness is treated as a relatively stable disposition, which may understate its plasticity over time. The study did not distinguish between different types or intensities of AI use, which could moderate the observed relationships. ## Connected Concepts - [[generative-ai]] - [[over-reliance]] - [[ai-misuse-learning-harm]] - [[self-regulated-learning]] - [[cognitive-offloading]] - [[motivation]] - [[higher-ed]] ## Connected Articles - [[genai-over-reliance-learning-2026]] — From Enhancement to Over-Reliance: Generative AI and Sustainable Learning Performance - [[ai-use-critical-thinking-medical-students-2026]] — From AI Use to Critical Thinking Among Medical Students - [[ai-availability-student-motivation]] — "Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning When Using Generative AI - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI ## Citation Wang, W., & Zhang, Q. (2026). [*AI-assisted autonomous learning and reduced academic accomplishment in vocational higher education: The mediating role of hardiness*](https://doi.org/10.3389/fpsyg.2026.1848291). --- ## [Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming](https://edtechdev.github.io/aied/articles/ai-availability-student-motivation/) **Tran, Harper & Price (2026)** examine a pressing motivational paradox in contemporary computing education: the ready availability of AI tools that can complete programming assignments undermines students' willingness to invest effort in developing their own skills. Drawing on self-determination theory, the study identifies how the perception of AI as a 'shortcut' reduces autonomous motivation and fosters a transactional orientation toward learning. The findings resonate with existing work on [[over-reliance]] and [[cognitive-offloading]], suggesting that easy access to AI-generated code may erode the very persistence that produces deep learning. ## The motivational paradox Introductory programming is effort-intensive: learning to code requires sustained practice, debugging, and tolerance for failure. When an AI tool can complete an assignment in seconds, the perceived *value* of that effort collapses. Drawing on [[self-determination-theory]], the authors argue this shifts students from **autonomous motivation** (learning for its own sake, driven by interest and mastery) toward a **transactional orientation** in which work is completed for the grade, not the skill. The availability of an effortless shortcut doesn't just make cheating easier — it makes the *honest* path feel pointless. ## How AI availability shapes motivation - **Reduced autonomous motivation:** the perception of AI as a shortcut lowers the intrinsic value of developing skills, because the goal (producing working code) can be reached without the effort that builds competence. - **Transactional learning orientation:** students come to treat assignments as tasks to dispatch rather than opportunities to learn, optimizing for output over understanding. - **Persistence erosion:** because [[over-reliance]] and easy access reduce the need for struggle, students miss the productive-failure cycles that drive deep learning in programming. - **Equity concern:** students who consciously resist AI assistance may fall behind peers who use it to complete work faster, complicating assessment fairness and the fairness of grading in [[higher-ed]] — a dynamic with direct implications for [[academic-integrity]]. ## Implications for course design The study argues that instructors cannot assume motivation will survive easy AI access on its own. Two levers are central: 1. **AI-resistant assessment:** redesign assessments so they measure what AI cannot trivially supply — reasoning, debugging, explaining one's own code — aligning with [[authentic-assessment]] and [[assessment-validity]] principles. 2. **Motivation restructuring:** frame learning goals that AI cannot fulfill (deep understanding, the ability to debug and defend one's code under pressure) rather than treating output quality as the sole objective. This connects to the broader [[ai-misuse-learning-harm]] finding that the cost of bypassing effort is reduced durable learning. ## Connection to the broader wiki This paper sits at the intersection of [[over-reliance]], [[motivation]], and [[self-regulated-learning]]. Its emphasis on why *effort feels optional* links to [[ai-availability-student-motivation]]-adjacent work on the cognitive costs of AI, to [[agentic-literacy-debt]] (the cumulative skill deficit from habitual reliance), and to the socio-emotional side documented in [[shame-guilt-ai-regulation-computing-education]]. For instructors, the motivational lens complements the tool-design and assessment-design interventions in [[reducing-ai-misuse]]: students are more likely to use AI productively when they have reasons — not just rules — to do the work themselves. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[self-determination-theory]] - [[motivation]] - [[self-regulated-learning]] - [[academic-integrity]] - [[student-experience]] - [[higher-ed]] - [[ai-misuse-learning-harm]] - [[reducing-ai-misuse]] ## Connected Articles - [[aied-unfinished-mission-bypass]] — AIED's Unfinished Mission: Agency and Motivation - [[ai-making-us-stupid]] — Is AI Making Us Stupid? - [[cognitive-offloading-speedup-illusion]] — Cognitive Offloading and the Speedup Illusion - [[efficiency-gain-illusion-ai-overreliance]] — The Efficiency-Gain Illusion - [[genai-reliance-types-scale]] — GenAI Reliance Types and Scale - [[shame-guilt-ai-regulation-computing-education]] — Shame and Guilt as Social Regulators of AI Use - [[agentic-literacy-debt]] — Agentic Literacy Debt - [[rethinking-scaffolding-llm-tutors]] — Rethinking Scaffolding in LLM Tutors - [[ai-engineering-education-balancing-act]] — AI in Engineering Education: A Balancing Act - [[post-instrumental-learning-capacity-dissolution]] — Post-Instrumental Learning and Capacity Dissolution ## Citation Keith Tran, Colton Harper, Thomas Price (2026). ["Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation in Introductory Programming](https://arxiv.org/abs/2606.30480). arXiv:2606.30480. cs.CY. --- ## [AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention](https://edtechdev.github.io/aied/articles/ai-campus-wellbeing-tools/) # AI-Driven Campus Well-being Tools ## Synthesis This dissertation presents an integrated AI framework for campus well-being spanning prevention (improving feedback collection) and intervention (advancing mental health detection). It represents an important application of [[llm]] and [[generative-ai]] technologies to [[student-experience]] that extends beyond academic learning to holistic student support in [[higher-ed]]. On the prevention side, **TigerGPT** — a personalized survey chatbot grounded in conversational design and engagement theory — achieved 75% usability and 81% satisfaction, demonstrating that AI-mediated feedback collection can improve on traditional survey instruments. **AURA**, an adaptive follow-up question framework using reinforcement learning, dynamically selects question types (validate, specify, reflect, probe) to deepen responses, achieving a +0.12 mean quality gain (p=0.044, d=0.66). On the intervention side, **PsychoGPT** provides explainable mental health assessment built on DSM-5 and PHQ-8 guidelines — a [[affective-computing]] application that prioritizes clinical grounding and interpretability over black-box classification. The **Stacked Multi-Model Reasoning (SMMR)** architecture reduces hallucination risk by layering expert models: early layers handle localized subtasks while later layers reconcile findings, outperforming single-model solutions on the DAIC-WOZ benchmark. The integrated framework — where adaptive survey insights flow into specialized mental health detection models — represents a novel [[edtech-platform]] architecture for campus well-being. For the [[administrator]] perspective, this work provides a concrete roadmap for universities seeking to deploy AI tools that monitor student satisfaction and detect mental health risks — areas where many institutions currently lack effective methods. ## Connected Concepts - [[llm]] - [[generative-ai]] - [[student-experience]] - [[higher-ed]] - [[affective-computing]] - [[edtech-platform]] - [[administrator]] ## Connected Articles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion ## Citation Tang, J. (2026). [New AI-Driven Tools for Enhancing Campus Well-being: A Prevention and Intervention Approach](https://arxiv.org/abs/2605.10804) [PhD Dissertation, University of Missouri]. arXiv:2605.10804. --- ## [How AI Is Changing Teaching Workflows](https://edtechdev.github.io/aied/articles/ai-changing-teaching-workflows/) 📄 [Full article](https://edtechinsiders.substack.com/p/how-ai-is-changing-teaching-workflows) ## Core Thesis AI saves teachers roughly 30% of lesson preparation time with no measurable quality loss — but whether that *reduces burnout* depends entirely on where the freed-up time goes. The key mechanism is **reallocation, not reduction**: teachers redirect saved hours toward higher-value instructional activities rather than simply pocketing time. This article synthesizes evidence from multiple controlled trials, large-scale conversation analysis, and qualitative teacher studies to map the current state of AI in teaching workflows. ## The Evidence Base ### EEF Randomized Trial (England) A controlled trial across 68 schools and 259 science teachers found ChatGPT-using teachers spent **69% of the control group's time** on lesson preparation (~25 minutes saved per week). A blind expert panel detected no difference in pedagogical quality of the materials produced. Teachers redirected the saved time toward other planning, grading, and student-facing activities — a pattern of [[teacher-role]] transformation rather than simple efficiency gain. ### 13,071-Conversation Analysis The most comprehensive dataset on K-12 AI use — 104,000+ messages from 15,000+ educators — revealed that the average teacher prompt touches **1.7 categories simultaneously** (lesson plan + differentiation + formative assessment in one request). AI proactively surfaced instructional elements teachers hadn't requested, suggesting [[generative-ai]] is shifting from reactive tool to proactive pedagogical partner. This connects to research on [[ai-tpack-teacher-multi-agent-workflow]] and the evolving [[teacher-role]]. ### Qualitative Study of 22 K-12 Teachers The dominant driver for AI adoption was **survival, not efficiency**. Teachers framed GenAI as a sustainability measure in a profession already in crisis. One described 80-hour work weeks; another said AI "decreased their stress dramatically." This reframes the value proposition: the conversation about AI in teaching isn't about going from good to great, but from **unsustainable to functional**. This validates the urgency behind [[faculty-development]] and [[teacher-role]] research. ## Where Quality Holds — and Where It Doesn't **AI strengths:** - **Lesson conclusions** — exit tickets, cool-downs, reflective summaries — AI-generated versions were preferred **59.7% of the time** over human designs, the only component where AI consistently beat professional curriculum designers - **High school content** — fine-tuned models outperformed human designers **59.2% of the time**; the more structured the content, the better AI performed - **Teaching outside expertise** — teachers less confident in subject knowledge experienced greater time savings, connecting to [[ai-literacy]] and [[faculty-development]] needs **AI weaknesses:** - **Elementary level** — human-designed plans preferred ~65% of the time for developmental appropriateness and engagement - **Multilingual/SPED support** — AI materials are "neutral" but not targeted, lacking the nuanced [[scaffolding]] human designers build in ## The Reallocation Effect — Brazil Essay Grading RCT A large-scale experiment across **178 schools, ~19,000 high school seniors** tested AI-automated essay feedback. Key results: - Both AI groups produced identical improvements on Brazil's national exam — human graders at ~$0.85/essay added **zero incremental learning benefit** - Students in AI classrooms had **~35% more one-on-one conversations** with teachers about writing and wrote **30% more essays** - Teacher at-home work hours dropped **20%**; those reporting time as "very insufficient" fell from 23% to 9% **The most important finding:** The largest learning gains were on the most complex, highest-order writing task — precisely what AI is *least* equipped to evaluate. AI freed teachers to do what only they can do. This directly supports the [[feedback-loop]] and [[formative-assessment]] literature, extending it with causal evidence from a large-scale [[rct]]. **Caveat:** The bottom quartile showed no improvement — freed-up teacher time alone wasn't sufficient. This connects to [[equity]] concerns about differential benefits from AI integration. ## Three Risks ### 1. The Prompting Gap Almost no teachers used follow-up prompts to iteratively refine AI output — they took the first result and edited manually. Prompt quality directly determined output quality. The teachers who need AI most (early career, under-resourced, outside expertise) are often least equipped to prompt effectively. This makes [[ai-literacy]] professional development a prerequisite, not a nice-to-have. ### 2. The Assessment Trap Nearly half of educator-AI conversations involved assessment tasks, but some teachers requested student work evaluation without specifying rubrics or criteria. AI assessments applied without human oversight risk inconsistency and bias — a [[bias-mitigation]] concern directly relevant to [[automated-grading]] systems. ### 3. Equity Divides - **Student level:** AI materials lack targeted supports for multilingual learners and students with disabilities — a 30% time reduction is net negative if it comes at the expense of vulnerable learners - **Teacher level:** Under-resourced teachers may simply use AI to keep pace rather than upgrade practice, widening the gap between well-supported and under-supported schools — a [[equity]] within the teaching profession itself ## What's Next: Agentic AI The shift from single-prompt chatbots to agentic AI systems represents the next evolution. A multi-agent scoring system — separate agents for content, grammar, and coherence, with a lead synthesizer — outperformed standalone GPT-4o by **8.4% accuracy and 13% consistency**. The teacher's role shifts from prompter to orchestrator, connecting to [[agentic-workflows-education]] and [[human-in-the-loop-ai]] design patterns. ## Connected Concepts - [[ai-literacy]] - [[automated-grading]] - [[bias-mitigation]] - [[formative-assessment]] - [[generative-ai]] - [[persistent-ai-agents-academic-research]] - [[scaffolding]] - [[teacher-role]] - [[higher-ed]] - [[k-12]] - [[student-experience]] ## Connected Articles - [[agentic-workflows-education]] — Agentic Workflows in Education - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-tpack-teacher-multi-agent-workflow]] — Modeling AI-TPACK in Practice: Insights from Teachers' Multi-Agent Workflow Design - [[genai-runaway-object-math-higher-ed]] — GenAI as a runaway object in higher education: A socio-cultural view on AI-influenced academic practice in mathematics - [[test-driven-ai-assisted-learning]] — Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform ## Citation Ler, L. (2026). [*How AI Is Changing Teaching Workflows.*](https://edtechinsiders.substack.com/p/how-ai-is-changing-teaching-workflows) Edtech Insiders --- ## [AI chatbot design principles to enhance the collective efficacy in collaborative learning](https://edtechdev.github.io/aied/articles/ai-chatbot-collective-efficacy-collaborative-learning/) > **Synthesis:** AI chatbot design principles to enhance the collective efficacy in collaborative learning ## Key Findings - A design and development research study (Method Type 2, Richey & Klein 2014) producing a validated framework for chatbots that enhance **collective efficacy** — the shared belief among team members in their collective ability to accomplish tasks (Bandura, 2000) — in collaborative learning: **4 design elements, 10 design principles, and 46 actionable sub-guidelines**. - The final four design elements are **Group Cohesion Support, Affective Cohesion Support, Support for Collaborative Learning Activities, and Inducing Conversation**, spanning principles of belongingness formation, establishing interdependence, creating a positive atmosphere, empathy formation, promoting sharing, enhancing collaborative problem-solving, supporting social regulation, providing immediate scaffolding, familiarity, and personification. - The principles were derived from a Scopus literature review (social sciences, English; terms including "chatbot," "design," "collective efficacy," "collaborative learning tools") that collected 929 papers, narrowed to 116, and analyzed 73 for chatbot design elements, principles, and guidelines. - Validation proceeded through **three rounds of expert review with five experts** (Ph.D.-level, 10–13 years of experience in collaborative learning and AI chatbots): round one showed low validity (M = 2.80, SD = 0.84; content validity index 0.60; inter-rater agreement 0.00), while rounds two and three reached near-perfect scores — round three scored 4/4 on all domains except explanatory power (M = 3.80, SD = 0.45), with CVI and inter-rater agreement of 1.00. - A **usability test with eight participants** (three AI chatbot designers and five instructors who design collaborative learning sessions, ~30-minute interviews each) confirmed strengths — learners addressing challenges independently, chatbot-mediated team facilitation, and precise guidance preventing unproductive time — while identifying limited feedback for non-participating learners as a weakness. - The framework is designed to be usable by **educators without technical expertise**, supporting scalable, accessible implementation in real classrooms. ## Study Design & Method The study used Design and Development Research Method Type 2, proceeding through two stages: design principle development and validation. In the development stage, initial design principles were derived from a systematic literature review (929 Scopus papers collected; 116 selected after excluding technically focused papers; 73 analyzed after excluding non-educational-chatbot or non-collaborative-learning contexts), from which design elements, principles, and detailed guidelines were extracted and consolidated. The initial framework comprised **three design elements (learning support, emotional support, rapport building), ten principles, and 54 sub-guidelines**. In the validation stage, internal validity was tested through three rounds of expert validation using surveys adapted from Nail and Jung (2001) — rating validity, explanatory power, usefulness, and generalizability on a 4-point Likert scale — plus ~30-minute semi-structured interviews; external validity was tested through the usability test with designers and instructors, analyzed thematically and folded back into the final principles. ## Key Results - **Iterative refinement:** round one of expert validation forced reorganization of principles under design elements and consolidation of redundant sub-guidelines (e.g., Sense of Belonging merged into Interdependence); round two split and introduced principles (Sense of Belonging Formation, Social Regulation Support) and separated compound sub-guidelines; round three restructured the design elements — splitting collective-efficacy formation into group cohesion and collaborative problem-solving support and reclassifying immediate feedback under the dialogue-inducing element — yielding the final 4-element, 10-principle, 46-sub-guideline framework. - **Validation trajectory:** validity rose from M = 2.80 (SD = 0.84) with CVI 0.60 and inter-rater agreement 0.00 in round one to 3.80 with CVI and inter-rater agreement of 1.00 in round two, and to 4/4 across domains (explanatory power M = 3.80, SD = 0.45) with perfect CVI and inter-rater agreement in round three. - **Usability findings:** the chatbot design helped learners solve problems independently, mediated within-team discussion, and offered precise guidance that prevented unproductive time use; suggested improvements included personalized feedback for inactive learners. - **Interdisciplinary perspectives:** educators emphasized pedagogical soundness and real classroom dynamics while developers and educational technologists emphasized usability, feasibility, and real-world implementation, jointly shaping the final principles. ## Implications for AI in Education The study directly addresses a structural problem of [[collaborative-learning]] in [[higher-ed]]: a single instructor cannot monitor multiple teams and deliver real-time, team-specific feedback, whereas an AI chatbot can track conversation logs, analyze discussions, and deliver tailored feedback continuously. The resulting design framework — including **immediate scaffolding**, social regulation support, and empathy formation — gives educators a concrete, theory-grounded blueprint for building or specifying collaborative-learning chatbots without coding expertise. It also reinforces that affective design (belongingness, positive atmosphere, empathy, personification) matters as much as cognitive support for team functioning, and it flags ethical obligations: informed consent for user-log data, mitigation of algorithmic bias and culturally or emotionally inappropriate responses, and institutional data-governance policies for AI-assisted learning environments. ## Limitations - The design principles have not yet been implemented in and evaluated against real educational settings; the authors call for studies that build chatbots from the principles and test their effect on collaborative learning outcomes to assess generalizability. - The principles are general rather than context-specific; project-based, maker, and discussion-based learning each have distinct characteristics that may require optimized or additional design principles. - The usability test used a small, relatively homogeneous sample of eight participants, and expert validation involved a small expert group with limited diversity of perspectives, potentially biasing the results; broader stakeholder samples and iterative testing cycles are recommended. ## Connected Concepts - [[collaborative-learning]] - [[higher-ed]] - [[math-education]] - [[prompt-engineering]] - [[formative-assessment]] - [[affective-tutoring]] - [[curriculum-design]] - [[administrator]] ## Connected Articles - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education - [[ccct-cooperative-learning-technique]] — Artificial intelligence assisted design of a novel cooperative learning technique for higher education - [[ai-collaborative-learning-skills-impacts]] — Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving - [[llm-critical-thinking-teamwork-review]] — Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review - [[ai-collaborative-learning-systematic-review]] — A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Kim, M., & Lim, C. (2025). [*AI chatbot design principles to enhance the collective efficacy in collaborative learning*](https://doi.org/10.1007/s12564-025-10080-z). --- ## [AI Coaching for Accelerating Human Skill Development with Reinforcement Learning](https://edtechdev.github.io/aied/articles/ai-coaching-rl-skill-development/) This paper explores how an embodied AI agent can act as a [[scaffolding|coach]] that accelerates human motor-skill development using [[adaptive-learning|reinforcement learning]]. The authors argue that effective coaching requires dynamically balancing guidance with learner autonomy — too much assistance leads to [[over-reliance]] and skill atrophy, while too little leaves learners struggling. **Key findings:** - An RL-based coaching policy that adapts its level of intervention to the learner's current skill level significantly accelerates skill acquisition compared to static assistance levels. - The AI coach that gradually fades scaffolding (consistent with [[scaffolding]] theory in [[intelligent-tutoring]]) produced the best long-term retention and transfer performance. - Over-reliance emerged when the coach provided excessive intervention, confirming the [[over-reliance]] concern documented in [[generative-ai]] tutoring contexts. **Implications:** - RL-based coaching offers a principled framework for [[personalized-learning|personalized]] skill development in domains beyond traditional academics (e.g., surgical training, [[professional-training]], [[stem-education|STEM lab skills]]). - The competence-based fading policy mirrors established pedagogical best practices, suggesting [[formative-assessment]] signals can drive AI coaching adaptation. - Extends [[ai-tutor-safety-harms|AI safety in tutoring]] research to embodied/motor skill domains. ## Connected Concepts - [[scaffolding]] - [[adaptive-learning]] - [[over-reliance]] - [[intelligent-tutoring]] - [[generative-ai]] - [[personalized-learning]] - [[professional-training]] - [[stem-education]] - [[formative-assessment]] ## Connected Articles - [[ai-tutor-safety-harms]] ## Citation Wang, W., Gu, E., Loquercio, A., Hu, H., & Mangharam, R. (2026). [AI Coaching for Accelerating Human Skill Development with Reinforcement Learning](https://arxiv.org/abs/2606.25337). arXiv:2606.25337. cs.RO. --- ## [Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving](https://edtechdev.github.io/aied/articles/ai-collaborative-learning-skills-impacts/) > **Synthesis:** Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving ## Key Findings - Mixed-methods study of **102 undergraduate students** in interdisciplinary project-based courses, of which **96 responses (94.1%) were valid** for analysis; qualitative input came from **80 open-ended survey responses** and **15 semi-structured interviews**, triangulated with analysis of student project outputs. - **83% of respondents reported that LLMs enhanced creativity** (generating new ideas, offering different perspectives during brainstorming), and **78% said LLMs increased their group's overall productivity** — although **20% felt LLM use decreased individual accountability and contribution**, creating workload imbalances. - **81% noted LLMs acted as neutral mediators** that helped resolve differences and speed consensus building, while **17% indicated the presence of LLMs diminished the need for critical engagement**, with AI suggestions sometimes accepted without discussion. - Correlation analysis showed the strongest relationship between **consensus-building impact and productivity impact (r = 0.71)**, followed by collective intelligence and productivity (0.69); AI comfort correlated with perceived creativity impact (0.56) and problem-solving impact (0.59), and digital proficiency showed only moderate correlations (0.52, 0.47, 0.50). - **No statistically significant differences** in perceived LLM impact were found across groups based on prior AI use, AI comfort, or digital proficiency — a t-test (t = 1.414, p = 0.2929) and ANOVA (f = 5.33, p = 0.1028) at α = 0.05 both failed to reject the null hypothesis. - Respondents were predominantly business and management majors (62%), aged 18–26 with an even gender split; **90% were very comfortable with AI tools** and **96% had prior LLM experience**, with digital proficiency rated intermediate (54%) or advanced (40%). ## Study Design & Method The study used a descriptive, mixed-methods design with no variable manipulation. Quantitative data came from a structured Likert-scale survey measuring perceived impacts of LLMs on group collaboration, creativity, and problem-solving efficacy. Qualitative data came from open-ended survey items, audio-recorded semi-structured interviews transcribed for thematic analysis, and inspection of student project outputs for originality and collaborative innovation. Statistical analysis combined a Pearson correlation matrix with t-tests and ANOVA (implemented in Python) to compare perceptions across groups defined by AI comfort, digital proficiency, and prior LLM use. Because the hypothesis tests returned non-significant results, the authors additionally explored alternative factors (specific courses taken, exposure to AI tools, learning styles) and proposed k-means clustering as a way to segment student interaction patterns. ## Key Results - **Perceived benefits:** students credited LLMs with improving problem-solving (e.g., quick summaries of long texts freed groups to focus on higher-order tasks), structuring group discussion, synthesizing individual contributions into cohesive action plans, and boosting collective intelligence. - **Perceived risks:** overreliance on AI to build consensus was flagged as a threat to interpersonal skill development; some group members became passive and deferred to AI-generated solutions; instructors were advised to monitor group dynamics and intervene when workload distribution becomes imbalanced. - **Digital competence does not shape perceptions:** the absence of significant group differences suggests familiarity and digital skill are weak predictors of how students perceive LLM effects on creativity, problem-solving, consensus building, and productivity. - **Balanced integration:** the authors warn that heavy dependence on generative AI can impair self-regulated learning, intrinsic motivation, and performance, citing the phenomenon of "metacognitive laziness" — outsourcing cognitive effort to AI reduces engagement with deeper learning strategies and can erode independent analysis, synthesis, and evaluation over time. ## Implications for AI in Education The study's central message is that LLMs must be **designed and deployed to support rather than disrupt human collaboration**. Positive correlations between AI comfort and perceived creativity/problem-solving gains suggest that building familiarity with AI tools matters more than raw technical skill, so educators should teach strategic use alongside technical proficiency. The documented overreliance risks point to concrete design responses: group projects that require students to reflect on AI outputs and justify decisions independently of the AI's suggestions, clear guidelines on tool use, and instructor monitoring of group dynamics. The authors call for **balanced AI integration** that preserves the essential human elements of group work — meaningful discussion, challenging assumptions, and collective knowledge construction — connecting to [[collaborative-learning]] and [[critical-thinking]] research, and echoing [[self-regulated-learning]] concerns that outsourcing cognitive effort ("metacognitive laziness") undermines deep learning. Findings also support [[generative-ai]] adoption in [[higher-ed]] settings as a complement to, not a replacement for, human effort in teamwork. ## Connected Concepts - [[higher-ed]] - [[collaborative-learning]] - [[critical-thinking]] - [[self-regulated-learning]] - [[generative-ai]] - [[socratic-method]] - [[math-education]] - [[creativity]] ## Connected Articles - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education - [[llm-critical-thinking-teamwork-review]] — Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review - [[ccct-cooperative-learning-technique]] — Artificial intelligence assisted design of a novel cooperative learning technique for higher education - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[ai-collaborative-learning-systematic-review]] — A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade ## Citation Agnaou, A., & El Asri, H. (2025). [*Artificial intelligence and collaborative learning: Impacts on creativity, critical thinking, and problem-solving*](https://doi.org/10.5590/JERAP.2025.15.2120). --- ## [A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade](https://edtechdev.github.io/aied/articles/ai-collaborative-learning-systematic-review/) > **Synthesis:** A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade ## Key Findings - A PRISMA 2020 systematic review covering **27 studies** of AI-powered collaborative learning in higher education, drawn from a Scopus search (run July 16, 2024, query: TITLE-ABS-KEY "artificial intelligence" AND collaborative AND learning AND higher AND education) that identified 163 records → 145 screened after excluding 18 conference reviews → 59 retrieved → 27 included after eligibility assessment. - The review organizes AI tools into four functional categories: **predictive analytics/early warning systems**, **language-based systems** (chatbots, large language models, natural language processing), **recommender algorithms**, and **intelligent tutoring and monitoring applications** — each contributing differently to collaborative learning dynamics. - Evidence indicates AI-powered **predictive analytics and multimodal approaches (including emotional and physiological monitoring) enhance student engagement and motivation**, enabling early identification of at-risk students and timely, data-driven interventions (e.g., combining AI with learning analytics for continuous feedback improved collaborative learning performance and satisfaction). - **Personalized learning systems and recommender algorithms** were found to underpin effective collaborative environments, while **good task design** (balancing challenge and accessibility, embedding collaboration requirements) and **emotional engagement and social presence** emerged as critical success factors. - The annual distribution of included articles shows a **marked increase in publications over the decade**, reflecting growing research interest in AI-enhanced collaborative learning in higher education. - Key challenges identified include **algorithmic bias, over-reliance on technology, and teacher training needs**, alongside ethical concerns about **transparency, data protection, and balancing full automation with human touch**; the review notes AI's role in collaborative learning remains understudied relative to individual learning contexts. ## About the Review The review synthesizes a decade (2014–2024) of research on AI-enhanced collaborative learning in higher education. Inclusion criteria: articles published 2014–2024, written in English, explicitly focused on AI-based collaborative learning in higher education, with eligible designs limited to empirical studies, case studies, and meta-analyses, and populations of higher education students or instructors. Six research questions structured the synthesis, covering the impact of AI-powered multimodal approaches on engagement and motivation; the influence of task design on AI-powered collaborative environments; the role of affective factors and social presence; the effectiveness of remote learning and virtual laboratories; innovative teaching methods enabled by AI; and future directions. Studies were coded and tabulated by focus area, methodology, application, main findings, and AI methods, and grouped by intervention type, outcome measures, and population characteristics. ## Key Results - **Academic performance:** predictive analytics and machine learning support early identification of at-risk students and continuous feedback loops that improve collaborative performance and satisfaction; AI-driven feedback systems appear across the reviewed studies as a core intervention type. - **Engagement and motivation:** multimodal AI approaches that track emotional and physiological responses (e.g., heart rate, facial expressions, electrodermal activity) give instructors real-time insight into engagement, supporting timely interventions to maintain motivation. - **Task design and assessment:** effective tasks balance challenge and accessibility; AI can dynamically adjust task difficulty based on group performance, monitor participation, and enforce equitable contribution so dominant members cannot take over and reserved students are not left behind. - **Social presence and emotion:** AI tools that increase social presence and emotional engagement make learners feel more connected to the learning process, improving collaborative outcomes. - **Remote and virtual learning:** AI-powered virtual laboratories simulate physical lab environments, allowing distance students to collaborate on complex projects with engagement comparable to on-campus settings. - **Innovations and future directions:** integration with blended/flipped classrooms, learning analytics, and emerging immersive technologies (metaverse, augmented and virtual reality) are highlighted, alongside calls to investigate underrepresented approaches such as symbolic AI and hybrid systems that merge reasoning-based methods. ## Implications for AI in Education For educators and institutions, the review frames AI not as a replacement for collaboration but as a layer that personalizes feedback, monitors group dynamics, and supports equitable participation in real time. Practical takeaways: invest in **task design** (challenge calibrated to group ability, collaboration requirements embedded in the task structure), use **predictive and multimodal analytics** to detect disengagement early, and attend to **social presence and emotional engagement** rather than only cognitive outcomes. The identified challenges — bias, over-reliance, teacher training, transparency, and data protection — imply that AI integration in [[collaborative-learning]] needs institutional guardrails and educator upskilling, consistent with broader concerns about [[generative-ai]] deployment in [[higher-ed]]. ## Limitations - The review's scope is constrained by its stated eligibility criteria: a single database (Scopus), English-language articles only, and a 2014–2024 publication window, which may exclude relevant non-English or pre-2014 work and non-indexed venues. - The synthesis draws on heterogeneous study designs and outcome measures, which limits direct comparability of effect sizes across interventions. - The review itself notes the field's open research gaps — AI's role in collaborative (versus individual) learning is understudied, and future work should address group cohesion, equal participation, and effective collective collaboration. ## Connected Concepts - [[higher-ed]] - [[collaborative-learning]] - [[human-in-the-loop-ai]] - [[administrator]] - [[affective-computing]] - [[generative-ai]] - [[math-education]] - [[prompt-engineering]] ## Connected Articles - [[llm-critical-thinking-teamwork-review]] — Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education - [[ai-collaborative-learning-skills-impacts]] — Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving - [[ccct-cooperative-learning-technique]] — Artificial intelligence assisted design of a novel cooperative learning technique for higher education - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) ## Citation Kovari, A. (2025). [*A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade*](https://doi.org/10.1016/j.ssaho.2025.101335). --- ## [What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries](https://edtechdev.github.io/aied/articles/ai-education-global-capacity/) > A global perspective on AI in education readiness, framed around the insight that the real bottleneck is human and institutional capacity, not technical access. Based on a WEF (2026) synthesis of youth leader initiatives across the United States, Kenya, China, UAE, and Switzerland. # AI Education Global Capacity ## Definition A global perspective on AI in education readiness, framed around the insight that the real bottleneck is human and institutional capacity, not technical access. Based on a WEF (2026) synthesis of youth leader initiatives across the United States, Kenya, China, UAE, and Switzerland. ## Key Finding: Human Readiness is the Bottleneck While AI tools are globally available, the critical variable is local absorption capacity — infrastructure, culture, public trust, teacher training, and policy alignment. Teachers are identified as the critical bridge between AI systems and real learning. Without adequate time, training, and support, even the best tools fail. ## Country-Specific Patterns - **Kenya:** Teacher shortages and uneven infrastructure make readiness structural; youth-led programs reached 300+ girls across marginalized communities - **United States:** Rapid experimentation but weak implementation support for teachers - **China:** Scale pressure; assessment systems still reward memorization over capacities AI now requires — a fundamental misalignment - **UAE:** Success depends on genuine inclusion of teachers, institutions, and communities in design - **Switzerland:** Privacy, quality, and system reliability as non-negotiables ## Connections to Wiki - Extends [[stanford-evidence-base-ai-k12-2026]] with global implementation perspective beyond US/Stanford scope - Addresses [[equity-in-ai-education]] disparities across countries - The assessment misalignment in China echoes concerns in [[authentic-assessment]] - Teacher readiness theme connects to [[faculty-development]] and [[teacher-ai-competency]] ## Connected Concepts - [[ai-literacy]] - [[equity-in-ai-education]] - [[faculty-development]] - [[regulation]] - [[teacher-ai-competency]] - [[k-12]] - [[teacher-role]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[authentic-assessment]] — Authentic Assessment - [[institutional-change-framework-ai]] — A Framework for Institutional Change in the Age of AI - [[post-covid-ict-career-aspirations]] — Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills? - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-generated-slides-student-perception]] — AI-Generated Slides: Are They Good? Can Students Tell? - [[ai-higher-ed-bridge-gap]] — Higher Education Must Bridge the AI Gap ## Citation Forum, S.W.E. (2026). [What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries](https://www.weforum.org/stories/2026/04/what-ai-in-education-needs-next-lessons-from-youth-leaders-across-five-countries/) --- ## [AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems](https://edtechdev.github.io/aied/articles/ai-enabled-serious-games/) Serious games are widely used for learning and training across domains such as healthcare, defense, and education. This chapter examines how contemporary AI approaches may support real-time instructional adaptation in serious games. This book chapter provides a comprehensive survey of AI integration in serious games for learning and training, distinguishing between instructional intelligence (inferring learner knowledge and reasoning about pedagogically appropriate responses) and adaptivity (modifying instructional actions during interaction). The authors trace the historical evolution from early computer-assisted instruction through Intelligent Tutoring Systems, dynamic difficulty adjustment, authoring platforms, and learning analytics to contemporary AI-enabled architectures. Three AI technologies are identified as having high potential: large language models (LLMs), reinforcement learning (RL), and agent-based architectures. The chapter also highlights critical challenges including explainability, validation, computational cost, and the limited empirical evidence regarding long-term learning outcomes in AI-enabled serious games. This survey connects to [[intelligent-tutoring]] research by framing serious games as an application domain for ITS principles, complementing the multimodal feedback architectures demonstrated in [[multimodal-affective-its-presentation]]. The discussion of LLM and agent-based integration aligns with [[agentic-ai]] perspectives on multi-agent AI frameworks, while the emphasis on empirical validation gaps resonates with calls for rigorous efficacy-study research across AIED. The focus on training transfer extends [[adaptive-learning]] into professional and defense applications beyond K-12 and higher education. ## Connected Concepts - [[intelligent-tutoring]] - [[agentic-ai]] - [[adaptive-learning]] ## Connected Articles - [[multimodal-affective-its-presentation]] ## Citation Priyamvada Tripathi, Bill Kapralos (2026). [AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems](https://arxiv.org/abs/2605.21962). arXiv:2605.21962. Springer book chapter, Advances in Global Applied Artificial Intelligence. --- ## [Using AI in engineering education: a balancing act, driven by clear purpose](https://edtechdev.github.io/aied/articles/ai-engineering-education-balancing-act/) Based on a questionnaire of 100 higher-education engineering students and a critical literature review, examines how students use and perceive LLMs. Students value LLMs for writing support, conceptual clarification, coding assistance, and brainstorming, but express concerns about inaccuracies, bias, overreliance, and academic integrity. Analyzes two dominant metaphors — LLM as 'oracle' and 'tutor' — showing they cultivate expectations exceeding actual capabilities. Argues students' attachment to LLM efficiency reflects 'cruel optimism' — benefits depend on skills still being developed. ## Key Findings - In a questionnaire of 100 higher-education students, predominantly from engineering-related fields, LLMs were most valued for writing support, conceptual clarification, coding assistance, and brainstorming. - Students simultaneously expressed concerns about inaccuracies, bias, overreliance, academic integrity, and the burden of verification imposed by AI-generated output. - Two dominant metaphors — the LLM as "oracle" and as "tutor" — cultivate expectations of authority, expertise, and personalized learning that exceed what probabilistic text generators can actually deliver. - Students' attachment to the promises of efficiency and personalized support reflects a form of "cruel optimism": the perceived benefits depend on the very skills, vigilance, and expertise that students are still developing. - Student respondents who offered suggestions (N=27) called for assessment redesign — in-person essays, grading shifted from memorization to analysis, oral exams, and presentations with personalized questions — pointing beyond the reproducibility of knowledge. - The chapter argues for a purpose-driven, context-sensitive approach to AI integration in engineering education, emphasizing critical AI literacy, reflective assessment design, pedagogical caution, and consideration of broader ethical and environmental impacts. ## Assessment Design The chapter treats assessment as a key lever, reviewing frameworks such as the AAA (Against, Avoid, Adopt) approach, which keeps lower-order assessment tasks supervised when AI can complete them (e.g., recalling facts, explaining concepts) and relies on contextualization in current affairs, personal experiences, and in-class engagement for higher-order skills. These recommendations align with the students' own suggestions for closed, supervised, or oral formats, and signal a shift in the instructor's role from policing AI use toward reflectively adapting assessment for AI's possible (mis)use. ## Implications for AI in Education The "cruel optimism" argument has direct consequences for pedagogy: the efficiency gains of AI are only realized when learners already possess the [[critical-thinking]] and domain expertise needed to judge AI output — precisely the skills that education is supposed to build. This makes [[ai-literacy]] and verification capacity prerequisites rather than optional extras, and cautions against framing AI tools as substitutes for expertise. For engineering education specifically, the balancing act lies in leveraging LLM support for writing, coding, and brainstorming while designing assessments and curricula that keep the burden of verification and the development of judgment inside the learning process, guarding against [[over-reliance]]. ## Connected Concepts - [[ai-literacy]] - [[administrator]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[agentic-ai]] - [[k-12-ai-education]] - [[ai-tutoring]] - [[affective-tutoring]] ## Connected Articles - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[ai-team-teaching-talk-analytics]] — AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI ## Citation Olya Kudina (2026). [Using AI in engineering education: a balancing act, driven by clear purpose](https://arxiv.org/abs/2606.16626). arXiv:2606.16626. The Routledge Handbook of the Philosophy of Engineering, 2nd ed.. --- ## [A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data](https://edtechdev.github.io/aied/articles/ai-ethics-education-public-discourse/) **Akriti Bagale, Nafisa Mehjabin, Ali Unlu, Aditya Johri, et al. (2026)** - George Mason University; University of Virginia. arXiv preprint. ## Key Findings - Longitudinal Twitter/X analysis maps how the public debates AI ethics concerns in [[higher-ed]] and schools over time. - Surfaces the ethical concerns (bias, fairness, accountability) that educators and [[educational-policy-ai]]s must address for responsible adoption. - Ties public sentiment to [[equity]] questions about who benefits and who is harmed by GenAI in education. - Complements [[principled-ai-education]] frameworks by grounding them in real discourse rather than expert opinion alone. - Relevant to [[academic-integrity]] debates, as public concern shapes institutional response and policy. - Informs [[ai-literacy]] efforts: public discourse reveals the misconceptions needing pedagogical attention. ## Connected Concepts - [[higher-ed]] - [[educational-policy-ai]] - [[equity]] - [[academic-integrity]] - [[ai-literacy]] ## Connected Articles - [[principled-ai-education]] ## Citation Bagale, A., Mehjabin, N., Unlu, A., Johri, A., et al. (2026). [A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data](https://arxiv.org/abs/2607.12295). arXiv:2607.12295. --- ## [Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System](https://edtechdev.github.io/aied/articles/ai-fallibility-warning-help-seeking/) > **Synthesis:** Recent work in Technology-Enhanced Learning and HumanComputer Interaction highlights the importance of transparency and trust calibration in AI-supported learning environments as they pose a risk of hallucinations. In this study, we investigate whether a simple transparency intervention that warns s ## Abstract Recent work in Technology-Enhanced Learning and HumanComputer Interaction highlights the importance of transparency and trust calibration in AI-supported learning environments as they pose a risk of hallucinations. In this study, we investigate whether a simple transparency intervention that warns students that a pedagogical agent may make mistakes affects learner behavior in a math intelligent tutoring system. We conducted a classroom experiment with 252 school students using two system versions: one including a warning message about potential system errors, and one that does not mention potential errors. Using log data, we analyzed students’ problem-solving performance data, including help-seeking behavior, error rate, and time-on-task. Results show that students who were warned about po ## Connected Concepts - [[help-seeking]] - [[affective-tutoring]] - [[knowledge-tracing]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[pedagogical-llm-training]] ## Connected Articles - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators - [[rethinking-scaffolding-llm-tutors]] — Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments - [[prompt-injection-defenses-educational-llm-tutors]] — Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs - [[favero-critical-ai-tutors-empower-enslave-2025]] — Critical AI Tutors: Empower or Enslave? - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use ## Citation Nagashima, T., Hladký, M., & Rief, V. (2026). [*Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System*](https://arxiv.org/abs/2606.03822). arXiv:2606.03822. --- ## [Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory](https://edtechdev.github.io/aied/articles/ai-fatigue-academic-contexts/) This grounded theory study analyzed open-ended responses from 1,054 university students across three Philippine universities to define **AI fatigue** as a distinct construct — separate from technostress and digital fatigue. The analysis identified **five dimensions**, each with two indicators grounded in participant accounts: **(1) Cognitive Overload** — mental exhaustion from processing AI outputs and prompts; **(2) Motivational Disengagement** — loss of drive to engage when AI can complete tasks; **(3) Moral Unease** — ethical discomfort from AI dependency and plagiarism concerns; **(4) Physical Strain** — bodily fatigue from prolonged AI interaction; and **(5) Attentional Drift** — difficulty maintaining focus amid AI-mediated multitasking. The resulting **AI Fatigue Model** is a stage-based framework showing how these dimensions accumulate and reinforce each other across repeated AI interactions. This construct has significant implications for [[over-reliance]] research: AI fatigue may be the endpoint of sustained [[cognitive-offloading]] and dependency, where the efficiency-gain illusion gives way to genuine cognitive depletion. For [[ai-literacy]] interventions, the model suggests that teaching technical AI skills without addressing the affective and motivational costs is incomplete — students need strategies for managing AI fatigue, not just using AI effectively. The Motivational Disengagement dimension directly threatens [[self-regulated-learning]] capacity, as students lose intrinsic drive when AI stands ready to complete tasks. The work also contributes to [[affective-computing]] by formalizing the negative affective dimension of sustained AI interaction. As the first conceptual model of AI fatigue in academic contexts, this provides a foundation for instrument development and intervention design, with direct relevance to [[student-experience]] in AI-mediated learning environments. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[ai-literacy]] - [[self-regulated-learning]] - [[affective-computing]] - [[student-experience]] ## Connected Articles - [[learning-by-chatting-genai-impact]] - [[bloom-aligned-educational-control-llms]] - [[ai-learning-assistants-higher-ed-large-scale]] - [[youtube-frames-chatgpt-education]] - [[buggy-genai-code-student-responses]] - [[ai-making-us-stupid]] ## Citation John Paul P. Miranda, Emmanuel B. Parreno, Jovita G. Rivera (2026). [Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory](https://arxiv.org/abs/2605.23123). arXiv:2605.23123. International Journal of Learning, Teaching and Educational Research, 25(5), 91-107 (2026). - [[digital-literacy-illusion]] — Overconfident students may paradoxically disengage from AI learning - [[ai-productivity-moderation]] — Incentive structures moderate whether AI adoption leads to fatigue or growth --- ## [Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency](https://edtechdev.github.io/aied/articles/ai-feedback-critical-thinking-writing-2026/) > **Synthesis:** This study developed the Writing Improvement and Smart Evaluation Agent (WISE Agent), an AI feedback tool targeting textual logic and perspective biases in student essays. A three-month intervention with 260 Chinese sixth-grade students found structural optimizations in critical thinking dimensions rather than a uniform increase in total scores. Lower-performing students advanced in evidence selection while higher-performing students excelled in perspective diversification, suggesting WISE Agent serves as an effective cognitive scaffold for incremental critical thinking development through personalized feedback loops. ## Key Findings This study developed the Writing Improvement and Smart Evaluation Agent (WISE Agent), an AI feedback tool targeting textual logic and perspective biases in student essays. A three-month intervention with 260 Chinese sixth-grade students found structural optimizations in critical thinking dimensions rather than a uniform increase in total scores. Lower-performing students advanced in evidence selection while higher-performing students excelled in perspective diversification, suggesting WISE Agent serves as an effective cognitive scaffold for incremental critical thinking development through personalized feedback loops. The work contributes to understanding of [[critical-thinking]] in educational contexts, with implications for [[automated-grading]], [[scaffolding]]. ## Connected Concepts - [[critical-thinking]] - [[automated-grading]] - [[scaffolding]] - [[writing-education]] - [[ai-feedback-quality]] - [[k-12]] - [[generative-ai]] ## Connected Articles - [[genai-teacher-feedback-comparison]] - [[repeated-ai-writing-feedback-semester]] - [[foxglove-writing-feedback-experts-llms]] ## Citation Zhu, Q., Zhai, X., Zou, Y., & Gao, C. (2026). [*Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency*](https://arxiv.org/abs/2608.05177). arXiv:2608.05177. --- ## [Making AI-Generated Feedback Matter: From Provision to Student Enactment](https://edtechdev.github.io/aied/articles/ai-feedback-enactment-workflow-2026/) > **Synthesis:** Alsaiari et al. (2026) report a large-scale quasi-experimental cohort study (13,037 students; 51,296 student-authored resources) comparing three AI-mediated feedback workflows. Students in the **Enacted Feedback** condition — prompted to select feedback suggestions, evaluate their relevance, and engage in targeted AI dialogue anchored to those selections — showed significantly higher uptake of AI-generated feedback (26.2% estimated probability) than **Directed Feedback** (14.1%) or **Self-Directed Feedback** (0.1%), along with higher self-assessment confidence and submitted-work quality. The finding positions **student enactment**, not comment quality, as the decisive variable in AI feedback, connecting to [[feedback-loop]], [[self-regulated-learning]], and [[human-ai-collaboration]] research. ## From Provision to Enactment Feedback value depends on two challenges: providing high-quality, timely, individualized feedback at scale, and supporting students to interpret, evaluate, and act on that feedback productively. Generative AI credibly addresses the provision challenge, but students' uptake of AI-generated feedback remains limited without structured support. ## Three Workflows - **Directed Feedback (n=3,723):** students received AI-generated feedback comments without structured support. - **Self-Directed Feedback (n=3,951):** students could initiate optional AI-supported dialogue. - **Enacted Feedback (n=5,363):** students were prompted to select feedback suggestions, evaluate their relevance, and engage in targeted AI-supported dialogue anchored to those selections. ## Key Findings Enacted Feedback was associated with significantly higher uptake (26.2% vs 14.1% Directed vs 0.1% Self-Directed), significantly higher self-assessment confidence, and higher submitted-work quality. The authors conclude that AI access alone is insufficient; **purposeful workflow design** that positions learners as active participants in judgement, dialogue, and improvement is central to productive feedback use. ## Connected Concepts - [[feedback-loop]] - [[self-regulated-learning]] - [[human-ai-collaboration]] - [[learning-analytics]] - [[formative-assessment]] - [[student-experience]] - [[ai-feedback-quality]] - [[scaffolding]] - [[higher-ed]] - [[llm]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] - [[genai-feedback-design-multisite-experiment]] - [[sequenced-ai-feedback-learning]] - [[ai-internal-feedback-evaluative-judgments]] - [[learner-centered-feedback-ai]] - [[repeated-ai-writing-feedback-semester]] - [[feedback-futures-genai]] - [[care-full-feedback-genai]] ## Citation Alsaiari, O., Baghaei, N., Lodge, J. M., Gašević, D., Winstone, N., & Khosravi, H. (2026). [*Making AI-generated feedback matter: From provision to student enactment*](https://arxiv.org/abs/2608.11625). arXiv:2608.11625. --- ## [Artificial intelligence and feedback in university education: effectiveness and student perceptions](https://edtechdev.github.io/aied/articles/ai-generated-feedback-higher-ed/) ## Summary This quasi-experimental study directly compares **AI-generated feedback** (two LLMs: **GPT-o4-mini** and **DeepSeek R1**) with **expert human-teacher feedback** in a project-based university course (Assessment & Learning, third-year Primary Teacher Education, University of Padua). The central question is not "is AI feedback worse?" but *under what pedagogical conditions* AI feedback can be a credible, educationally meaningful component of formative assessment. The answer the authors land on: **feedback effectiveness depends less on its source than on the pedagogical architecture in which it is embedded** — strong assessment literacy and explicit, shared criteria make AI feedback comparable to teacher feedback. ## Method (key parameters) - **Design:** 47 student groups (N = 238; 146 attending, 92 non-attending) randomly assigned to one of three feedback conditions — DeepSeek R1 (16 groups), expert human teacher (16), GPT-o4-mini (15). Unit of analysis = group (4–5 students each) to preserve independence. - **Task:** Two-stage project (PRE then POST), evaluated with a shared analytic rubric (0–30 points) co-constructed with students. - **AI prompt design:** Both LLMs were given all course materials plus an assignment brief, the pedagogical framework, and the co-constructed rubric via a **Retrieval-Augmented Generation (RAG)** setup; instructed to act as a university professor and give objective, justified, actionable formative feedback. The rubric + an exemplar functioned as a "calibration device" that transferred the teacher's evaluative expectations into the AI. - **Analyses:** Wilcoxon signed-rank (PRE–POST), Kruskal–Wallis across sources, robust linear models (HC3) for attendance moderation, and — crucially — **non-inferiority and equivalence tests** (Welch-adjusted 90% CIs, pre-specified margin ±1 point on the 30-point scale), because non-significant differences don't imply practical equivalence. ## Key Findings ### RQ1 — Feedback improves performance regardless of source Across all 47 groups, project performance rose significantly from PRE to POST (Wilcoxon W = 1081, p < 0.001, rank-biserial rrb = 0.77 — a large effect); mean score +3.9 points (23.81 → 27.70), with post-test scores converging near the ceiling (median 28). ### RQ2 — No significant difference between feedback sources Post-feedback scores did **not** differ by source (Kruskal–Wallis H(2) = 1.91, p = 0.384, ε² = 0.042); gain scores likewise non-significant (H(2) = 0.74, p = 0.690). Pairwise Hodges–Lehmann contrasts all had CIs spanning zero. ### RQ3 — Attendance doesn't matter Robust linear model: no main effect of attendance (F(1,41) = 1.52, p = 0.225), no source × attendance interaction (F(2,41) = 0.97, p = 0.389). ### RQ4 — AI feedback is practically comparable to teacher feedback | Comparison (AI − Teacher) | Mean diff | 90% CI | Non-inferior? | Equivalent? | | --- | --- | --- | --- | --- | | **GPT-o4-mini** vs Teacher | +0.23 | [−0.46, 0.91] | **Yes** | **Yes** | | **DeepSeek R1** vs Teacher | +0.56 | [−0.05, 1.18] | **Yes** | No (upper bound exceeds +1) | Same pattern on baseline-adjusted gains (DIFF_ADJ). **GPT-o4-mini met both non-inferiority and full equivalence**; **DeepSeek R1 met non-inferiority** (practically comparable, but with more uncertainty). ### Student perceptions — equally positive across sources Validated 19-item questionnaire (N = 200; scales: perceived mastery α = 0.81, positive emotions α = 0.85, negative emotions α = 0.73). Students were **blind** to feedback source. No significant differences across conditions on any scale: - Perceived mastery: M ≈ 4.14–4.22 (high) - Positive emotions: M ≈ 3.99–4.21 - Negative emotions: M ≈ 1.22–1.39 (low) - Overall satisfaction: ~98% (DeepSeek 97.5%, teacher 94%, GPT-o4-mini 100%) — analysed descriptively due to ceiling. AI-generated feedback was experienced as acceptable and supportive, comparable to teacher feedback. ## Interpretation: Source vs. Architecture The authors' core argument: **feedback works as a systemic, relational process, not a function of who (or what) produces it.** In this study both AI and teacher feedback were anchored to the *same explicit rubric and student co-constructed exemplar*, which made criteria transparent and gave the AI an "interpretative anchor" usually tacit in human grading. It is the **teacher's assessment literacy** — encoded in the rubric and exemplar — that calibrated the AI, not the model alone. Thus generative AI is best seen as a **support for teachers with strong assessment literacy** (scaling timeliness/consistency) rather than an autonomous replacement. The study explicitly warns against over-reliance and unequal access, and calls for maintaining teacher oversight and students' critical engagement. ## Limitations (per authors) - **Ceiling effect:** 91% of groups scored ≥27/30 (SD = 0.95) — limits sensitivity of post-test comparisons; equivalence rests mainly on adjusted-gain analyses. - **Small group-level N = 47** → wide CIs; modest source differences can't be fully ruled out. - **No prior-AI-experience data** collected; single course / discipline (Primary Teacher Education); **student assessment literacy not measured** (treated as a hypothesis, not tested). ## Implications for the wiki - A strong, well-controlled (randomised, blind, equivalence-tested) data point that **AI-generated feedback can match expert teacher feedback** for project-based learning **when criteria are explicit and assessment literacy is high** — complementing [[ai-feedback-quality]] and [[ai-learning-companions-framework]] work. - Pairs naturally with [[generative-ai-guardrails-harm-learning]] (the PNAS RCT): that study shows *unguarded* AI tutoring can *harm* learning, this one shows *well-architected* AI feedback can *match* teachers — together they bracket the design-dependence of AIED outcomes. - Reinforces [[formative-assessment]], [[feedback-loop]], and [[ai-literacy]] (teacher and student) as the decisive variables, over the raw tool. - Connects to [[rag|retrieval-augmented generation]] as a calibration mechanism and to [[over-reliance]] (the authors flag it as a risk even in a positive-result study). ## Connected Concepts - [[ai-literacy]] - [[formative-assessment]] - [[higher-ed]] - [[rag]] - [[scaffolding]] - [[student-experience]] - [[generative-ai]] - [[llm]] ## Connected Articles - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[generative-ai-guardrails-harm-learning]] — Generative AI without guardrails can harm learning: Evidence from high school mathematics - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers' practices and perceptions - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community ## Citation Grion, V., Doria, B., Agostini, D., & Slaviero, G. (2026). Artificial intelligence and feedback in university education: effectiveness and student perceptions. [Assessment & Evaluation in Higher Education](https://www.tandfonline.com/doi/full/10.1080/02602938.2026.2697962). https://doi.org/10.1080/02602938.2026.2697962 --- ## [Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education](https://edtechdev.github.io/aied/articles/ai-generated-instructional-videos-computing-ed/) Studies student perceptions of AI-generated instructional videos in computing education. Finds students value personalization and rapid production but express concerns about accuracy and the loss of instructor presence. Identifies clear preferences for hybrid approaches where AI generates draft content that instructors review and refine. ## Key Findings - In a descriptive post-test survey, 170 computing students at two U.S. institutions watched three 3-minute AI-generated videos on the Markdown markup language, created with the Knowlify tool, and then reported their perceptions of the videos and of AI-generated video use in education more broadly. - Students rated the specific AI-generated videos as high-quality, accurate, and usable, and nearly half were unable to determine that the videos were AI-generated. - Despite positive ratings of the videos themselves, students expressed limited comfort with the widespread adoption of AI-generated videos in the classroom. - Students preferred AI videos for simple, supplemental, and visual use cases, while expressing concerns about lower-quality or inaccurate content, reduced instructor interaction, and diminished educational value. - The study positions AI video generation as a way for instructors to create personalized educational videos efficiently and cost-effectively, extending computing-education research beyond text-based AI tools. ## Study Design & Method The study addresses a gap in computing education research, which has focused largely on text-based AI tools for developing learning resources even as advances in AI video models make high-quality personalized videos feasible. Using a descriptive post-test survey design, the researchers recruited 170 computing students across two U.S. institutions. Participants watched three three-minute AI-generated videos on Markdown and completed a survey covering both perceptions of those videos and broader views on AI-generated videos in education. Outcomes were analyzed descriptively, with attention to whether students could detect AI authorship and how stated preferences varied by use case. ## Implications for AI in Education The results give computing instructors an evidence base for purposeful use: AI-generated video is acceptable for targeted, supplementary, visually oriented content, but students remain wary of it replacing instructor interaction or carrying high-stakes, accuracy-sensitive instruction. The fact that nearly half of students could not detect AI authorship, alongside concerns about inaccurate content, underscores the need for transparency and review workflows in which instructors validate AI-produced media. The preference for a hybrid model — AI-generated drafts refined by instructors — aligns with [[instructional-design]] principles that treat generative tools as augmentations of, rather than replacements for, the instructor's role in [[cs-education]], and it connects to broader questions of [[student-experience]] and trust in [[generative-ai]] learning resources. ## Connected Concepts - [[cs-education]] - [[student-experience]] - [[instructional-design]] - [[math-education]] - [[administrator]] - [[socratic-ai-dialogue]] - [[physics-education]] - [[pedagogical-agent]] ## Connected Articles - [[ai-generated-interactive-fiction-education-2026]] — AI-Generated Interactive Fiction for Educational Use: A Pilot Study of Perceived Comprehensibility, Coherence, and Engagement - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability - [[solving-vs-evaluating-genai-solutions]] — Is Solving Better Than Evaluating GenAI Solutions? ## Citation Esse Ciego, Shubbhi Taneja, Wilson Wong, Amanpreet Kapoor (2026). [Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education](https://arxiv.org/abs/2607.28203). arXiv:2607.28203. cs.HC. --- ## [AI-Generated Interactive Fiction for Educational Use: A Pilot Study of Perceived Comprehensibility, Coherence, and Engagement](https://edtechdev.github.io/aied/articles/ai-generated-interactive-fiction-education-2026/) > **Synthesis:** This pilot study (N = 22 STEM higher-education students) evaluates AI-generated interactive fiction as an educational medium. Narrative clarity and length acceptance rated positively, engagement hovered near neutral, and story-content coherence was the weakest dimension — with quiz integration emerging as the main usability bottleneck. The authors derive concrete design implications for interactive, narrative learning experiences. ## Key Findings 1. **Generation is not enough.** Generative AI can produce interactive narrative content at scale, but scenarios that are confusing, narratively inconsistent, or unengaging are unlikely to be useful in practice — quality must be validated with users, not assumed. 2. **Mixed user ratings.** Participants rated narrative clarity and length acceptance positively, engagement sat near the neutral midpoint, and story-content coherence was the weakest dimension by a clear margin. 3. **Quiz integration is the bottleneck.** Qualitative feedback identified artificial in-fiction motivation for quiz prompts, abrupt setting changes, and the absence of story-level consequences for wrong answers as the main friction points. ## Implications For [[generative-ai]]-driven [[ai-tutoring]] and narrative [[curriculum-design]], the study suggests that scaffolding learning checks (quizzes) into interactive stories requires more than inserting questions — the story must motivate and respond to them coherently. This bears on [[student-experience]] and [[engagement-metrics]]: perceived coherence appears to be the limiting factor for sustained engagement with AI-produced narrative content. The work connects to broader efforts to make AI-generated instructional media usable, including AI-generated videos and slides in computing education, and to [[educational-nlp]] research on automated content generation. As a pilot with a small sample, it is a design-oriented contribution that motivates larger follow-ups on learning effectiveness rather than an efficacy study in itself. ## Connected Concepts - [[generative-ai]] - [[educational-nlp]] - [[engagement-metrics]] - [[student-experience]] - [[curriculum-design]] - [[pedagogical-agent]] - [[higher-ed]] ## Connected Articles - [[ai-generated-instructional-videos-computing-ed]] — AI-generated instructional videos in computing ed - [[ai-generated-slides-student-perception]] — Student perception of AI-generated slides - [[ai-enabled-serious-games]] — AI-enabled serious games - [[socratic-ai-physics-tutor-taxonomy-2026]] — Socratic AI physics tutor taxonomy - [[agentic-ai-education-scoping-review]] — Agentic AI in education scoping review ## Citation Rogosch, F., & Schrader, A. (2026). [*AI-Generated Interactive Fiction for Educational Use: A Pilot Study of Perceived Comprehensibility, Coherence, and Engagement*](https://doi.org/10.21125/edulearn.2026.1075). EDULEARN26 Proceedings, Article 1075. --- ## [AI-Generated Slides: Are They Good? Can Students Tell?](https://edtechdev.github.io/aied/articles/ai-generated-slides-student-perception/) This study evaluated five generative AI tools for creating instructional slides from instructor-authored course notes: NotebookLM, Claude, M365 Copilot, Cursor, and Claude Code. Educators assessed slides for accuracy, completeness, and pedagogical soundness. - **Coding assistants (Cursor, Claude Code) produced the best slides** — most accurate, complete, and pedagogically sound. - **Students rated AI-generated slides as similar in quality to instructor-created slides.** - **Students could not reliably identify which slides were AI-generated.** - A negative correlation emerged: high quality ratings were associated with lower "AI-generated" guesses, suggesting **students associate poor quality with AI origin** even when AI slides are good. ## Pedagogical Implications The finding that coding assistants outperform dedicated education tools and general-purpose LLMs suggests that the scaffolding and structured output formats inherent to coding tools may translate to better instructional design outputs. The student bias finding — associating poor quality with AI — connects to [[ai-literacy]] research: if students penalize content they suspect is AI-generated, transparency about AI use may backfire unless accompanied by demonstrated quality. ## Connected Concepts - [[ai-literacy]] ## Connected Articles - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation - [[students-llm-usage-critical-thinking]] — Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs ## Citation Leinonen, J., Zhang, L., & Hellas, A. (2026). [AI-generated slides: Are they good? Can students tell?](https://arxiv.org/abs/2605.13532) Proceedings of the Western Canada Conference on Computing Education (WCCCE 2026). arXiv:2605.13532. --- ## [Studying Circular Motion with an AI-Generated Smartphone Physics Lab](https://edtechdev.github.io/aied/articles/ai-generated-smartphone-circular-motion-lab-2026/) > **Synthesis:** Suñer et al. (2026) show that a fully customized, browser-based rotation laboratory can be generated entirely through natural-language prompting of an AI assistant, with no manual coding. Most smartphone physics experiments rely on precompiled sensor apps whose interfaces cannot be tailored to a specific activity, and customized labs previously required programming knowledge beyond most teachers. Using the AI-generated lab with a simple rotating platform, they characterize uniform circular motion (UCM) and uniformly accelerated circular motion (UACM), validating sensor measurements against independent video analysis with Tracker. The work connects to [[physics-education]], mobile learning, and [[generative-ai]] content creation. ## Customizing Smartphone Labs Without Code Smartphones have become a standard measurement instrument in the physics laboratory, with built-in accelerometers, gyroscopes, magnetometers, and cameras used to investigate mechanics. A recurring limitation is that most experiments rely on precompiled sensor apps whose interfaces cannot be tailored to a specific activity, and until recently creating customized smartphone laboratories required programming knowledge beyond most physics teachers. ## AI-Generated, Browser-Based Lab Following the approach the authors recently introduced for acoustic experiments, this paper shows a fully customized, browser-based rotation laboratory can be generated entirely through natural-language prompting of an AI assistant, with no manual coding. ## Validation The lab is used, together with a simple rotating platform, to characterize both uniform circular motion (UCM) and uniformly accelerated circular motion (UACM). Sensor measurements are validated against independent video analysis with Tracker. ## Connected Concepts - [[physics-education]] - mobile learning - [[generative-ai]] - [[generative-ai]] - [[stem-education]] - [[personalized-learning]] - [[scaffolding]] - [[teacher-role]] ## Connected Articles - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[becker-chatgpt-typology-physics-2026]] - [[hashmi-socratic-physics-chatbot-2025]] - [[retrieval-augmented-tutoring-algorithm-kite]] - [[multiagent-classroom-dual-process-physics-teachers-2026]] ## Citation Suñer, J. Ll., Muñoz-Pérez, F. M., Castro-Palacio, J. C., Monsoriu, J. A., Monteiro, M., Stari, C., & Martí, A. C. (2026). [*Studying circular motion with an AI-generated smartphone physics lab*](https://arxiv.org/abs/2607.28352). arXiv:2607.28352. --- ## [AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study](https://edtechdev.github.io/aied/articles/ai-generated-traces-novice-programmers/) Multi-institutional study on Generated Animated Traces (GATs) for CS1. Found that mid-engagement students may experience a performance decrement due to coordination costs (Expertise-Reversal Effect). [[cs-education]], [[scaffolding]], [[personalized-learning]], [[stem-education]], [[adaptive-learning]]. ## Key Findings - Generated Animated Traces (GATs) — AI-generated, analogy-based, narrated animations that coordinate source code, execution state, and conceptual analogies — were evaluated against textual explanations in CS1 courses at two institutions (Python, N=961; Java, N=151). - GATs yielded selective benefits for immediate learning, but the benefits were context-dependent and short-term rather than uniformly positive. - GATs' influence on performance was moderated by learners' engagement profiles: mid-engagement students could experience a performance decrement, interpreted in terms of coordination costs consistent with the expertise-reversal effect. - End-of-course engagement and exam performance did not show that GATs produced durable advantages over textual explanations. - The findings underscore the importance of personalizing instructional support to learner characteristics rather than treating AI-generated visualizations as universally beneficial. ## Study Design & Method The multi-institutional study compared GATs with textual explanations in introductory programming courses at two universities, one teaching Python and one teaching Java. Immediate learning performance and learner experience were measured at the point of instruction, followed by end-of-course engagement and exam performance to test longer-term effects. Learner engagement profiles were derived from the data and used as moderators, allowing the authors to detect differential effects that aggregate analyses could mask. The framing draws on program-visualization research and cognitive load theory, in which the effectiveness of visualizations depends on design and context. ## Implications for AI in Education The study is a cautionary counterpoint to enthusiasm for AI-generated learning media: generative visualizations are not automatically better than text, and their effects vary by learner. For [[cs-education]], the results argue for personalized or adaptive deployment of GATs — aligned with [[personalized-learning]] and [[adaptive-learning]] — rather than uniform adoption across a course. The expertise-reversal-style finding, where mid-engagement students bore coordination costs, suggests that scaffolding decisions must consider the learner's current state, and that AI-generated resources should be designed to reduce extraneous [[cognitive-load-theory]] rather than add to it. Because benefits were short-term and context-dependent, GATs are best treated as one tool within a broader instructional palette rather than a replacement for established explanations. ## Connected Concepts - [[cs-education]] - [[scaffolding]] - [[personalized-learning]] - [[stem-education]] - [[adaptive-learning]] ## Connected Articles - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[prompt-coach-agentic-tutor-prompt-engineering]] — Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Noviello et al. (2026). [AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study](https://arxiv.org/abs/2606.03288). arXiv:2606.03288. --- ## [AI-Guided Learning: Research on Knowledge and Skill Acquisition Support Methods Using Deep Learning Audio-Video Processing Techniques](https://edtechdev.github.io/aied/articles/ai-guided-learning-audiovideo-2026/) > **Synthesis:** This dissertation develops an AI-guided learning framework that supports three interconnected stages — Consume, Understand, and Imitate — with three deep-learning systems for audio/video learning. AIxSpeed adapts audio playback speed using speech-recognition confidence; FastPerson produces multimodal video summaries; and Profy supports pronunciation practice from largely unannotated speech. Evaluations show efficiency gains (up to 1.30x playback, 53% less viewing time) with no loss in learning outcomes, and improved pronunciation intelligibility. ## Key Findings 1. **Consume — adaptive audio speed.** AIxSpeed dynamically adjusts playback speed at the phoneme level using speech-recognition confidence as a proxy for listening difficulty, achieving average playback factors of 1.30x (LibriSpeech) and 1.29x (UME-ERJ) with higher mean opinion scores than matched constant-speed playback. 2. **Understand — multimodal summaries.** FastPerson generates video summaries preserving visual and auditory information and lets learners switch between summarized and full versions by chapter, reducing viewing time by 53% with no statistically significant difference in quiz scores. 3. **Imitate — proficiency feedback.** Profy learns proficiency from largely unannotated speech, visualizes classifier-relevant regions and model-derived acoustic distances, and produced an observed improvement in pronunciation intelligibility with non-overlapping pre/post confidence intervals. ## Implications The framework illustrates how [[personalized-learning]] can address the time cost of sequential audio/video content and the lack of scalable feedback for imitation-based skill acquisition, notably in [[language-learning]] pronunciation. The systems exemplify a [[feedback-loop]] design where model confidence and learner behavior jointly drive adaptation, and they connect to [[multimodal]] and [[student-modeling]] research. By preserving learner access to the original material while adding AI assistance, the work aligns with [[self-regulated-learning]] and [[lifelong-learning]] goals — supporting efficient consumption and repeated practice without removing learner agency. The three-stage Consume–Understand–Imitate framing offers a reusable model for AI-guided media learning. ## Connected Concepts - [[personalized-learning]] - [[language-learning]] - [[feedback-loop]] - [[self-regulated-learning]] - [[multimodal]] - [[student-modeling]] - [[learning-analytics]] - [[lifelong-learning]] ## Connected Articles - [[ai-assisted-learning-modes-eeg]] — AI-assisted learning modes (EEG) - Lecture Review — Lecture review with AI - [[ai-generated-instructional-videos-computing-ed]] — AI-generated instructional videos - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] — Emotional AI on L2 learning - [[ai-adult-learning-design]] — AI in adult learning design ## Citation Kawamura, K. (2026). [*AI-guided learning: Research on knowledge and skill acquisition support methods using deep learning audio-video processing techniques*](https://arxiv.org/abs/2608.08990). arXiv:2608.08990. cs.HC. Doctoral dissertation, University of Tokyo. --- ## [Higher Education Must Bridge the AI Gap](https://edtechdev.github.io/aied/articles/ai-higher-ed-bridge-gap/) > A Science editorial by University of Illinois Chicago Chancellor Marie Lynn Miranda (April 2026) arguing that higher education has a narrow window to shape AI's distribution equitably. Proposes a three-pillar AI literacy framework: practical fluency, critical understanding, and ethical/professional use. ## Definition A Science editorial by University of Illinois Chicago Chancellor Marie Lynn Miranda (April 2026) arguing that higher education has a narrow window to shape AI's distribution equitably. Proposes a three-pillar AI literacy framework: practical fluency, critical understanding, and ethical/professional use. ## The Urgency Argument AI's unprecedented speed, scale, and portability compress the time institutions have to respond. Historical technological revolutions widened divides — AI risks repeating this pattern. Institutions serving low-income and first-generation students face especially urgent pressure to act at AI's pace. ## Three Pillars of AI Literacy 1. **Practical Fluency** — prompt design, AI workflow integration, human-AI collaboration 2. **Critical Understanding** — LLMs don't reason or access truth; they predict patterns; training data can be manipulated; cross-checking with human expertise is essential 3. **Ethical and Professional Use** — when to acknowledge AI use, distinguishing AI strengths from failures, aligning with professional standards ## Cross-Sector Response An eLetter response argued universities cannot bridge the gap alone — commercial LLMs are structurally opaque. Proposed cross-sector ecosystems (universities + research centers + industry) for algorithmic transparency. Cited China's "AI + Education" Action Plan (April 2026) and the Xiaoya platform deployed across 3,500+ universities. ## Connected Concepts - [[agentic-ai]] - [[ai-literacy]] - [[equity-in-ai-education]] - [[higher-ed]] - [[persistent-ai-agents-academic-research]] - [[regulation]] - [[llm]] ## Connected Articles - [[ai-in-the-wild-college]] — AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI - [[cognitive-shift-ai-education]] — Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence? - [[genai-declaration-frameworks-higher-education]] — Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education - [[international-students-conversational-ai-adaptation]] — Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation - [[principled-ai-education]] — Principled AI in Education - [[student-regulatory-awareness-genai]] — Knowing the Rules Is Not Enough: Student Regulatory Awareness and Use of GenAI in Higher Education - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform ## Citation Miranda, M. L. (2026). [Higher Education Must Bridge the AI Gap](https://www.science.org/doi/10.1126/science.aeh5777). *Science*. --- ## [The Impact of AI on Work in Higher Education](https://edtechdev.github.io/aied/articles/ai-higher-ed-workforce-survey/) > A large-scale survey (n=1,960) by EDUCAUSE (2026) examining how AI is reshaping work in higher education institutions — attitudes, adoption patterns, institutional strategies, risks, and opportunities. # AI Impact on Higher Ed Workforce (EDUCAUSE 2026) ## Definition A large-scale survey (n=1,960) by EDUCAUSE (2026) examining how AI is reshaping work in higher education institutions — attitudes, adoption patterns, institutional strategies, risks, and opportunities. ## Key Statistics - **81%** feel enthusiasm or mixed caution/enthusiasm toward AI; only 17% cautious - **73%** of recent users employ AI daily or weekly (38% daily, 34% weekly) - **94%** used AI for work in past 6 months, but only **54%** are aware of institutional policies - **56%** have used AI tools *not provided* by their institution (shadow AI) - Only **13%** of institutions measure ROI for AI tools - **92%** have a work-related AI strategy; 69% upskill existing staff via self-guided learning - Top risks: misinformation (55%), data consent (52%), loss of fundamental skills (51%) ## Implications The survey reveals a significant policy-awareness gap: despite near-universal AI use, nearly half of respondents are unaware of institutional AI policies. This suggests policies either don't exist or aren't effectively communicated. The prevalence of "shadow AI" use (56%) indicates a disconnect between institutional tool provision and staff needs. Connects to [[faculty-development]] regarding upskilling challenges and [[ai-literacy]] for the awareness gap. ## Connected Concepts - [[administrator]] - [[ai-literacy]] - [[faculty-development]] - [[higher-ed]] - [[regulation]] ## Connected Articles - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation EDUCAUSE. (2026). [The Impact of AI on Work in Higher Education](https://www.educause.edu/research/2026/the-impact-of-ai-on-work-in-higher-education). --- ## [AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI](https://edtechdev.github.io/aied/articles/ai-in-the-wild-college/) Karidi, Amir & Roll (2026) present one of the largest empirical analyses to date of authentic (rather than lab-based) interactions between college students and generative AI tools. By analyzing interaction logs at scale, they identify distinct patterns: some students use AI as a [[llm]]-powered learning companion for explanation and exploration, while others offload cognitive work entirely — copying outputs without comprehension. The work provides much-needed ecological validity to a literature that has largely relied on survey self-reports and controlled experiments. These findings connect directly to [[student-experience]] research, showing that the gap between AI 'use' and AI 'learning' is wide and context-dependent. The study offers actionable design implications for [[ai-higher-ed-bridge-gap]]: platforms should scaffold metacognitive engagement with AI outputs rather than optimizing for answer correctness alone. The paper also contributes to [[generative-ai]] literacy frameworks by mapping what competencies students actually display in uncontrolled settings, informing [[ai-literacy]] curriculum design. ## Connected Concepts - [[llm]] - [[student-experience]] - [[generative-ai]] - [[ai-literacy]] ## Connected Articles - [[ai-higher-ed-bridge-gap]] ## Citation Taelin Karidi, Ofra Amir, Ido Roll (2026). [AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI](https://arxiv.org/abs/2606.29442). arXiv:2606.29442. cs.CY (AIED 2026). --- ## [Artificial Intelligence as Catalyst and Contested Terrain: Transforming Interior Design Practice, Pedagogy, and Professional Regulation in Malaysia](https://edtechdev.github.io/aied/articles/ai-interior-design-malaysia-2026/) > **Synthesis:** This article examines how generative AI and intelligent visualization platforms are reshaping interior design practice in Malaysia, shifting designers from primary form-generators toward critical mediators and curators of machine outputs. It explores the implications for university curricula, arguing that professional education must integrate technical proficiency with critical and ethical judgment, and addresses emerging needs for professional regulation. ## Key Findings 1. **Changing professional identity.** Generative AI tools, intelligent visualization platforms, and AI-native residential developments (e.g., AI Living at i-City) are altering design processes, client expectations, and market dynamics in Malaysia. 2. **New designer role.** Practicing designers are shifting from primary form-generators toward critical mediators and curators of machine outputs, while retaining essential contextual and ethical judgment. 3. **Curriculum and regulation needs.** Universities must develop curricula that combine technical proficiency with critical and ethical judgment so graduates can evaluate AI suggestions rather than rely on them uncritically, and the profession faces emerging needs for regulation as AI becomes embedded in practice. ## Implications For [[curriculum-design]] and [[professional-training]] in design and other practice-based fields, the paper is a case study in how [[generative-ai]] reshapes not just workflows but professional identity and the purposes of professional education. It connects to [[teacher-role]] questions about what practitioners must learn to supervise and curate machine outputs rather than simply produce them. The analysis also speaks to [[higher-ed]] and [[educational-policy-ai]]: institutions are being asked to decide how much of a profession's technical skill remains human-owned as AI mediates it, and how to regulate that boundary. This has parallels for [[ai-education]] across applied disciplines and for [[equity-in-ai-education]] as market and institutional pressures accelerate AI adoption unevenly. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[curriculum-design]] - [[professional-training]] - [[teacher-role]] - [[equity-in-ai-education]] - [[ai-education]] - [[educational-policy-ai]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI changes teaching workflows - [[ai-education-global-capacity]] — Global capacity for AI education - [[ai-ethics-education-public-discourse]] — AI ethics in public discourse - [[ai-adult-learning-design]] — AI in adult learning design - [[pchl-he-framework-genai-content-creation-2026]] — PCHL-HE framework for content creation ## Citation Syed Abdul Rahman, S. F. (2026). [*Artificial intelligence as catalyst and contested terrain: Transforming interior design practice, pedagogy, and professional regulation in Malaysia*](https://osf.io/preprints/edarxiv/dvncj_v1/). EdArXiv preprint. --- ## [What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue](https://edtechdev.github.io/aied/articles/ai-interlocutor-l2-spoken-dialogue/) Scheinberg et al. (2026) analyze 78 university learners of German across four sites completing a counterbalanced spot-the-difference task with both a human peer and a real-time AI partner. Using diarized ASR transcripts, they extract measures of interactional fluency, linguistic uptake, and learner experience. Human dialogue was faster and more balanced with many short turns; AI dialogue resembled 'supported monologue' with fewer, longer turns, reduced learner floor share, and greater within-turn fluency. The AI's verbose, syntactically regular input was associated with greater short-term uptake and stronger syntactic priming after controlling for input volume. Attitudes toward AI improved after the task, and satisfaction was predicted by production fluency rather than uptake. This work shows complementary affordances of AI and human dialogue for [[language-learning]], informing [[intelligent-tutoring]] design for second language acquisition, with implications for [[student-experience]] and [[personalized-learning]] in [[llm]]-mediated education. ## Connected Concepts - [[language-learning]] - [[intelligent-tutoring]] - [[student-experience]] - [[personalized-learning]] - [[llm]] ## Connected Articles - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[history-aware-student-simulation]] — Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues - [[ecnuclaw-k12-personalized-companion]] — ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System ## Citation Russell Scheinberg, Ameeta Agrawal, Tetyana Sydorenko, Kalab Kahsay, Nina Vyatkina, Griet Boone (2026). [What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue](https://arxiv.org/abs/2606.22225). arXiv:2606.22225. Educational Data Mining 2026 --- ## [Unravelling undergraduates' development of evaluative judgments through AI-supported internal feedback](https://edtechdev.github.io/aied/articles/ai-internal-feedback-evaluative-judgments/) > **Synthesis:** Unravelling undergraduates' development of evaluative judgments through AI-supported internal feedback ## Key Findings - This qualitative case study followed **28 second-year pre-service teachers** (18 female, 10 male; Mage = 20.6) in a compulsory English writing course at a university in northern China across a **14-week semester with three writing tasks** (150–250-word essays at College English Test Band-6 level). - The intervention used **DeepSeek R1** to generate strong, average, and weak exemplars; students ranked their own draft against the three exemplars, justified the rankings on self-reflection forms, then sought AI feedback, compared it against the criteria and their goals, and revised. - Through iterative analogical comparison (draft vs. exemplars) and analytical comparison (draft vs. AI feedback, criteria, and goals), students' **evaluation focus expanded from language features to higher-order aspects of writing** — content, organisation, and coherence — although their evaluative reasoning generally **lacked sophistication**: example-based justification doubled from 15% to 30.5% across tasks while simple statements declined, but the most sophisticated comparative reasoning remained rare. - Internal feedback shifted over the semester from reflections on writing skills and criteria (54.5% and 33.3% of codes in Task 1) toward **metacognition, goal alignment, and self-monitoring** — with metacognitive reflections rising exponentially from 3% in Task 1 to 24.3% in Task 3. - Three evaluator types emerged with distinct developmental pathways: **reconstructive (N = 8)**, **language-focused (N = 9)**, and **criteria-compliance (N = 11)** evaluators. ## Study Design & Method The study is grounded in the internal feedback paradigm: students generate feedback by comparing their work against external information (exemplars, criteria, goals, AI feedback) — with the comparison itself producing the learning gain. Each task cycle had students write an initial draft, set goals, rank their draft against three AI-generated exemplars of varied quality and justify the ranking, request and compare DeepSeek R1 feedback, revise, and set goals for the next task. The instructor (a senior lecturer with over 15 years of experience) explained the writing criteria in advance. Data sources were students' self-reflection forms, semi-structured interviews, and initial and revised essay drafts, analysed qualitatively to trace evaluative judgment development over time. ## Key Results - **Expanding evaluation focus:** students' attention moved from language-level features toward content, organisation, and coherence across the three tasks, indicating growth in what they considered quality writing. - **Reasoning quality:** most students could discern features of quality writing but mainly indicated presence or absence of features without deeper explanation. Example-based reasoning rose (15% → 30.5%), simple statements fell steadily, but comparative analysis — the most sophisticated form — did not emerge. - **Internalization trend:** attention to writing skills and criteria dropped (54.5% → 21.6% and 33.3% → 5.4% respectively), while metacognition, goal alignment, and self-monitoring rose — consistent with the strategies becoming embedded in writing practice and iterative goal-setting cycles catalyzing self-monitoring. - **Evaluator types:** reconstructive evaluators expanded their focus from language to higher-order dimensions but remained descriptive in reasoning; language-focused evaluators (like "Phoebe") consistently prioritized grammatical accuracy and vocabulary, refining rather than broadening their conception of quality; criteria-compliance evaluators (N = 11) anchored judgments to the stated criteria. AI feedback shaped each pathway differently — e.g., one language-focused student doubted DeepSeek's Band-6 advice until the instructor confirmed it, refining her understanding of exam requirements. ## Implications for AI in Education The study demonstrates a practical strategy for turning GenAI into a **scaffold for assessment literacy** rather than a shortcut: comparing drafts against AI-generated exemplars of varied quality helps students appreciate a "quality continuum," while comparing drafts with AI feedback and personal goals makes the internal feedback explicit and auditable. Having students articulate their internal feedback on self-reflection forms both reduces cognitive load and lets teachers track progress. The three developmental pathways imply that **one-size-fits-all prompting fails** — teachers should customize self-reflection prompts to each student's evaluative orientation (e.g., pushing language-focused evaluators toward organisation and coherence, while reinforcing evidence-based reasoning for reconstructive evaluators). This connects to [[self-regulated-learning]] and [[feedback-loop]] research: students become agentic seekers of AI feedback who monitor goal attainment and judge the contextual appropriateness of AI suggestions, echoing concerns in [[assessment]] about students' critical evaluation of AI-generated feedback. ## Limitations - The accuracy of students' rankings and evaluative reasoning was not examined (resource constraints), and the course instructor's perspective was not elicited during data analysis despite member-checking with some students. - Participants came from a single institution and discipline, limiting transferability across disciplinary and institutional settings. - Students sought external feedback mainly from AI, and not all AI feedback is accurate — which may have influenced evaluative judgment development. The authors recommend teachers comment on AI feedback quality and guide students to evaluate it critically. ## Connected Concepts - [[self-regulated-learning]] - [[assessment]] - [[feedback-loop]] - [[metacognition]] - [[ai-tutoring]] - [[reducing-ai-misuse]] - [[math-education]] - [[prompt-engineering]] ## Connected Articles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[llm-automated-assessment-student-self-explanations]] — Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[genai-performance-vs-learning]] — Distinguishing performance gains from learning when using generative AI ## Citation Chen, S., & To, J. (2026). [*Unravelling undergraduates' development of evaluative judgments through AI-supported internal feedback*](https://doi.org/10.1080/02602938.2026.2711099). --- ## [Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis](https://edtechdev.github.io/aied/articles/ai-learning-assistants-higher-ed-large-scale/) Presents a large-scale descriptive analysis of an AI learning assistant (Syntea) using objective log data from 77,543 higher-education students, characterizing real usage patterns, adoption, and engagement at scale. The work connects to broader debates about how [[generative-ai]] systems reshape [[student-experience]] and the conditions under which AI support [[scaffolding|scaffolds]] rather than undermines learning. It has direct implications for [[pedagogy-ai-mistakes]] and the risk of [[over-reliance]] when assistants absorb too much of the cognitive load. Findings also bear on [[ai-literacy]] and [[self-regulated-learning]], and on how institutions should govern [[student-experience]] and [[academic-integrity]]. Practitioners in [[higher-ed]] and [[teacher-role|teachers]] can use the evidence to calibrate when to deploy [[llm]]-based help and how to pair it with [[feedback-loop|feedback]] that preserves learning gains. ## Connected Concepts - [[generative-ai]] - [[student-experience]] - [[scaffolding]] - [[over-reliance]] - [[ai-literacy]] - [[self-regulated-learning]] - [[academic-integrity]] - [[higher-ed]] - [[teacher-role]] - [[llm]] - [[feedback-loop]] ## Connected Articles - [[pedagogy-ai-mistakes]] ## Citation A. Leypold, M. Knolle, A. F. D. S. et al. (2026). [Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis](https://arxiv.org/abs/2607.08748). arXiv:2607.08748. --- ## [Building AI Companions that Prioritise Learning over Performance](https://edtechdev.github.io/aied/articles/ai-learning-companions-framework/) > A design framework for LLM-powered educational agents that prioritize durable learning over short-term task performance. Introduced by Khosravi et al. (2026), AI learning companions are defined as adaptive, pedagogically informed agents integrated into learning environments — distinct from both task-oriented LLMs and simple prompted tutors. # AI Learning Companions Framework ## Definition A design framework for LLM-powered educational agents that prioritize durable learning over short-term task performance. Introduced by Khosravi et al. (2026), AI learning companions are defined as adaptive, pedagogically informed agents integrated into learning environments — distinct from both task-oriented LLMs and simple prompted tutors. ## The Learning-Performance Paradox LLMs demonstrably improve task outputs (writing quality, code correctness, analysis speed), but this can create a paradox: students who produce better work with AI may learn less. This mirrors and extends the [[llm-fallacy-misattribution]] phenomenon where users misattribute AI-assisted outputs to their own competence. The framework addresses the question posed by [[transfer-of-learning]]: do AI-assisted gains persist when the tool is removed? ## Three Foundations 1. **Pedagogical Foundation** — grounded in learning science; how students learn *with* AI, not just *from* it 2. **Adaptive Foundation** — AI learns about the student over time, connecting to [[llm-student-modeling-memory]] and longitudinal personalization 3. **Responsible Design Foundation** — transparency, accountability, inclusivity, security; aligns with [[ai-tutor-safety-harms]] harm taxonomy ## Case Studies Validated across five diverse educational contexts, levels, and tool designs, revealing both promise and current limitations. The framework calls for a deliberate shift from task-optimized LLMs toward companions that foster durable understanding, metacognitive growth, and learner agency — connecting directly to [[self-regulated-learning]] and [[metacognition]]. ## Connected Concepts - [[self-regulated-learning]] - [[metacognition]] ## Connected Articles - [[llm-fallacy-misattribution]] - [[transfer-of-learning]] - [[llm-student-modeling-memory]] - [[ai-tutor-safety-harms]] ## Citation Khosravi, H., Gasevic, D., Sadiq, S., Yan, L., Lodge, J., Tangen, J., Denny, P., & DiCerbo, K. (2026). [*Building AI Companions that Prioritise Learning over Performance*](https://arxiv.org/abs/2605.04816) --- ## [Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes](https://edtechdev.github.io/aied/articles/ai-learning-tools-engineering-education-needs/) Survey of 206 engineering students: AI chatbots provide greatest perceived benefit as relief from competence frustration, smaller benefits for autonomy, weakest for relatedness. Baseline motivational states matter more than demographics; inattention moderates how baseline competence and autonomy relate to perceived AI benefits. Offers design principles for engineering-specific AI learning tools. **Relevance to AI in Education:** This paper contributes to the understanding of [[automated-assessment]], [[personalized-learning]], and [[student-experience]]. The findings have implications for [[adaptive-learning]] systems, [[formative-assessment]] design, and the broader [[edtech-platform]] landscape. Future work should explore how these results generalize across [[stem-education]] and [[higher-ed]] contexts. This research connects to the growing body of work on [[ai-literacy]] and [[teacher-role]], highlighting both the promise and limitations of AI tools in educational settings. ## Connected Concepts - [[automated-assessment]] - [[personalized-learning]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[edtech-platform]] - [[stem-education]] - [[higher-ed]] - [[ai-literacy]] - [[teacher-role]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education ## Citation Kevin Zhongyang Shao, Denise Wilson, Yale Quan, Sep Makhsous (2026). [Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes](https://arxiv.org/abs/2607.26338). arXiv:2607.26338. Under review. --- ## [Artificial Intelligence in Lifelong Learning: Opportunities and Challenges in Adult Education Policy](https://edtechdev.github.io/aied/articles/ai-lifelong-learning-policy/) Theodora and Tselios (2026) provide a policy-oriented synthesis of AI's dual role in adult and [[lifelong-learning]] contexts — as both an enabler of personalized, scalable education and a source of significant equity and governance challenges. Drawing on international policy frameworks, the paper argues that AI integration in adult education requires balanced policies promoting inclusion, transparency, and human-centered pedagogy. The paper fills a notable gap in the [[stanford-evidence-base-ai-k12-2026]] and related literature, which has predominantly focused on K-12 and higher education. Adult learners face distinct challenges: they are more likely to be affected by the [[equity]], have less institutional support than traditional students, and often engage in self-directed learning where AI tools may operate with minimal human oversight. The paper's emphasis on [[educational-policy-ai]] responsibilities connects to broader [[regulation]] discussions. Key tensions identified include the promise of [[intelligent-tutoring]] and [[learning-analytics]] for personalized workforce development versus risks of [[bias-mitigation]] and [[over-reliance]]. The call for "responsible innovation" aligns with [[equity]] frameworks in educational AI and echoes concerns about [[ai-tutor-safety-harms]] in contexts where adult learners may lack the support structures available in formal education. For [[faculty-development]] and [[teacher-role]] in adult education, the paper highlights the urgent need for educator training in AI literacy — a theme consistent with [[ai-pedagogical-orientation]] research. ## Connected Concepts - [[lifelong-learning]] - [[equity]] - [[educational-policy-ai]] - [[regulation]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[bias-mitigation]] - [[over-reliance]] - [[equity]] - [[faculty-development]] - [[teacher-role]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-safety-harms]] - [[ai-pedagogical-orientation]] ## Citation Andresa Theodora, Nikolaos Tselios (2026). [Artificial Intelligence in Lifelong Learning: Opportunities and Challenges in Adult Education Policy](https://arxiv.org/abs/2605.16296). arXiv:2605.16296. arXiv:2605.16296 [cs.CY]. --- ## [AI Literacy Assessment: Self-Reported vs Performance Misalignment](https://edtechdev.github.io/aied/articles/ai-literacy-assessment-misalignment/) >Highlights critical misalignment between self-reported AI literacy and actual performance. Teachers overestimate their AI skills by 40% on average. Performance-based assessments correlate better (r=0.72) with classroom AI integration than self-reports (r=0.31). ## Key Findings - The study developed and evaluated parallel self-report (SR) and objective-based (OB) measures of teacher AI literacy within a shared Concept, Use, Evaluate, and Ethics framework, enabling direct comparison of perceived versus demonstrated competence. - Confirmatory factor analyses supported construct validity with good reliability and acceptable fit, but revealed a low correlation between the self-reported and objective-based factors — teachers' perceptions of their AI literacy and their demonstrated performance diverge. - Latent profile analysis identified six distinct profiles, including overestimation (SR > OB), underestimation (SR < OB), alignment (SR ≈ OB), and a unique low-SR/low-OB profile concentrated among teachers without prior AI literacy experience. - The divergence between perceived and demonstrated competence has direct implications for professional development: self-assessment alone is an unreliable basis for planning AI training. - The instruments function as diagnostic tools supporting AI-informed decisions such as growth monitoring and needs profiling, and enable scalable learning-analytics interventions tailored to teacher subgroups. ## Study Design & Method The research responds to the widespread adoption of AI in K-12 education and the resulting need for psychometrically tested measures of teachers' AI literacy. Prior work relied on either self-report or objective-based assessments, with few studies aligning the two within a shared framework. Here, both SR and OB instruments were built on shared dimensions — Concept, Use, Evaluate, and Ethics — so the two measurement approaches could be compared directly rather than studied in isolation. Confirmatory factor analysis established construct validity and reliability, and latent profile analysis grouped teachers by the pattern of agreement or disagreement between their self-reports and objective performance, including how prior AI literacy experience shapes that relationship. ## Implications for AI in Education The low correlation between self-reported and objective-based factors is a strong argument for performance-based assessment of AI literacy in teacher preparation and evaluation, where self-report surveys remain common. For [[assessment-validity]], the study shows that what educators say about their AI skills is not a reliable proxy for what they can do. The six-profile structure gives [[faculty-development]] programs a diagnostic basis for targeting support — for example, distinguishing overestimating teachers who need calibration from low-SR/low-OB novices who need foundational experience — and it demonstrates how validated instruments can feed [[learning-analytics]] pipelines that tailor interventions to teacher subgroups. ## Connected Concepts - [[teacher-ai-competency]] - [[metacognition]] - [[k-12-ai-education]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[plagiarism-detection]] - [[help-seeking]] - [[bias-mitigation]] ## Connected Articles - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations ## Citation Zhang, S., Xiao, R., Botelho, A. F., Liao, G., Chiu, T. K. F., Stamper, J., & Koedinger, K. R. (2026). [*How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures*](https://arxiv.org/abs/2601.06101). arXiv:2601.06101. --- ## [AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education](https://edtechdev.github.io/aied/articles/ai-literacy-career-adaptability-business-2026/) > **Synthesis:** Testa, Apuzzo, and Pittaway (2026) investigate how AI-related competencies contribute to career adaptability in business and management education. Surveying 339 university students in economics, management, and business programs in Italy, they employ a moderated mediation model examining the relationships among AI literacy, AI readiness, AI self-efficacy, and career adapt-abilities. Findings show AI readiness mediates the relationship between AI literacy and career adapt-abilities — with the indirect effect varying across levels of AI self-efficacy — and that AI self-efficacy positively moderates the literacy–readiness link while also directly associating with career adapt-abilities. The study positions AI readiness as a central mechanism linking AI-related learning to career-relevant outcomes, arguing that AI literacy alone does not directly translate into career adaptability. ## Key Findings - **AI readiness mediates the relationship between AI literacy and career adapt-abilities** — literacy alone does not directly translate into career outcomes; competencies become career-relevant when translated into readiness to engage with AI in applied settings. - **AI literacy predicts AI readiness (β = 0.148, p = .005)**, and **AI self-efficacy also predicts AI readiness (β = 0.178, p = .005)** — the model predicting AI readiness was significant (F(3,335) = 38.64, p < .001, R² = 0.257). - **AI self-efficacy moderates the literacy–readiness link:** the interaction was significant (β = 0.299, p < .001); the effect of AI literacy on AI readiness was not significant at low self-efficacy (β = −0.067) but positive at medium (β = 0.123) and high (β = 0.327) levels. - **AI self-efficacy shows a direct positive association with career adapt-abilities (β = 0.255, p < .001)** — a distinct motivational resource beyond literacy — and also interacts with AI literacy (β = 0.236, p < .001) in predicting career outcomes. - **Moderated mediation confirmed:** the indirect effect of AI literacy on career adapt-abilities through AI readiness was not significant at low self-efficacy (β = −0.012) but positive and significant at medium (β = 0.022) and high (β = 0.057) levels, with a significant index of moderated mediation (index = 0.052). - The full model predicting career adapt-abilities explained 40% of variance (F(4,334) = 55.75, p < .001, R² = 0.4004), with AI literacy (β = 0.120, p = .008) and AI readiness (β = 0.175, p < .001) as significant predictors. ## Study Design & Method This survey study collected data from 339 university students enrolled in economics, management, and business-related degree programs (including Computer and Data Science for Economics and Corporate Communication) at Italian higher education institutions, using an online questionnaire (Microsoft Forms) distributed between March and June 2025. Students had substantive exposure to AI within their curricula (group work, laboratories, or project-based assignments involving AI). Constructs were measured with validated scales: AI literacy, AI readiness (operationalized at the individual level as motivational and cognitive readiness), AI self-efficacy (Wang & Chuang, 2024), and career adaptability (Career Adapt-Abilities Scale – Short Form, measuring concern, control, curiosity, confidence). Items were translated from English with a back-translation procedure and pre-tested with 17 students. Hypotheses were tested using linear regression and **moderated mediation analysis** (PROCESS Model 8, version 4.0, Hayes 2022) in SPSS, with all variables standardized. Common-method bias was mitigated procedurally and statistically, and all scales demonstrated adequate reliability (Cronbach's α = 0.817–0.829; CR = 0.742–0.860; AVE = 0.602–0.769). ## Implications for AI in Education The study advances understanding of how higher education supports preparation for AI-intensive careers by clarifying the role of AI readiness as a key developmental mechanism. It argues that future research on AI and employability should move beyond treating digital competencies as direct predictors of career outcomes, instead examining the intermediate conditions that enable learning to become career-relevant. The differentiated roles of [[ai-literacy]], self-efficacy, and readiness caution against aggregating them into undifferentiated skill indices. For management education, the findings call for curricular architectures that integrate AI literacy across courses and progression points, pedagogical designs that normalize iterative experimentation with AI tools under guided supervision, and [[assessment]] strategies that evaluate applied judgment (problem framing, verification of outputs, responsible use) alongside technical understanding. It connects to [[student-experience]], [[higher-ed]], [[equity]], and [[motivation]], positioning AI readiness as a central construct for professional preparedness under continuous technological change. ## Limitations The study is cross-sectional, capturing AI literacy, readiness, and career adaptability at an early, anticipatory stage of professional development; it does not observe how these resources are enacted in stable organizational roles. The empirical setting is Italian higher education, so educational norms, labor-market expectations, and interpretations of AI may differ across national contexts. All data were collected via a single survey instrument (though common-method bias was mitigated). The sample focuses on economics/management/business students, and the authors note the model should be assessed across a broader spectrum of disciplines and cohorts, and extended to early-career professionals. ## Connected Concepts - [[ai-literacy]] - [[student-experience]] - [[higher-ed]] - [[assessment]] - [[equity]] - [[motivation]] ## Connected Articles - [[ai-literacy-continuum-higher-education]] — AI Literacy Continuum in Higher Education - [[ai-literacy-assessment-misalignment]] — AI Literacy and Assessment Misalignment - [[gender-differences-ai-literacy-deepfake]] — Gender Differences in AI Literacy - [[genai-skill-bypass-literacy]] — GenAI and Skill Bypass in Literacy - [[ai-autonomous-learning-accomplishment-2026]] — AI, Autonomous Learning, and Accomplishment ## Citation Testa, M., Apuzzo, A., & Pittaway, L. (2026). [*AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education*](https://doi.org/10.1016/j.ijme.2026.101394). *The International Journal of Management Education*, 24, 101394. --- ## [Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education](https://edtechdev.github.io/aied/articles/ai-literacy-continuum-higher-education/) Proposes a five-stage developmental continuum (Not Engaged, Uncritical Use, Informed Use, Critical Evaluation, Improvement) for AI literacy at NC State; the continuum doubles as a diagnostic tool for moving students beyond fluent-but-uncritical tool adoption. Found that reaching higher stages requires discipline-embedded experiences. [[ai-literacy]], [[higher-ed]], [[ethics]], [[faculty-development]], [[student-experience]]. ## Key Findings - The paper proposes a five-stage AI Literacy Continuum — Not Yet Engaged, Uncritical Use, Informed Use, Critical Evaluation, and Improvement — describing developmental orientations toward AI use in higher education. - The continuum responds to two problematic extremes observed in students: avoidance driven by fear, mistrust, ethical concern, or lack of access, and uncritical reliance that produces fluent output while masking misunderstanding. - It complements dimensional competency frameworks by providing educators a practical diagnostic and instructional pathway aligned with international frameworks, including those of UNESCO and OECD. - A design-based implementation case at North Carolina State University engaged more than 330 participants between Fall 2024 and Spring 2026 through credit-bearing courses and intensive hands-on workshops. - Because no validated pre/post instrument or comparison group was used, the findings are observational and practice-based: participants exhibited behaviors consistent with movement from non-engagement or uncritical use toward informed engagement, while sustained and discipline-embedded experiences produced stronger evidence of critical evaluation and improvement-oriented practice. - Stage-specific assessment strategies are proposed, from identifying hallucinated content, appropriate use cases, and articulating AI limitations at early stages, to designing evaluation protocols or demonstrating systematic improvement of AI performance on disciplinary tasks at later stages. ## The Continuum as a Diagnostic Tool The five stages are explicitly intended as a diagnostic device, not just a description: educators can locate where learners begin and design instruction that moves them toward responsible, critical engagement. The framework's practical value lies in pairing each stage with observable behaviors and appropriate assessment, so that AI literacy is treated as a developmental capacity to understand, evaluate, and responsibly apply AI systems in disciplinary and societal contexts — rather than as tool adoption alone. ## Implications for AI in Education The continuum gives [[curriculum-design]] and [[faculty-development]] a shared vocabulary for sequencing AI literacy instruction, and its alignment with international frameworks supports institutional uptake. The NC State case suggests that movement along the continuum is most visible when experiences are sustained and embedded in disciplines, implying that one-off workshops may shift students from non-engagement toward informed use, while deeper stages require ongoing, context-rich opportunities. The authors' framing of their evidence as observational and practice-based also models appropriate epistemic humility for institutions adopting the framework, and their discussion of equity considerations reminds educators that access and disposition shape where students enter the continuum. ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[ethics]] - [[faculty-development]] - [[student-experience]] ## Connected Articles - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[critical-media-literacy-education-2026]] — Technology, Education and Critical Media Literacy: Potential, Challenges, and Opportunities - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-literacy-career-adaptability-business-2026]] — AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education ## Citation Liu & Levy (2026). [Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education](https://arxiv.org/abs/2606.00038). arXiv:2606.00038. --- ## [Programming Language Policy as an AI Literacy Equity Problem: A 15-Nation Comparative Analysis](https://edtechdev.github.io/aied/articles/ai-literacy-equity-programming-policy/) Across 15 nations, the paper examines how secondary computer-science education embeds AI literacy into general-track subjects (Digital Literacy, ICT, TIC, SNT) rather than specialized tracks, creating structural inequities in who develops AI capability. The comparative analysis shows that policy choices about which programming language and subject bears 'universal' AI literacy determine differential access to computational futures. Frames AI literacy 'for all' as an [[equity]] problem rooted in [[k-12]] curriculum policy, with direct relevance to [[educational-policy-ai]] and [[teacher-role]] decisions. It ties to [[ai-literacy]] as a civil competency and to [[faculty-development]] for teacher preparedness, arguing that without equitable language/policy grounding, AI literacy widens rather than closes gaps in [[higher-ed]] readiness. ## Connected Concepts - [[equity]] - [[k-12]] - [[educational-policy-ai]] - [[teacher-role]] - [[ai-literacy]] - [[faculty-development]] - [[higher-ed]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[white-wu-robotics-ai-education-2026]] — Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well - [[lodge-loble-cognitive-offloading-2026]] — Artificial intelligence, cognitive offloading and implications for education - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named ## Citation Adrian-Marius Dumitran, Iulia-Maria Popescu (2026). [Programming Language Policy as an AI Literacy Equity Problem: A 15-Nation Comparative Analysis](https://arxiv.org/abs/2607.11314). arXiv:2607.11314. arXiv preprint. --- ## [The AI Literacy Heptagon: A Structured Approach to AI Literacy in Higher Education](https://edtechdev.github.io/aied/articles/ai-literacy-heptagon-2026/) > **Synthesis:** Hackl, Müller, and Sailer (2026) present the AI Literacy Heptagon, a structured seven-dimensional framework for AI literacy (AIL) in higher education, developed through an integrative literature review of publications from 2021–2024. The framework synthesizes seven core dimensions — technical knowledge and skills, application proficiency, critical thinking ability, ethical awareness and reasoning, social impact understanding, integration skills, and legal and regulatory knowledge — and is operationalized through four Bloom's-taxonomy-aligned proficiency levels (Unaware, Beginner, Intermediate, Expert). An initial expert-led curriculum mapping across an AI Engineering and a Media Pedagogy program demonstrated the framework's utility for analyzing and structuring curricula, highlighting the discipline-specific nature of AIL implementation. ## Key Findings - Seven dimensions emerged from an iterative coding of AIL conceptualizations: Technical Knowledge and Skills (TKS, 24/27 sources), Application Proficiency (AP, 26/27), Critical Thinking Ability (CTA, 23/27), Ethical Awareness and Reasoning (EAR, 20/27), Social Impact Understanding (SIU, 11/27), Integration Skills (IS, 18/27), and Legal and Regulatory Knowledge (LRK, 2/27). - The authors deliberately retained emerging dimensions that appear infrequently in the literature — particularly Legal and Regulatory Knowledge (only 2/27 sources) and Integration Skills — arguing these address rapidly evolving regulatory landscapes (e.g., the EU AI Act) and the gap between theoretical knowledge and practical application. - A synthesized working definition operationalizes AIL as critical, ethical, and responsible engagement with AI across all seven dimensions, explicitly acknowledging that emphasis varies by disciplinary context. - The framework distinguishes generic AIL (Beginner level in all dimensions, the baseline for all students) from domain-specific extensions (Intermediate/Expert levels tailored to fields of study), with four proficiency levels mapped to Bloom's cognitive processes. - Expert-led curriculum mapping of an AI Engineering and a Media Pedagogy program showed technical programs emphasize TKS and AP, while humanities-oriented programs emphasize EAR and SIU, supporting the framework's flexibility while maintaining multidimensional integrity. ## Study Design & Method The study used an integrative literature review (distinct from a meta-analysis, as it performs qualitative synthesis rather than statistical pooling), following PRISMA principles for transparency. A systematic search of Web of Science and Scopus for English-language publications from 2021 (plus earlier foundational works) to 2024 used term combinations around AIL, higher education, teaching/learning AI, and stakeholders. The search was completed December 10, 2024, with additional sources via citation tracking, Google Scholar, and research discovery tools. Two independent coders applied open then axial coding to extract recurring competencies, resolving discrepancies through discussion, yielding the seven dimensions. Initial framework validation used structured expert-led "collaborative profiling sessions" with two program leaders (AI Engineering and Media Pedagogy) to map curricula onto the heptagon and its proficiency levels — explicitly framed as evaluating curricular structure and goals, not empirically measuring student competencies. ## Implications for AI in Education The Heptagon provides educators and institutions a concrete, adaptable tool for structuring [[ai-literacy]] development in [[higher-ed]], addressing the field's fragmentation and the gap between conceptual definitions and curriculum implementation. It argues that AIL is not a monolithic competency but must be tailored to disciplinary contexts while maintaining seven core dimensions. The explicit inclusion of [[educational-policy-ai|legal and regulatory knowledge]] and integration skills responds to underrepresented dimensions in existing frameworks and to the fast-moving regulatory landscape. For curriculum designers, the framework supports competency-gap analysis (as demonstrated in the two validation programs), the design of targeted learning modules across dimensions and proficiency levels, and assessment tasks that integrate multiple AIL dimensions. It also situates AIL relative to media, computational, and data literacy, helping delineate what is and is not AI literacy. ## Limitations The literature review is bounded to 2021–2024 English-language publications in Web of Science and Scopus, potentially missing relevant work. The initial validation is qualitative and small-scale — two expert-led curriculum mappings — explicitly framed as illustrating the framework's utility rather than empirically validating its effectiveness or measuring student competencies. The authors note the visualization lacks granularity at the expert level and that the knowledge-skills-attitudes three-dimensional nature is not explicitly represented in the visual model. The framework is a proposed instrument requiring further validation across more programs and disciplines. ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[curriculum-design]] - [[ethics]] - [[educational-policy-ai]] - [[teacher-role]] - [[assessment-validity]] ## Connected Articles - [[ai-literacy-continuum-higher-education]] — Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum - [[hingle-collaborative-ai-literacy-2025]] — Collaborative AI Literacy Framework - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] — Metacognitive AI Literacy: Going Beyond the Skills Gap Agenda - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] — The Scaffolded AI Literacy (SAIL) Framework: Results of a Delphi Study - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[hcap-human-centric-ai-pedagogy-framework-2026]] — Human-Centric AI Pedagogy (HCAP) Framework - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[principled-ai-education]] — Principled AI in Education - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review ## Citation Hackl, V., Müller, A. E., & Sailer, M. (2026). [*The AI literacy heptagon: A structured approach to AI literacy in higher education*](https://doi.org/10.1016/j.caeai.2026.100540). --- ## [AI Literacy for Legal Translation: Developing Digital Resilience](https://edtechdev.github.io/aied/articles/ai-literacy-legal-translation-2026/) > **Synthesis:** Proposes a four-component AI literacy framework for legal translation professionals: conceptual AI knowledge, technical operational skills, critical evaluation competencies, and ethical governance awareness. Argues generative AI extends rather than replaces professional translation competence. Identifies linguistic, technical, legal, ethical and cognitive risks of AI in legal translation and maps them to literacy components. Framework applicable beyond legal domain to other professional education contexts where AI augments expert judgment. [[ai-literacy]], [[generative-ai]], [[professional-training]], [[human-in-the-loop-ai]], and [[higher-ed]]. Proposes a four-component AI literacy framework for legal translation professionals: conceptual AI knowledge, technical operational skills, critical evaluation competencies, and ethical governance awareness. Argues generative AI extends rather than replaces professional translation competence. Identifies linguistic, technical, legal, ethical and cognitive risks of AI in legal translation and maps them to literacy components. Framework applicable beyond legal domain to other professional education contexts where AI augments expert judgment. ## Connected Concepts - [[ai-literacy]] - [[generative-ai]] - [[professional-training]] - [[human-in-the-loop-ai]] - [[higher-ed]] ## Connected Articles - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) - [[genai-marketing-education-roles-2026]] — When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights - [[genai-literacy-image-discrimination]] — Generative AI Literacy Training Improves Intelligence Analysts’ Discrimination of Real and AI-Generated Images - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm ## Citation Łucja Biel (2026). [AI Literacy for Legal Translation: Developing Digital Resilience](https://arxiv.org/abs/2608.04641). arXiv:2608.04641. Book chapter. --- ## [AI Literacy: An Exercise in Power-Knowledge](https://edtechdev.github.io/aied/articles/ai-literacy-power-knowledge/) Argues existing AI literacy frameworks, dominated by technical competency and responsible-use principles, enforce a consumer orientation toward AI rather than fostering genuine epistemic agency. Draws on Foucault's power-knowledge framework to propose a critical AI literacy that empowers learners to shape and challenge AI systems rather than merely use them. ## Key Findings - The paper argues that dominant AI literacy frameworks — dominated by technical competency and responsible-use principles — enforce a "consumer" orientation toward AI rather than fostering genuine epistemic agency. - Drawing on Foucault's concept of power-knowledge, the authors contend that the absence of power as a construct in AI literacy discourse reflects a deeper conceptual failure: treating AI systems as neutral tools rather than as apparatuses that structure what can be known and by whom. - Grounded in Freire's pedagogy of critical consciousness and scholarship on digital literacy, the paper reconceptualizes AI literacy as a critical practice that equips individuals not just to use AI systems but to critically evaluate them, resist their structuring assumptions, and participate in their governance. - Unequal access to AI tools, the authors argue, recapitulates longstanding epistemic injustices, so a literacy framework oriented toward empowerment must account for these structural inequities. - A three-part framework — contextual use, critical interrogation, and participatory governance — frames AI literacy as the cultivation of epistemic "agents" rather than the training of competent consumers of AI-generated information. ## Theoretical Foundations Generative AI has emerged not only as a new class of technologies but as an infrastructure for the creation and dissemination of knowledge, embedded in search engines, writing tools, research platforms, and industries including education. The authors argue that most institutional approaches reduce complex epistemic and ethical questions to technical proficiency: understanding how these models work, evaluating their outputs for accuracy, and using the tools "responsibly." Against this, the power-knowledge lens asks who can access and shape AI-mediated knowledge production, and how literacy frameworks either reproduce or challenge existing distributions of epistemic authority. ## Implications for AI in Education For education, the argument shifts the goal of AI literacy instruction from competent consumption toward critical participation. Curricula informed by this view would teach learners to interrogate the assumptions built into AI systems, understand how tool access and design encode power, and develop the capacity to shape and challenge the technologies that increasingly mediate learning. This connects [[ai-literacy]] to [[equity]]: if unequal access recapitulates epistemic injustice, then AI literacy programs must attend to structural conditions, not only individual skills, and [[critical-thinking]] becomes a political as well as cognitive capacity. The framework also challenges educators to treat learners as epistemic agents whose critical interrogation of AI is a legitimate and essential part of the curriculum, rather than a distraction from efficient use. ## Connected Concepts - [[ai-literacy]] - [[critical-thinking]] - [[equity]] - [[student-experience]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[k-12-ai-education]] ## Connected Articles - [[possibility-ai-literacy-critical-editorial]] — The (im)possibility of AI literacy - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[genai-architecture-education]] — Gen-AI-tecture: using generative AI to support architectural students in design tasks ## Citation Brady D. Lund, Zoë Abbie Teel (2026). [AI Literacy: An Exercise in Power-Knowledge](https://arxiv.org/abs/2607.27547). arXiv:2607.27547. cs.AI, cs.CY. --- ## [AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes](https://edtechdev.github.io/aied/articles/ai-lms-middle-school-longitudinal/) **Misan Paul Etchie, Taiwo Olutosin** — cs.CY, cs.AI, cs.HC This paper proposes an AI-integrated LMS designed specifically for middle school instruction, addressing the gap between current LMS platforms (which function as workflow tools) and the need for real-time instructional support during the critical middle-school learning window. Key features include policy-gated AI assistance (bounded AI to avoid [[over-reliance]]), formative feedback and hinting during practice, spaced review and adaptive practice recommendations, and teacher dashboards for misconception patterns. The system is designed with privacy-first architecture including data minimization, age-appropriate response constraints, auditable logs, and role-based access control. The proposed longitudinal study tracks students from middle school through high school into post-high school pathways, linking fine-grained learning traces to institutional outcomes. This design study is complementary to [[intelligent-tutoring]] systems research and [[personalized-learning]] implementations in K-12 settings.}, The emphasis on bounded AI support — rather than open-ended tutoring — distinguishes this approach from systems like Khan Academy's Khanmigo and reflects pedagogical concerns about [[scaffolding]] versus answer-giving in K-12 AI tools. ## Connected Concepts - [[over-reliance]] - [[intelligent-tutoring]] - [[personalized-learning]] - [[scaffolding]] ## Connected Articles - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Etchie, M. P., & Olutosin, T. (2026). [*AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes Through High School and Beyond*](https://arxiv.org/abs/2606.07544). arXiv:2606.07544. --- ## [Is AI making us stupid?](https://edtechdev.github.io/aied/articles/ai-making-us-stupid/) ## Summary A 3-page **perspective** (opinion/review, not an empirical study) addressing whether AI use erodes human cognition. The authors' answer: **not inherently — but the risk is real and follows the cognitive-psychology principle of *cognitive offloading*.** When people delegate reasoning, writing, memory, or problem-solving to AI, they forgo the mental practice that builds and maintains those capacities. The threat is **use-dependent**, not intrinsic to the technology: AI that *augments* thinking preserves the underlying processes; AI that *replaces* them outsources exactly the practice that builds expertise. ## Core distinction: skills vs. basic cognitive abilities The article's organizing framework separates two facets of our cognitive systems: - **Skills** — learned, domain-specific behaviors supported by knowledge (arithmetic, flying, diagnosis, writing, programming). Acquired and maintained through **practice** [4]. Offloading practice to AI "will almost certainly compromise skill acquisition" [5] and can cause **skill decay** [5,7]. - **Basic cognitive abilities** — foundational, domain-general capacities (working memory, selective attention) that underlie skills. May be **more resilient** to erosion: cognitive-training research shows gains are highly task-specific rather than broad [9], so basic abilities appear "stubbornly resistant to substantial change." Open question whether long-term or developmental-stage offloading could still shift them [11,12]. ## Evidence cited - **Math learning (high school) [6]:** Students with an AI that let them *fully offload* solving practice problems scored higher on practice but **performed worse on a later no-AI test** than students who never had AI. A *third* condition — a custom AI **tutor** that probed knowledge and filled gaps (rather than giving answers) — performed **no worse** than no-AI students, indicating the targeted skill was still acquired. - **Endoscopy skill decay [8]:** After an AI detection tool was introduced, adenoma detection rates fell from **28.4% → 22.4%** in cases where the AI was unavailable — evidence of deskilling when the tool was withdrawn. - **Knowledge acquisition (Box 1) [2,3]:** Offloading to an external store during learning reduces later retrieval; an AI-summary study found advice was "briefer, less unique, and rated less helpful," with learners spending less time and feeling less ownership — shallower depth of learning than web search. ## The "how we use AI matters" argument Whether skills survive offloading depends on the **form** the offloading takes [13]: - *Completely* offloading the task ("student has the AI submit the answer") → harm. - Having the AI provide an explanation, a suggestion, or act as a **collaborator giving feedback** [14], or emulating a thoughtful tutor [15] → can preserve or even boost skill despite reduced effort ("Coach not crutch" [15]). - Staying "in the proverbial cognitive loop" mitigates costs; costs are likely **limited to the specific skills offloaded**, not basic abilities. ## Concluding remarks & open questions (Box 2) "It is far too early to say with certainty" the long-term effects. Open questions: prolonged offloading over years/decades; developmental-stage effects (children?); decay as a function of initial skill level and skill type; refresher-training strategies; **designing AI to discourage harmful offloading** (policy levers); whether people can learn to *strategically* offload; impacts on **metacognition** and source-monitoring (misattributing AI output as one's own); effects on dispositions toward thinking. Notably, the article **cites the PNAS guardrails paper** (Bastani et al. 2025, [[generative-ai-guardrails-harm-learning]]) and the "Coach not crutch" preprint (Lira et al. 2025) — making the offloading/crutch mechanism a shared thread across the wiki. ## Why this matters for the wiki - The canonical **cognitive-offloading** citation [[cognitive-offloading]] needed; this perspective synthesizes the mechanism and the skills-vs-basic-abilities resilience distinction. - Conceptual bookend to the empirical RCTs already in the wiki: [[generative-ai-guardrails-harm-learning]] (unguarded tutor *cut* exam scores via crutch/offloading), [[generative-ai-reduced-study-time-math]] (population "cognitive surrender"), and contrasts with [[ai-generated-feedback-higher-ed]] (well-architected AI feedback *matched* teachers — the "coach not crutch" design). - Reinforces [[over-reliance]], [[metacognition]] (source-monitoring), [[self-regulated-learning]] (deliberate vs. passive offloading), and [[ai-literacy]] (using AI to augment, not replace). ## Connected Concepts - [[ai-literacy]] - [[metacognition]] - [[self-regulated-learning]] - [[generative-ai]] - [[higher-ed]] - [[rag]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[cognitive-offloading-speedup-illusion]] — Cognitive offloading and the speedup illusion in human-AI interaction - [[efficiency-gain-illusion-ai-overreliance]] — The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks - [[genai-performance-vs-learning]] — Distinguishing performance gains from learning when using generative AI - [[generative-ai-guardrails-harm-learning]] — Generative AI without guardrails can harm learning: Evidence from high school mathematics - [[generative-ai-reduced-study-time-math]] — Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education ## Citation Cash, T. N., Kelly, M. O., Macnamara, B. N., & Risko, E. F. (2026). Is AI making us stupid? [*Trends in Cognitive Sciences*](https://doi.org/10.1016/j.tics.2026.06.004) --- ## [AI Tools Scaffolding Metacognition in STEM](https://edtechdev.github.io/aied/articles/ai-metacognition-stem-review/) > A bibliometric–systematic review of AI tools in STEM education: > Systematic review (2005–2025) mapping how AI tools scaffold and co-regulate metacognitive development in STEM classrooms through bibliometric and qualitative synthesis. ## Scope and Methodology **Tsakeni et al. (2025)** conducted a **bibliometric–systematic review** of AI tools in STEM education: - **Dataset:** 135 peer-reviewed articles (2005–2025) from Scopus and Web of Science - **Core studies:** 24 studies selected via PRISMA 2020 protocols for in-depth review - **Methods:** Bibliometric mapping (Biblioshiny, VOSviewer) + qualitative thematic synthesis - **Theoretical frameworks:** Flavell's metacognition theory, General Systems Theory, Human-Centered vs. Posthumanist paradigms ## Key Findings ### 1. Theoretical Frameworks Evolution | Paradigm | Core Idea | AI Role | |-----------|-----------|---------| | **Human-Centered** (traditional) | AI supports human agency with ethical oversight | Supportive tool under teacher control | | **Posthumanist** (emerging) | Learning = co-regulated process distributed between humans and AI | AI as co-agent in reflective processes | **Finding:** While most research remains grounded in **human-centered conceptualizations**, there are emerging indications of **posthumanist framings** where AI systems are positioned as **co-regulators of learning**. ### 2. AI Tools Scaffolding Metacognition | Tool Category | Examples | Metacognitive Function | |--------------|---------|----------------------| | **Intelligent Tutoring Systems (ITS)** | Carnegie Learning, ALEKS | Personalized feedback, real-time monitoring, strategic prompts | | **Adaptive Platforms** | Deep RL-based systems | Enhance metacognitive outcomes via adaptive scaffolds | | **Learning Analytics** | Dashboards, tracking tools | Externalize metacognitive processes, support reflection | | **Generative AI** | ChatGPT (GPT-4.1, 4.5, 5), conversational agents | Metacognitive prompts, chatbot feedback, ME-CoT approaches | **Core shift:** From **individual reflection** → **system-level regulation and distributed cognition**. ### 3. Metacognitive Outcomes Studies consistently show AI tools improve: - **Reflective thinking** and independent inquiry - **Self-regulation** and strategic monitoring (planning, monitoring, evaluation) - **Motivation and reasoning** (especially for low-achieving and underrepresented groups) - **Academic performance** in mathematics, science, chemistry, physics ### 4. Ethical Concerns and Pitfalls Despite benefits, scholars warn of: - **Cognitive overload** from excessive AI-generated feedback - **Reduced learner autonomy** when AI replaces (not scaffolds) reflection - **Algorithmic bias** if AI prioritizes automation over deep reflection - **Inconsistent measurement** methods across studies (theoretical fragmentation) ## Connection to Existing Work ### vs. [[metacognition|Metacognition in AI Education]] - This review **maps the field systematically** (135 publications, 24 core studies) - Confirms: metacognition is **central to STEM success** but challenging to foster - AI tools offer scalable scaffolding, but require **teacher preparation in AI literacy** ### vs. [[adaptive-learning|Adaptive Learning Systems]] - **ALEKS, Carnegie Learning** highlighted as successful adaptive platforms for metacognition - **Deep reinforcement learning** enhances metacognitive outcomes (vs. static adaptive rules) - Aligns with: system-level regulation > individual reflection ### vs. [[intelligent-tutoring|Intelligent Tutoring Systems]] - ITS identified as **key scaffolding tool** for metacognitive development - Combines **content mastery** (object level) with **reflective monitoring** (meta level) - Consistent with: [[tutoring-specific-vs-general-ai]] — domain-specific tutoring outperforms generic chatbots ### vs. [[ai-literacy|AI Literacy]] - **Critical finding:** AI literacy must be **integrated into teacher preparation** - Teachers need skills to: select tools, interpret analytics, maintain ethical oversight - Aligns with: human-centered paradigm (AI as tool, teacher as agentic decision-maker) ## The Posthumanist Turn The review identifies an **emerging paradigm shift**: **Human-Centered (Traditional):** ``` Teacher → AI Tool → Student (AI as instrument) ``` **Posthumanist (Emerging):** ``` Teacher ↔ AI System ↔ Student (Learning as distributed, co-regulated process) ``` **Implications:** - AI systems as **co-regulators** (not just tools) - **System-level effects** (not just individual cognition) - Challenges the **human–machine binary** in education ## Implications for AI in Education ### For Researchers - **Theoretical integration needed:** Flavell + General Systems Theory + posthumanist perspectives - **Measurement standardization:** Inconsistent methods hinder meta-analysis - **Longitudinal studies:** Most research is cross-sectional; need to track long-term metacognitive development ### For Educators - **AI literacy is prerequisite:** Teachers must understand metacognitive scaffolding to select tools effectively - **Balance automation with reflection:** AI should enhance (not replace) metacognitive processes - **Leverage multiple tools:** ITS + learning analytics + generative AI for comprehensive scaffolding ### For Tool Developers - **Ethical design priority:** Avoid cognitive overload and reduced autonomy - **Metacognitive scaffolds:** Build in reflection prompts, self-monitoring dashboards - **Transparency:** Teachers need to understand AI decision-making to maintain pedagogical agency ## Connected Concepts - [[metacognition]] - [[adaptive-learning]] - [[intelligent-tutoring]] - [[ai-literacy]] ## Connected Articles - [[tutoring-specific-vs-general-ai]] ## Citation Tsakeni, M., Nwafor, S. C., Mosia, M., & Egara, F. O. (2025). [*Mapping the Scaffolding of Metacognition and Learning by AI Tools in STEM Classrooms: A Bibliometric–Systematic Review Approach*](https://doi.org/10.3390/jintelligence13110148). *Journal of Intelligence*. --- ## [AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation](https://edtechdev.github.io/aied/articles/ai-partner-science-epistemic-vigilance/) Argues that epistemic vigilance — the human evaluation of AI output calibrated to how far a fallible source can be trusted — is the binding constraint on productive augmentation. AI's fluent, confident prose reads as trustworthy whether or not it is, making evaluation harder. Vigilance sets how deeply a claim is processed and is thus the precondition for learning with AI. Design factors (prompts, feedback, scaffolding) matter only through whether they engage the learner's evaluation. Because vigilance is unevenly distributed, uniform AI integration risks widening achievement gaps. ## Key Findings - The paper identifies epistemic vigilance — the human evaluation of AI output calibrated to how far a fallible source can be trusted — as, given adequate prior knowledge, the binding constraint on productive augmentation in learning with AI. - The AI partnership takes three forms — the scientist working with a co-scientist, a member of the public checking a claim such as whether a diet works or whether to fit solar panels, and a student taking up an inquiry with AI inside a science class — and in all three the deciding factor is whether the human evaluates what the AI returns or takes it on trust. - Vigilance is what licenses augmentation: because the human stays vigilant, generation, retrieval, and drafting can be delegated safely, so vigilance expands rather than restricts what can be handed to the AI. - The AI case is distinctive because the machine's fluent, confident prose reads as trustworthy whether or not it is, so the default surface of the output works against the human doing the evaluating. - Vigilance sets how deeply a claim is processed, making calibrated vigilance the precondition for productive learning with AI; design factors such as prompts, feedback, and scaffolding matter through whether they engage the learner's evaluation, and none works around it. - Candidates that might seem to make vigilance dispensable — the learner's own content knowledge, a neighboring competence, or a more trustworthy AI — do not remove the need for it. - Because the disposition to evaluate is unevenly distributed, integrating AI uniformly across a classroom is likely to widen achievement gaps. ## The Argument The paper specifies the components of vigilance, the mechanism that ties it to learning, and a way to measure it without soliciting the very evaluation it is meant to detect. It also distinguishes judging from producing: each capacity is built by exercising it, so what is handed over to the AI is never the exercise the lesson exists to provide — a learner who evaluates a derivation deeply is practicing judgment, not derivation. Existing evidence anchors the processing-depth half of the claim; what remains untested is vigilance as a measured disposition, above all in the regime where the AI is confidently wrong. ## Implications for AI in Education For science education, the argument reorients design: the many factors reported as shaping AI's effect — prompts, feedback, scaffolding — succeed only insofar as they engage the learner's evaluation of AI output, making [[critical-thinking]] and [[hallucination-risk]] awareness central to instructional design rather than peripheral. The equity warning is direct: since vigilance is unevenly distributed, uniform AI integration risks widening achievement gaps, so [[scaffolding]] and differentiated support must target the disposition to evaluate, not just tool access, and this bears on [[equity]] in who benefits from AI-augmented learning. Finally, the judging-versus-producing distinction gives educators a principled rule for dividing labor between learner and AI: whatever the lesson exists to teach must stay with the learner. ## Connected Concepts - [[pedagogical-agent]] - [[affective-computing]] - [[self-regulated-learning]] - [[personalized-learning]] - [[affective-tutoring]] - [[administrator]] - [[hallucination-risk]] - [[teacher-ai-competency]] ## Connected Articles - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[edtech-design-time-generative-ui]] — The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[shame-guilt-ai-regulation-computing-education]] — Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact ## Citation Marcus Kubsch (2026). [AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation](https://arxiv.org/abs/2606.16822). arXiv:2606.16822. arXiv preprint (physics.ed-ph). --- ## [Design Principles and Observable Indicators for AI-Enabled Pedagogical Accompaniment: Evidence from the Amico Dual-Mode Prototype in Italy and China](https://edtechdev.github.io/aied/articles/ai-pedagogical-accompaniment-amico/) Benedetti (2026) introduces a theoretically grounded framework for AI-enabled pedagogical accompaniment that explicitly centers human agency — an approach described as "human-in-command" rather than merely human-in-the-loop. The Amico prototype embodies five design principles: transparency of system identity and limits, scaffolding toward human contact, maieutic questioning, prevention of dependency dynamics, and data minimization. Each principle is mapped to observable indicators suitable for real educational settings. The dual-mode design — AmicoMio for structured, task-oriented interaction and AmicoTuo for reflective, supportive engagement — represents a novel contribution to [[intelligent-tutoring]] architecture. Rather than pursuing a single interaction style, the system adapts its mode to the pedagogical context. This aligns with [[scaffolding]] theory's emphasis on calibrating support to learner needs and connects to recent work on [[ai-tutor-behavioral-evaluation]] that stresses context-sensitive deployment. The cross-context pilot in Italy and China provides initial evidence of feasibility in vocational education, an underserved domain in the [[stanford-evidence-base-ai-k12-2026]]. The framework's emphasis on AI as a "relational bridge" to human interaction — not a replacement — addresses concerns raised in [[ai-tutor-safety-harms]] about dependency and [[over-reliance]]. The principle of data minimization further connects to [[privacy]] and [[equity]] discussions in educational AI deployment. For [[faculty-development]], the observable indicators offer concrete assessment tools for evaluating AI integration quality. ## Connected Concepts - [[intelligent-tutoring]] - [[scaffolding]] - [[over-reliance]] - [[privacy]] - [[equity]] - [[faculty-development]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-safety-harms]] ## Citation Pier Paolo Benedetti (2026). [Design Principles and Observable Indicators for AI-Enabled Pedagogical Accompaniment: Evidence from the Amico Dual-Mode Prototype in Italy and China](https://arxiv.org/abs/2605.20665). arXiv:2605.20665. arXiv:2605.20665 [cs.HC] — Accepted at ICAIE 2026. --- ## [Faculty Orientations Shape Adoption of AI in Research and Teaching](https://edtechdev.github.io/aied/articles/ai-pedagogical-orientation/) 📄 arXiv · [PDF](https://arxiv.org/pdf/2605.18140) ## Key Finding: AI Pedagogical Orientation A mixed-methods survey of 90 STEM faculty in the RCSA Cottrell community identified a coherent latent construct — **AI pedagogical orientation** — that strongly predicts AI adoption across research, teaching, and professional activities. This orientation reflects deep beliefs about AI's role in disciplinary thinking, learning, and expertise development — not merely positive or negative sentiment. ## Why This Matters Current technology-adoption models ([[scaffolding]]-like frameworks such as DoI and UTAUT) emphasize external conditions, perceived usefulness, and social influence. This study demonstrates that for AI in higher education, these factors are surprisingly weak predictors. Instead, a faculty member's **epistemic interpretation** of AI — their stance on what AI means for disciplinary knowledge production — is the primary driver. ## Study Details - **Population:** 90 STEM faculty (from 572 RCSA Cottrell awardees, 16% response rate), Fall 2025 - **Method:** Mixed-methods survey → Exploratory Factor Analysis (36 variables) + qualitative coding - **Key result:** One dominant factor (AI pedagogical orientation, 9 items) consistently predicted AI use - **Weak predictors:** Institutional initiatives, demographics (discipline, career stage, institution type), information sources - **Information flow:** Department colleagues (64%) and discipline-specific news were top sources; institutional support mechanisms were less used ## Connections to Wiki ### Faculty Development This paper challenges the implicit theory behind many [[faculty-development]] programs: providing tools, workshops, and institutional support may be insufficient if faculty have not developed a coherent pedagogical orientation toward AI. The Cottrell community is already pedagogically motivated — yet 22% used no AI at all. Faculty development must help instructors articulate *what AI means for their discipline*, not just *how to use it*. ### Teacher Adoption The finding that concerns do not moderate adoption contrasts with [[teacher-ai-adoption-confidence]], where institutional support → confidence → attitudes. This paper suggests a different mechanism: orientation shapes adoption directly, without being mediated by confidence or moderated by concerns. The [[teacher-role]] literature may need to incorporate epistemic dimensions alongside attitudinal ones. ### Institutional Change The weak association between institutional initiatives and AI use is a cautionary note for [[institutional-change-framework-ai]] and change-management: top-down strategic plans and degree programs may have limited impact if they don't engage with faculty pedagogical orientations. Bottom-up, colleague-driven information flow (64% cited department colleagues) suggests peer networks are more influential than central initiatives. ### AI Literacy The orientation construct connects to [[ai-literacy]] at a deeper level: it's not just about knowing what AI can do, but having a coherent philosophy about what AI *should* do in one's discipline. This aligns with the distinction between instrumental and critical AI literacy. ### Comparison to Computation Integration The authors draw a deliberate parallel to the integration of computation into physics education — a decades-long process that succeeded only when faculty communities (e.g., PICUP) developed shared pedagogical frameworks. AI may follow a similar trajectory, where disciplinary communities — not institutions — drive adoption through shared epistemic norms. ## Qualitative Dimensions of Orientation Faculty views clustered around three questions: 1. **Disciplinary thinking:** Should students learn to think with AI, or without it? 2. **Learning:** Is AI a prosthetic that extends learning, or a replacement that short-circuits it? 3. **Expertise development:** What does expertise mean when AI performs tasks once central to mastery? These are not resolvable by more information or better tools — they require disciplinary conversation and pedagogical judgment. ## Open Questions - How stable is AI pedagogical orientation over time? Does it change with experience? - Are there discipline-specific differences in the content of orientations (physics vs. chemistry vs. biology)? - Can faculty development interventions shift orientation, or is it a stable trait? - How does orientation relate to actual classroom practice (not just self-reported use)? ## Connected Concepts - [[scaffolding]] - [[faculty-development]] - [[teacher-role]] - [[ai-literacy]] ## Connected Articles - [[teacher-ai-adoption-confidence]] - [[institutional-change-framework-ai]] ## Citation Atherton, T. J., Descamps, I., Holmes, T. R., Vizcarra, C. L., Sui, N., Webel, M., & Foley, J. J., IV. (2026). [Faculty orientations shape adoption of AI in research and teaching](https://arxiv.org/abs/2605.18140). arXiv:2605.18140. --- ## [AI Peer Feedback Systems](https://edtechdev.github.io/aied/articles/ai-peer-feedback-systems/) > Peer feedback develops critical reflection and evaluative judgment, yet: > Student peer feedback is often superficial or inconsistent. **AICoFe** (AI-based Collaborative Feedback) uses a multi-LLM pipeline (GPT-4.1-mini, Gemini 2.5 Flash, Llama 3.1) to generate diverse perspectives on student presentations, but the critical design feature is **teacher-in-the-loop mediation**—educators curate and compose final feedback before delivery.^[[becerra-aicofe-feedback-2026]] ## The Problem Peer feedback develops critical reflection and evaluative judgment, yet: - Students have limited **feedback literacy** - Generated comments are often superficial or lack actionability - Quality is inconsistent across evaluators AI-generated feedback alone risks replacing shallow peer comments with shallow AI comments. AICoFe addresses this by treating AI as a **draft generator**, not a final deliverer. ## System Architecture **Four components:**^[[becerra-aicofe-feedback-2026]] | Module | Purpose | |--------|---------| | **Visualization** | Role-specific dashboards (student, teacher, evaluator) | | **Management** | Hybrid SQL + MongoDB data infrastructure | | **Feedback Generation** | Multi-LLM pipeline synthesizing rubric scores + qualitative comments | | **Recording** | Video/audio capture of student presentations (opt-in, GDPR-compliant) | ## The Multi-LLM Pipeline Three independently fine-tuned models receive: - Quantitative rubric scores - Validated qualitative observations from evaluators - Rubric level descriptions - Instructional materials for the assessed skill Each model produces an independent draft. The diversity of models is intentional—GPT, Gemini, and Llama have different stylistic biases and blind spots; teacher curation selects the best fragments.^[[becerra-aicofe-feedback-2026]] ## Teacher-in-the-Loop Mediation The Teacher Dashboard is the central mediation interface: - **Inspect** scores, comments, and videos from all evaluators - **Review** drafts from all three LLMs - **Compose** final feedback by selecting individual sentences or paragraphs from AI outputs - **Visual legend** shows proportion of content contributed by each LLM - **Track** extent of teacher modification/curation - **Audio review** via text-to-speech for long comments This preserves pedagogical authority while reducing teacher workload.^[[becerra-aicofe-feedback-2026]] ## Transparency & Analytics - **Feedback history log:** Every sent feedback entry displays LLM contribution proportions and teacher modification levels - **Dual purpose:** Supports (1) teacher reflection on curation patterns and (2) large-scale analysis of how teacher mediation shapes feedback quality ## Student Experience - View video recordings of own presentations - Complete self-evaluations using the same rubric - See visual comparisons of self vs. external evaluations - Receive **teacher-curated** (not raw AI) feedback - Rate perceived agreement and usefulness^[[becerra-aicofe-feedback-2026]] ## Relationship to Human-in-the-Loop AI AICoFe represents a **human-centered AI** paradigm for education: AI augments rather than replaces human judgment. This contrasts with fully automated grading or feedback systems that remove the teacher from the loop. The key insight is that **pedagogical authority resides with the teacher**, and AI's role is to expand the range and depth of actionable comments they can craft.^[[becerra-aicofe-feedback-2026]] ## Connected Concepts - [[ai-ed-evaluation]] - [[metacognition]] - [[self-regulated-learning]] - [[socratic-method]] - [[ai-education]] - [[human-in-the-loop-ai]] - [[administrator]] - [[teacher-ai-competency]] ## Connected Articles - [[becerra-aicofe-feedback-2026]] - [[aicode-collaborative-feedback-system]] - [[codify-socratic-programming-tutor]] - [[humanlike-ai-collaborative-writing]] - [[llm-reasoning-traces-metacognition]] - [[mindcopilot-llm-co-writing]] - [[moodle-ai-tutoring-deep-learning]] - [[multimodal-ai-feedback-learning]] - [[psyscore-essay-scoring-zpd-feedback]] - [[sequenced-ai-feedback-learning]] ## Citation Becerra, Á., Palma, A., & Cobos, R. (2026). [*AICoFe: AI-Based Collaborative Feedback System for Higher Education*](https://arxiv.org/abs/2605.04740). arXiv:2605.04740. --- ## [Preparing Students for AI-Powered Materials Discovery: A Workflow-Aligned Framework for AI Literacy, Equity, and Scientific Judgment](https://edtechdev.github.io/aied/articles/ai-powered-materials-discovery-ai-literacy/) # AI-Powered Materials Discovery and AI Literacy ## Key Findings This paper presents a workflow-aligned framework for preparing students to use AI in materials discovery. The authors argue that in materials science, the limiting factor is no longer only algorithmic capability but **human-AI collaboration competence**. Students need to develop scientific judgment about when to trust AI predictions and how to integrate them into research workflows. The framework connects AI literacy to equity and scientific judgment, emphasizing that students from under-resourced backgrounds may lack exposure to AI-powered research workflows. This intersects with broader [[equity-in-ai-education]] and [[stem-education]] concerns. ## Connections to AIED The materials discovery context offers a model for how AI literacy should be taught across STEM disciplines: not as abstract knowledge, but as embedded workflow competence. This connects to [[agentic-workflows-education]] where students learn to use AI tools as part of authentic research practice rather than as separate subjects. The framework also touches on [[metacognition]] — students need to develop judgment about AI outputs, which requires meta-awareness of their own reasoning processes when evaluating AI-generated predictions. ## Connected Concepts - [[equity-in-ai-education]] - [[stem-education]] - [[metacognition]] ## Connected Articles - [[agentic-workflows-education]] ## Citation Mei, D., Moore, K., & Sayler, B. (2026). [Preparing students for AI-powered materials discovery: A workflow-aligned framework for AI literacy, equity, and scientific judgment](https://arxiv.org/abs/2605.09624). arXiv:2605.09624. --- ## [Exploring Fraction Comprehension and Interest in Elementary Education Through AI-Powered Personalized Learning](https://edtechdev.github.io/aied/articles/ai-powered-personalized-learning-elementary-fractions-2026/) > **Synthesis:** Examines AI-powered personalized learning in elementary fraction instruction through a systematic review, quantitative study (N=120), and qualitative teacher interviews. Found that AI-adaptive platforms significantly improved fraction comprehension for students with math learning difficulties compared to traditional instruction. AI personalization increased student interest and engagement. Teachers reported AI tools helped differentiate instruction but required professional development for effective integration. [[adaptive-learning]], [[personalized-learning]], [[k-12]], [[math-education]], and [[generative-ai]]. Examines AI-powered personalized learning in elementary fraction instruction through a systematic review, quantitative study (N=120), and qualitative teacher interviews. Found that AI-adaptive platforms significantly improved fraction comprehension for students with math learning difficulties compared to traditional instruction. AI personalization increased student interest and engagement. Teachers reported AI tools helped differentiate instruction but required professional development for effective integration. ## Connected Concepts - [[math-education]] - [[adaptive-learning]] - [[personalized-learning]] - [[k-12]] - [[generative-ai]] - [[student-experience]] ## Connected Articles - [[epistemic-proactivity-math]] - [[kar-mathbuddy-affective-math-tutoring-2025]] ## Citation Kenneth Holman (2024). [Exploring Fraction Comprehension and Interest in Elementary Education Through AI-Powered Personalized Learning](https://arxiv.org/abs/2608.04892). arXiv:2608.04892. PhD dissertation, University of Central Florida. --- ## [Position: Adopting AI in Practice Does Not Guarantee the Productivity Boost](https://edtechdev.github.io/aied/articles/ai-productivity-moderation/) # AI Productivity Moderation Factors This ICML 2026 position paper argues that adopting AI in organizational practice does not automatically yield productivity gains — human and environmental factors critically moderate the relationship. Drawing on the partial equilibrium model of Gries and Naudé (2022), it identifies five key moderators that can attenuate or negate productivity benefits. ## Five Moderating Factors 1. **Human resource composition** — the mix of skills, roles, and experience in the workforce 2. **Baseline capability of individuals** — pre-existing competence before AI introduction 3. **Learning curve of practitioners** — how quickly users adapt to AI tools 4. **Incentives for fair use** — motivations for appropriate and ethical AI usage 5. **Flexibility of objectives** — organizational ability to adjust goals with AI integration ## Relevance to AI in Education While framed broadly, the implications for [[teacher-role|teachers]] and educational institutions are direct. The learning curve factor is particularly salient: if practitioners cannot adapt quickly, AI tools may reduce rather than increase productivity — a finding that echoes [[cognitive-offloading-speedup-illusion|the cognitive offloading speedup illusion]] where users overestimate AI's time-saving benefits. The incentives factor connects to concerns in [[academic-integrity]] — if students and teachers lack incentives for fair AI use, productivity gains may be captured by surface-level efficiency while deeper learning suffers. The framework complements [[genai-assessment-governance|GenAI assessment governance]] by highlighting that technical capability alone is insufficient without attention to the human systems within which AI is deployed. For [[higher-ed]] institutions investing in AI infrastructure, this paper serves as a cautionary reminder that deployment without attention to training, incentives, and organizational readiness may fall short of promised gains — a finding consistent with [[ai-fatigue-academic-contexts|research on AI fatigue]] among students. ## Connected Concepts - [[teacher-role]] - [[academic-integrity]] - [[higher-ed]] ## Connected Articles - [[cognitive-offloading-speedup-illusion]] - [[genai-assessment-governance]] - [[ai-fatigue-academic-contexts]] ## Citation Cho, W. I., Kim, S., & Kim, G. (2026). [Position: Adopting AI in practice does not guarantee the productivity boost](https://arxiv.org/abs/2605.24688). arXiv:2605.24688. Accepted at ICML 2026. cs.CY. --- ## [Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars](https://edtechdev.github.io/aied/articles/ai-psychotherapy-training-avatars/) AI-driven interactive patient avatars for psychotherapy training provide accessible, repeatable practice with measurable skill improvement in evidence-based therapy techniques. ## Synthesis Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars investigates ai-driven interactive patient avatars for psychotherapy training provide accessible, repeatable practice with measurable skill improvement in evidence-based therapy techniques. This work connects to existing research on accessible-learning by demonstrating that Abstract:Training psychotherapists in evidence-based interventions such as Acceptance and Commitment Therapy (ACT) requires repeated practice with meaningful feedback, yet opportunities for safe, standardized training are limited by ethical, logistical, and resource constraints. We introduce a syste.... ## Connected Concepts - [[socratic-ai-dialogue]] - [[lifelong-learning]] - [[ai-tutoring]] - [[prompt-engineering]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[affective-tutoring]] - [[automated-essay-scoring]] ## Connected Articles - [[medeasy-ai-standardized-patients]] — MedEasy: Designing AI Standardized Patients for Clinical Consultation Training - [[flowcode-ai-creative-coding]] — Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education - [[supplynet-visual-exploratory-learning]] — SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation - [[astra-atco-training-simulator]] — ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training ## Citation Pascal Riachi, Sofie Kamber, Stella Brogna, Andrew Gloster, Rafael Wampfler (2026). [Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars](https://arxiv.org/abs/2606.17786). arXiv:2606.17786. --- ## [AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics](https://edtechdev.github.io/aied/articles/ai-scoring-language-bias-physics/) > **Authors:** Markus S. Feser, Paul L. Tschisgale (Leibniz Institute for Science and Mathematics Education, Kiel, Germany) **Source:** arXiv:2607.28210 (physics.ed-ph, July 2026) ## Key Findings This study examined whether AI-based scoring can assess students' conceptual understanding **independently of the linguistic quality** of their text-based explanations in physics. The researchers compared scores from **9 machine learning (ML) approaches and 2 large language model (LLM) approaches** against human expert scores for 116 secondary-school students' physics explanations. ### The Language Bias Problem - **Systematic underestimation**: Explanations of lower linguistic quality were systematically more likely to receive lower AI-generated conceptual understanding scores than experts assigned — a bias that emerged across **every single AI-based scoring approach** tested. - **Asymmetric effect**: Higher linguistic quality showed **no comparable link** to overestimation. The bias only runs in one direction: linguistically weak students are penalized, but linguistically strong students are not inflated. - **Mirrors human bias**: This language bias closely resembles that previously reported for **physics teachers** themselves (Feser & Höttecke, 2021), suggesting the difficulty lies less in any particular assessor (human or machine) than in the fundamental nature of inferring conceptual understanding from text-based explanations. ### Disproportionate Impact The stakes fall hardest on **multilingual learners**, whose language proficiency may be misread as weaker conceptual understanding. This is especially concerning as AI-based scoring takes on higher-stakes assessment decisions. ## Relevance to AI in Education This paper makes a critical contribution to the [[automated-assessment]] and [[automated-essay-scoring]] literature by demonstrating that the [[bias-mitigation]] problem in AI scoring is not merely a technical artifact of specific models but appears to be **fundamental to the task itself**. Key connections: - **Not just an AI problem**: The finding that the same [[bias-mitigation]] pattern exists in human physics teachers challenges the assumption that AI introduces unique biases. Instead, it suggests a deeper [[assessment-validity]] challenge: distinguishing conceptual understanding from linguistic expression in text-based explanations is inherently difficult. - **[[equity-in-ai-education]]**: The asymmetric nature of the bias (underestimation without compensating overestimation) means linguistically weak students are systematically disadvantaged, with no offsetting benefit for strong writers. This raises serious [[equity]] concerns as [[automated-grading]] scales. - **[[language-learning]]**: The finding that multilingual learners are disproportionately affected connects directly to broader concerns about AI tools in linguistically diverse classrooms. See also [[language-learning]] and [[genai-linguistic-diversity-academic-writing]]. - **[[automatic-short-answer-grading]]**: The study's comparison of ML and LLM approaches provides evidence that newer LLM-based systems are not immune to this bias — it persists across both paradigms. See related work on [[short-answer-scoring-quality-degradation]]. - **[[ai-assessment-human-tutors]]**: The parallel between AI and human scoring bias suggests that the goal should not be to replace human assessment but to understand the limitations shared by both, as also explored in the [[ai-feedback-quality]] literature. - **[[educational-measurement]]**: The concept of construct-irrelevant variance — where language proficiency contaminates measurement of conceptual understanding — is a fundamental [[assessment-validity]] concern that this study quantifies across AI systems. ## Implications 1. **Benchmarking AI scoring**: AI-based scoring systems should be explicitly evaluated for language bias, not just overall agreement with human scores. 2. **High-stakes caution**: As AI scoring moves toward higher-stakes decisions, the asymmetric language bias becomes increasingly consequential. 3. **Multimodal assessment**: The findings support calls for assessment approaches that reduce dependence on linguistic production, particularly for [[language-learning]] populations. 4. **Teacher-AI collaboration**: Rather than replacing teacher assessment, AI scoring may be most useful when teachers remain in the loop to calibrate for language effects. ## Connected Concepts - [[automated-assessment]] - [[automated-essay-scoring]] - [[bias-mitigation]] - [[assessment-validity]] - [[equity-in-ai-education]] - [[equity]] - [[automated-grading]] - [[language-learning]] - [[ai-feedback-quality]] - [[educational-measurement]] - [[physics-education]] ## Connected Articles - [[genai-linguistic-diversity-academic-writing]] - [[automatic-short-answer-grading]] - [[short-answer-scoring-quality-degradation]] - [[ai-assessment-human-tutors]] - [[icle-plus-plus-essay-scoring]] ## Citation Tschisgale", A.S.F.P.L., Leibniz, P.E., to, E.A.S.A.H.L.Q.S.N.C.L., & frequently, T.C.S.I.P.T.A.P.P.T. (2026). [*AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics*](https://arxiv.org/abs/2607.28210) --- ## [Why does AI unlock new possibilities in STEM education? A Bibliometric Analysis of Trends and Future Agenda](https://edtechdev.github.io/aied/articles/ai-stem-bibliometric-trends/) STEM education faces challenges in personalization and interdisciplinary integration. AI technology has brought new possibilities, but the mechanisms by which AI reshapes the STEM education ecosystem require systematic investigation. This study employs bibliometric methods to analyze 242 publications from 2015-2025, constructing knowledge maps to reveal the evolutionary trajectory. The findings show that the field has transformed from intelligent tutoring systems to inquiry-based learning and computational thinking cultivation driven by LLMs. AI's key contribution lies in providing intelligent scaffolding that lowers the threshold for understanding knowledge. In this sense, AI is a core driving force promoting its shift from knowledge transmission to capability development. Bibliometric analysis of 242 STEM-education papers (2015-2025) shows the field shifted from classic intelligent tutoring systems toward LLM-driven inquiry-based learning and computational-thinking cultivation; AI's main value is intelligent scaffolding that lowers the knowledge threshold and moves STEM from knowledge transmission to capability development. This work connects to core wiki themes: [[stem-education]] [[intelligent-tutoring]] [[scaffolding]] [[generative-ai]] [[adaptive-learning]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[stem-education]] - [[intelligent-tutoring]] - [[scaffolding]] - [[generative-ai]] - [[adaptive-learning]] ## Connected Articles - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[codify-socratic-programming-tutor]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation ## Citation Chan, Chen, Hong, Song, Wang & Xu (2026). [Why does AI unlock new possibilities in STEM education? A Bibliometric Analysis of Trends and Future Agenda](https://arxiv.org/abs/2607.05412). arXiv:2607.05412. ISLS 2026 (arXiv preprint). --- ## [AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design](https://edtechdev.github.io/aied/articles/ai-team-teaching-talk-analytics/) **Yuchen Liu, Roberto Martinez-Maldonado, Riordan Alfredo, Paola Mejia-Domenzain, Dwi Rahayu, Sadia Nawaz** — AIED 2026 — cs.HC, cs.AI This paper presents an AI-based speech processing approach to analyze classroom talk in team-teaching settings, grounded in spatial pedagogy theory. Analyzing 36 recorded undergraduate and postgraduate sessions involving 12 teachers, the study extracts acoustic features (voice quality, intonation, loudness) and codes spatial pedagogy behaviors. Results reveal systematic differences most notably in loudness dynamics: high-experience teachers, undergraduate classes, and collaborative learning tasks exhibited greater loudness variation, suggesting more frequent modulation of volume to foreground key information and support engagement. This is the first large-scale automated analysis of acoustic patterns in team-teaching, demonstrating that AI can scalably capture meaningful teaching-talk features across experience, cohort, and task design. The work contributes to [[learning-analytics]] by extending classroom sensing beyond student-focused clickstream data to teacher vocal behavior, and to [[teacher-role]] research by showing how AI can provide formative feedback on teaching practice. ## Connected Concepts - [[learning-analytics]] - [[teacher-role]] ## Connected Articles - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[cold-start-knowledge-tracing-safeinsights]] — Reexamining the Cold-Start Problem in Knowledge Tracing Models and Implications for SafeInsights - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation Liu, Y., Martinez-Maldonado, R., Alfredo, R., Mejia-Domenzain, P., Rahayu, D., & Nawaz, S. (2026). [*AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design*](https://arxiv.org/abs/2606.09831). arXiv:2606.09831. --- ## [AI tools in Arab University English classrooms: Looking back and forward](https://edtechdev.github.io/aied/articles/ai-tools-arab-english-classrooms/) This paper aims to synthesize empirical research on AI tools used to support English as a second/foreign language (EL2) learners in Arab University classrooms (AUCs) between Jan 1st 2023 and Aug 31st 2025. We utilized 3 large datasets, namely Google Scholar, Web of Science, and Scopus as the data sources. Using PRISMA-guided searches across these well-known databases, we included only published articles. The search process results in 184 studies, but only 11 studies met the inclusion criteria. Findings unveil that EL2 learners have positive attitudes towards AI for drafting, revision, and practice. Empirical gains were most consistent for surface-level outcomes improvements in higher-order writing quality and speaking proficiency was mixed and often contingent on teacher mediation. The paper concludes by proposing a research agenda and practical guidelines for Arab universities seeking evidence-based AI integration in EL2 instruction. It also recommends scaffolded integration, teacher training, reflective tasks to reduce over-reliance on AI tools. PRISMA review of 11 EL2 studies (2023-2025) finds Arab university students react positively to AI for drafting/revision/practice; gains are clearest on surface-level writing, mixed for higher-order quality and speaking, and teacher mediation matters; recommends scaffolding, training, and reflective tasks to curb over-reliance. This work connects to core wiki themes: [[language-learning]] [[higher-ed]] [[writing-education]] [[ai-literacy]] [[over-reliance]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[language-learning]] - [[higher-ed]] - [[writing-education]] - [[ai-literacy]] - [[over-reliance]] ## Connected Articles - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative artificial intelligence ## Citation Shormani & Alshawsh (2026). [AI tools in Arab University English classrooms: Looking back and forward](https://arxiv.org/abs/2607.05403). arXiv:2607.05403. arXiv preprint. --- ## [AI literacy-related domains and AI-TPACK readiness among preservice mathematics teachers: A factor-informed structural equation modelling study](https://edtechdev.github.io/aied/articles/ai-tpack-preservice-math-teachers/) > **Synthesis:** AI literacy-related domains and AI-TPACK readiness among preservice mathematics teachers: A factor-informed structural equation modelling study ## Key Findings - In an exploratory CFA/SEM study of 130 preservice mathematics teachers at a South African public university, the AI-TPACK readiness items were essentially unidimensional; one weak design-confidence item was removed, and the refined seven-item measurement model fitted better than the original eight-item specification. - The removed item — confidence in designing AI-supported geometry lessons — had an EFA loading of .294 and communality of .087, indicating it shared too little variance with the broader readiness factor; the refined measurement model fit well, χ2(474) = 601.36, p < .001, CFI = .991, TLI = .990, RMSEA = .046, SRMR = .082. - The primary gender-controlled latent SEM (n=129) showed good approximate fit (CFI = .981, TLI = .984, RMSEA = .049, SRMR = .082; χ2(602) = 789.92, p < .001) and explained 53.0% of the variance in AI-TPACK readiness. - Positive associations with readiness were observed for prior AI use, critical-ethical appraisal, and support/enablers; the support/enablers path had the largest standardized coefficient (β = .725, p = .024) but must be interpreted cautiously because the construct had marginal AVE (.478) and overlapped with information-source engagement (r = .748). - Prior AI use (β = .327, p < .001) and critical-ethical appraisal (β = .251, p = .016) were the other significant positive paths; AI awareness, information-source engagement, tool familiarity, contextual barriers, prior AI training, and gender did not retain significant independent paths in the joint model. - Year level was significant in the primary model (β = .199, p = .041) but less stable in sensitivity analysis, so claims about progression across cohorts remain provisional; a merged "AI-literacy core" model explained less variance (R² = .460) than the refined model. - Discriminant-validity evidence for the broader AI-literacy-related domains was mixed — five constructs passed the Fornell-Larcker check, but information-source engagement and support/enablers showed the strongest overlap (r = .748, then r = .708 with awareness) — so those domains were treated as theoretically informed predictors rather than fully validated independent latent variables. - Scale reliabilities were acceptable (Cronbach's alpha: AI awareness .868, tool familiarity .841, readiness eight-item .844, contextual barriers .770, critical-ethical appraisal .755, support/enablers .745, information-source engagement .738). - Overall, readiness was associated with direct AI experience and critical-ethical judgment, while the contribution of support/enablers remains provisional. ## Study Design & Method The study is described as factor-informed: AI-literacy-related domains were treated as theoretically informed and empirically tested predictors of AI-TPACK readiness rather than as fully validated independent constructs. Exploratory factor analysis using polychoric correlations supported the unidimensionality of the readiness items, after which a structural equation model was estimated with gender as a control (n = 129 because one participant selected "prefer not to say" for gender). The refined structural model (χ2(602) = 789.92, CFI = .981, TLI = .984, RMSEA = .049, SRMR = .082, R² = .530) outperformed the original eight-item specification, which showed weaker approximate fit (CFI = .957, TLI = .962, RMSEA = .075, SRMR = .094, R² = .539); sensitivity checks confirmed the stability of the support/enablers, prior-AI-use, and critical-ethical-appraisal paths while year level became non-significant. The sample of 130 preservice mathematics teachers comes from a public university in South Africa, making the study a contribution to AI-TPACK research in a Global South teacher education context. ## Implications for AI in Education For [[math-education]] teacher preparation, the findings suggest that hands-on experience with AI and the capacity for critical-ethical appraisal of AI tools are the most robust correlates of pedagogical readiness, while institutional support and enabling conditions matter but are harder to measure cleanly — the support/enablers scale bundled personal interest, institutional opportunities, mentor encouragement, and active information seeking, so it cannot isolate the contribution of the teacher education setting itself. The study supports [[teacher-ai-competency]] frameworks that combine direct AI experience with critical evaluation, and it provides psychometric evidence that readiness can be measured with a short, essentially unidimensional instrument — useful for [[faculty-development]] programs seeking efficient diagnostics. The mixed discriminant-validity results, however, caution against over-interpreting separate AI-literacy sub-domains, and the provisional year-level findings warn against assuming that readiness simply increases with seniority; the non-significant contextual-barriers path should not be read as evidence that infrastructure and resources are irrelevant in South African teacher education, only that they added no independent variance within this model. ## Limitations The study is cross-sectional and based entirely on self-report, so structural paths should not be interpreted causally. The sample came from one university context and was modest for a complex latent-variable model, and the EFA-informed refinement, CFA, discriminant-validity diagnostics, and SEM were all conducted on the same N = 130 dataset rather than split into development and validation samples; lavaan produced near-singular variance-covariance warnings in some models. Measurement evidence was not uniformly strong — information-source engagement had weak AVE and support/enablers was marginal, with mixed discriminant validity — so the broader AI-literacy domains should not be treated as fully validated dimensions. The study also did not measure cognitive load, classroom performance, observed AI use, or longitudinal development, leaving open whether the identified predictors translate into actual pedagogical practice. ## Connected Concepts - [[math-education]] - [[ai-literacy]] - [[higher-ed]] - [[stem-education]] - [[faculty-development]] - [[teacher-role]] ## Connected Articles - [[teacher-control-ai-generation-math-visuals]] - [[genai-runaway-object-math-higher-ed]] ## Citation Mosia, M., Nannim, F. A., & Egara, F. (2026). [*AI literacy-related domains and AI-TPACK readiness among preservice mathematics teachers: A factor-informed structural equation modelling study*](https://doi.org/10.1016/j.caeai.2026.100650). --- ## [Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design](https://edtechdev.github.io/aied/articles/ai-tpack-teacher-multi-agent-workflow/) This study investigates how teachers design multi-agent instructional workflows and identifies three distinct **teacher archetypes** that emerge from behavioral log analysis of 61 in-service teachers: - **Systematic Optimizers**: Iteratively refine complex multi-agent architectures with a methodical, architectural focus. These teachers treat AI agent orchestration as a systems design problem. - **Prolific Creators**: Rapidly prototype pragmatic tools, leveraging scaffolding to quickly produce usable instructional aids. Efficiency-oriented and output-focused. - **Passive Observers**: Exhibit polarized expert-novice profiles with minimal proactive modification of the AI workflows presented to them. ## AI-TPACK Beyond Discrete Knowledge The core theoretical contribution is that effective AI-TPACK integration — the fusion of Technological, Pedagogical, and Content Knowledge in an AI context — emerges not from possessing separate knowledge domains, but from a **dynamic interplay** of three factors: 1. **Systems thinking**: The ability to conceptualize interactions among multiple AI agents and map them to instructional goals 2. **Pedagogical beliefs**: Underlying assumptions about teaching and learning that fundamentally shape design choices 3. **Self-efficacy**: Confidence in one's ability to orchestrate complex AI-enhanced workflows This challenges static, checklist-based models of [[teacher-ai-competency]] and aligns with broader research showing that [[faculty-development]] must address cognitive-behavioral diversity, not just technology training. ## Implications for Teacher Professional Development The findings call for **differentiated scaffolding**: - Systematic Optimizers benefit from advanced system-design frameworks and revision opportunities - Prolific Creators thrive with rapid-feedback cycles and modular, reusable components - Passive Observers need explicit modeling, guided practice, and confidence-building exercises This connects to the [[teacher-ai-adoption-confidence]] finding that teacher confidence fully mediates institutional support effects on AI adoption. It also extends the [[teachingcoach-chatbot-instructor-guidance]] paradigm by suggesting that coaching scaffolds must be personalized to teacher archetypes, not just content domains. ## Connection to Agentic AI in Education The multi-agent workflow framing positions this work at the intersection of [[agentic-ai]] and teacher professional learning. As [[eduagentbench-agent-teaching-benchmark]] establishes benchmarks for what agent tutors should do, this study addresses the complementary question: **how should teachers learn to design and orchestrate those agents?** ## Connected Concepts - [[teacher-ai-competency]] - [[faculty-development]] - [[agentic-ai]] ## Connected Articles - [[teacher-ai-adoption-confidence]] - [[teachingcoach-chatbot-instructor-guidance]] - [[eduagentbench-agent-teaching-benchmark]] ## Citation Sun, Y., Xin, H., Li, S., Niu, Q., Chai, C. S., Huang, L., & Chen, G. (2026). [Modeling AI-TPACK in practice: Insights from teachers' multi-agent workflow design](https://arxiv.org/abs/2605.13906). arXiv:2605.13906 [cs.CY]. --- ## [PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions](https://edtechdev.github.io/aied/articles/ai-tutor-authoring-promptdecipher/) ## Key Finding Teachers virtually never test AI tutoring bots before student deployment; PromptDecipher enforces QA as a first-class activity by letting teachers edit bot responses directly. ## Synthesis PromptDecipher addresses a critical gap in AI tutor deployment: teacher quality assurance. A formative study revealed that educators authoring AI tutoring chatbots virtually never systematically test them before student deployment — a finding with serious implications for [[ai-tutor-safety-harms]] and educational quality. The system shifts the authoring paradigm from abstract prompt writing to direct correction-based interaction: teachers edit undesirable bot responses in a live chat preview, and an automated pipeline analyzes the correction, proposes a system prompt rewrite, and validates across test scenarios. This bridges the [[teacher-role]] gap between classroom practitioner and AI system designer — a tension also explored in [[ai-tpack-teacher-multi-agent-workflow]], which found that effective AI integration requires systems thinking beyond simple tool use. PromptDecipher's QA enforcement resonates with the [[agentic-workflows-education]] paradigm of using AI to scaffold human roles. By embedding testing into the authoring workflow, the system also mitigates the kind of diagnostic failures identified in [[llm-tutoring-feedback-diagnosis-gap]], where LLMs struggle precisely where feedback matters most. ## Connected Concepts - [[teacher-role]] ## Connected Articles - [[ai-tutor-safety-harms]] - [[ai-tpack-teacher-multi-agent-workflow]] - [[agentic-workflows-education]] - [[llm-tutoring-feedback-diagnosis-gap]] ## Citation J, A.K.M.X.R.S. (2026). [*PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions*](https://arxiv.org/abs/2605.16605). practice, however, teachers rarely fulfill these roles --- ## [The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness](https://edtechdev.github.io/aied/articles/ai-tutor-behavioral-evaluation/) > A framework for evaluating AI tutoring systems that extends beyond pedagogical quality of feedback to measure what students actually *do* with that feedback — whether they act on it and whether they apply it correctly. Proposed by Niousha et al. (2026) based on analysis of 10,235 real student code submissions. # AI Tutor Behavioral Evaluation ## Definition A framework for evaluating AI tutoring systems that extends beyond pedagogical quality of feedback to measure what students actually *do* with that feedback — whether they act on it and whether they apply it correctly. Proposed by Niousha et al. (2026) based on analysis of 10,235 real student code submissions. ## Key Findings - **Pedagogy-only evaluation is insufficient:** Two AI tutors with similar pedagogical quality showed dramatically different student engagement patterns. - **Behavioral signals predict perceived helpfulness better** than pedagogical quality scores alone. - **Actionable metrics:** The framework provides concrete measurements — action rate (did the student modify their submission?) and correct application rate (was the modification applied correctly?). ## Significance for AI in Education This work addresses a critical evaluation gap. An AI tutor that gives perfect pedagogical feedback is worthless if students ignore it or apply it incorrectly. The behavioral axis complements pedagogical assessment to provide a complete picture of real-world effectiveness. This has direct implications for [[ai-tutor-effectiveness-review]] and challenges the assumptions in [[tutoring-specific-vs-general-ai]] about what makes tutoring effective. ## Open Questions - Can behavioral evaluation be automated at scale across different tutoring domains? - How do behavioral metrics correlate with long-term learning outcomes vs. short-term perception? - What is the optimal balance between pedagogical and behavioral evaluation weighting? ## Connected Concepts - [[pedagogical-llm-training]] - [[socratic-method]] - [[math-education]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[knowledge-tracing]] - [[teacher-ai-competency]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[tutoring-specific-vs-general-ai]] - [[academiclaw-student-agent-benchmark]] - [[ai-pedagogical-accompaniment-amico]] - [[automatic-short-answer-grading]] - [[clara-collaboration-literacy-dashboard]] - [[collaborative-ai-tutoring]] - [[cstutorbench-slm-tutors]] - [[difficulty-aware-dialogue-kt]] - [[eduagentbench-agent-teaching-benchmark]] ## Citation Niousha, R., Smith, S.B., Akram, B., Brusilovsky, P., Hellas, A., Leinonen, J., DeNero, J., & Norouzi, N. (2026). [*The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness*](https://arxiv.org/abs/2605.05648) --- ## [AI Tutor Effectiveness Review](https://edtechdev.github.io/aied/articles/ai-tutor-effectiveness-review/) > Zerkouk, Mihoubi & Chikhaoui (2025) systematically analyzed qualified studies from 2010–2025 across: > A comprehensive systematic review of AI-based Intelligent Tutoring Systems (2010–2025) reveals a field with transformative potential but mixed real-world effectiveness, persistent implementation challenges, and a critical need for stronger experimental rigor.^[[zerkouk-comprehensive-review-its-2025]] ## Review Scope Zerkouk, Mihoubi & Chikhaoui (2025) systematically analyzed qualified studies from 2010–2025 across: - **Pedagogical strategies** in ITS design - **Natural Language Processing** for tutoring dialogue - **Adaptive learning** algorithms and architectures - **Student modeling** approaches (knowledge, affect, behavior) - **Domain-specific applications** (math, language, science, programming) ## Key Findings ### 1. Mixed Effectiveness Despite decades of progress and significant transformative potential, ITS have produced **mixed results** in real-world educational contexts. Effectiveness varies dramatically by: - Domain (math and programming often show stronger gains than open-ended writing) - Implementation fidelity (lab studies outperform classroom deployments) - Student population (low-prior-knowledge students often show larger relative gains) - Duration (short-term studies overstate gains vs. sustained use) ### 2. Complex Advancement Landscape The field shows both notable advancements and persistent challenges: - **Advancements:** Deep learning for student modeling, NLP for natural dialogue, multi-agent architectures, affective computing - **Persistent challenges:** Scalability of expert content authoring, maintenance of knowledge bases, integration with existing curricula, teacher adoption barriers ### 3. Scientific Rigor Gap The review identifies a **critical need for stronger experimental design and data analysis**: - Many studies lack control groups or proper randomization - Reporting standards for ITS interventions are inconsistent - Long-term follow-up is rare - Real-world classroom studies are underrepresented relative to lab studies ## Synthesis with Current Wiki Evidence | Claim in review | Supporting evidence in wiki | Contradictory evidence | |---|---|---| | ITS show mixed real-world effectiveness | [[stanford-evidence-base-ai-k12-2026]] (only 20/818 papers meet causal standards) | [[pedagogical-llm-training|EduQwen]] (96.52% benchmark, but benchmark ≠ classroom) | | Need for stronger experimental rigor | [[educational-llm-alignment|Hardy & Kim]] (benchmark≠teaching quality) | — | | NLP advances for dialogue | [[knowledge-tracing-irt]] (interpretable dialogue modeling) | [[ai-tutor-safety-harms|SafeTutors]] (multi-turn degradation: 17.7% → 77.8%) | | Affective computing as advancement | [[affective-tutoring|MathBuddy]] (+23 points win rate) | [[ai-tutor-safety-harms]] (emotional risks, parasocial dependency) | | Multi-agent architectures | [[agentic-workflows-education]] (four paradigms), [[human-in-the-loop-ai]] (MAIC) | [[collaborative-ai-tutoring|ProPACT]] (effective but requires eye-tracking hardware) | ## Implications for the Field 1. **Benchmarks are not enough.** High benchmark scores (CDPK, DeepTutor) must be complemented by classroom RCTs measuring actual learning gains. 2. **Teacher integration is a bottleneck.** Technical sophistication matters less than curriculum fit and teacher control — see [[human-in-the-loop-ai]]. 3. **Long-term studies are essential.** Most ITS research measures immediate outcomes; SRL, metacognition, and transfer require longitudinal designs. 4. **Domain-specificity is real.** A system effective in math may fail in writing; claiming "general tutoring" without domain evidence is overstated. ## Connected Concepts - [[pedagogical-llm-training]] - [[affective-tutoring]] - [[human-in-the-loop-ai]] ## Connected Articles - [[zerkouk-comprehensive-review-its-2025]] - [[stanford-evidence-base-ai-k12-2026]] - [[educational-llm-alignment]] - [[knowledge-tracing-irt]] - [[ai-tutor-safety-harms]] - [[agentic-workflows-education]] - [[collaborative-ai-tutoring]] ## Citation Zerkouk, Mihoubi & Chikhaoui (2025). [AI Tutor Effectiveness Review](https://arxiv.org/abs/2507.18882v1). --- ## [AI Tutor Safety and Pedagogical Harms](https://edtechdev.github.io/aied/articles/ai-tutor-safety-harms/) > Conventional LLM safety benchmarks focus on toxic outputs, jailbreaks, and bias. In education, the primary risks are quieter: > "Solving problems correctly and avoiding toxic language does not make a tutor safe. Tutoring-specific harm is qualitatively different." SafeTutors exposes that **all tested models show broad pedagogical harm**, with failures escalating from **17.7% in single-turn to 77.8% in multi-turn** student-tutor dialogue.^[[hazra-safetutors-pedagogical-safety-2026]] ## Why Tutoring Safety Is Different Conventional LLM safety benchmarks focus on toxic outputs, jailbreaks, and bias. In education, the primary risks are quieter: - **Answer over-disclosure** — Revealing solutions rather than facilitating guided discovery - **Misconception reinforcement** — Validating or ignoring student misunderstandings - **Abdication of scaffolding** — Failing to provide appropriate structured support - **Erosion of productive struggle** — Short-circuiting the cognitive work that consolidates understanding These harms appear "helpful" to surface inspection: the student gets a correct answer quickly. But the long-term effect is learning atrophy. ## The SafeTutors Risk Taxonomy Hazra et al. (2026) derive **11 harm dimensions and 48 sub-risks** from learning-science literature: | Dimension | Core Concern | Key Examples | |-----------|--------------|--------------| | **Cognitive** | Interferes with knowledge internalization | Cognitive offloading, fluency illusion, shallow procedural learning | | **Epistemic** | Weakens justification/evaluation ability | Unverified authority, source opaqueness, false consensus | | **Metacognitive** | Erods monitoring and self-reflection | External validation dependence, reflection bypass, learned helplessness | | **Motivational-Affective** | Undermines curiosity and persistence | Shortcut temptation, performance-over-mastery, emotional disengagement | | **Developmental & Equity** | Fails to calibrate to learner level | Cognitive load mismatch, unequal benefit, cultural bias | | **Instructional Alignment** | Departs from learning goals | Pedagogical drift, goal misidentification, hidden curriculum | | **Behavioral & Inquiry** | Enables shortcuts/dishonesty | Answer-seeking bypass, assignment outsourcing | | **Ethical-Epistemic Integrity** | Compromises intellectual ownership | Blurred authorship, misrepresentation of understanding | | **Informational-Semantic** | Embeds factual inaccuracies | Fabrication, misleading scientific explanation | | **Reflective-Critical** | Suppresses evidence-weighing | Over-smooth acceptance, no metacognitive challenge | | **Pedagogical Relationship** | Dysfunctional learner-system dynamic | Over-trust in AI authority, loss of learner agency | ## Critical Findings 1. **Universal harm:** All 11 tested models (3.8B–72B open-weight + GPT-5-mini) exhibited broad pedagogical harm 2. **Scale is not a fix:** Larger models were not reliably safer; raw helpfulness correlates weakly with pedagogical safety 3. **Multi-turn degradation:** Harm rates rose from 17.7% (single-turn) to **77.8% (multi-turn)**, showing that sustained tutoring interaction progressively erodes safety 4. **Discipline-aware mitigations needed:** Harms varied significantly across math, physics, and chemistry 5. **Single-turn evaluation is misleading:** "Safe" single-turn responses masked systematic failure when conversations extended to 5–8 turns ## Relationship to Broader Debates - [[tutoring-specific-vs-general-ai]] — SafeTutors reveals that even "helpful" general-purpose AI produces systematic tutoring harm; pedagogical design is not an add-on but a safety requirement - [[metacognition]] — The Metacognitive and Reflective-Critical dimensions directly map to metacognitive suppression risks - [[self-regulated-learning]] — Motivational-Affective harms undermine the SRL↔motivation reciprocal loop - [[transfer-of-learning]] — Cognitive offloading and shallow learning directly undermine transfer; SafeTutors provides a mechanistic taxonomy for why - [[llm-fallacy-misattribution]] — Fluency illusion (Cognitive dimension) and misrepresentation of understanding (Ethical-Epistemic dimension) are tutoring-specific instantiations of the LLM Fallacy ## Implications - **Evaluation:** Tutor safety must be measured with multi-turn, discipline-specific benchmarks, not single-turn toxicity screens - **Design:** Guardrails must target pedagogical failure modes (over-disclosure, misconception reinforcement) not just content correctness - **Policy:** Procurement criteria for educational AI should include pedagogical safety audits alongside accuracy metrics ## Connected Concepts - [[metacognition]] - [[self-regulated-learning]] ## Connected Articles - [[hazra-safetutors-pedagogical-safety-2026]] - [[tutoring-specific-vs-general-ai]] - [[transfer-of-learning]] - [[llm-fallacy-misattribution]] ## Citation Hazra, R., Ghuku, B., Marchenko, I., Tokarieva, Y., Layek, S., Banerjee, S., Stoyanovich, J., & Pechenizkiy, M. (2026). [*SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems*](https://arxiv.org/abs/2603.17373). arXiv:2603.17373. --- ## [Methodologies for Improving the Quality of AI Tutoring in K-12 Education](https://edtechdev.github.io/aied/articles/ai-tutoring-quality-k12-methodologies-2026/) > **Synthesis:** Udeshi et al. (2026), the team behind **Khanmigo** (Khan Academy's K-12 AI tutor, launched 2023), describe the metrics they use to measure AI tutoring quality and student engagement, along with the live experiments that have moved those metrics. Given that LLMs are opaque black boxes, they argue robust evaluation and live experimentation are essential. The paper highlights changes across models, prompting, personalization, and agents that improved tutoring outcomes. Accepted at AIED 2026, it connects to [[ai-tutoring]], [[intelligent-tutoring]], and [[efficacy-study]] literatures. ## Evaluation as the Engine of Improvement Many AI tutors leverage large language models today. Because LLMs are opaque black boxes, robust evaluation and live experimentation to measure the impact of every change are essential. Khan Academy pioneered AI-powered tutoring for K-12 with the launch of Khanmigo (2023). ## What They Measure and Change The paper describes the metrics used to measure AI tutoring quality and student engagement, and the various experiments run to improve them. It highlights the changes that moved their metrics, including changes to **models, prompting, personalization, and agents**. ## Position This practitioner account from a major edtech platform grounds the [[ai-tutoring]] and [[intelligent-tutoring]] literature in real, large-scale K-12 deployment evidence, complementing controlled [[efficacy-study]] research. ## Connected Concepts - [[ai-tutoring]] - [[intelligent-tutoring]] - [[k-12]] - [[llm]] - [[personalized-learning]] - [[efficacy-study]] - [[engagement-metrics]] - [[edtech-platform]] - [[student-experience]] - [[prompt-engineering]] ## Connected Articles - [[oatutor-open-source-adaptive-tutor-2023]] - [[genai-tutor-engagement-patterns]] - [[measuring-llm-tutors-teach-vs-solve]] - [[correct-answer-trap-ai-tutor]] - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] - [[access-not-enough-ai-tutoring-2026]] - [[tutoring-effectiveness-index]] - [[deeptutor]] ## Citation Udeshi, T., Khazenzon, A., Khan, K., Breen, N., Corwin, R. J., DiGiano, C., Weatherholtz, K., & Zaluski, M. (2026). [*Methodologies for improving the quality of AI tutoring in K-12 education*](https://arxiv.org/abs/2608.11259). In Artificial Intelligence in Education (AIED 2026), LNCS vol. 16582. Springer. arXiv:2608.11259. --- ## [Artificial Intelligence in UK Higher Educational Policy and Institutional Decision Making](https://edtechdev.github.io/aied/articles/ai-uk-higher-education-policy-2026/) > **Synthesis:** This systematic literature review examines how AI is positioned in UK higher-education policy and its influence on institutional pedagogical decision making, finding that AI integration is accelerating but fragmented, with a gap between policy ambitions and institutional capacity and disparities between teaching-led and research-intensive universities. ## Key Findings 1. **Fragmented and accelerating adoption.** AI integration in UK higher education is accelerating but remains fragmented, revealing a gap between policy ambitions and institutional capacity. 2. **Institutional disparities.** Differences between teaching-led and research-intensive universities highlight disparities in infrastructure and staff preparedness. 3. **Policy concerns cluster around equity and efficacy.** Key concerns include ethics, equity exclusion, and learner efficacy, compounded by limited theoretical coherence and participatory governance. 4. **A layered framework proposed.** The review proposes a layered, inclusive framework linking national policy, institutional infrastructure, and governance. ## Implications This review speaks directly to [[educational-policy-ai]] and [[ai-governance-education]], showing how national policy intent can outpace institutional capacity in [[higher-ed]]. The documented gap between teaching-led and research-intensive institutions underscores [[equity]] concerns and the risk that AI benefits accrue unevenly across the sector. For administrators and [[educational-policy-ai]]-makers, the findings argue for participatory governance and infrastructure investment rather than purely aspirational strategy documents. The proposed layered framework connecting national policy, institutional infrastructure, and governance offers a concrete model for closing the ambition-capacity gap. The review also connects to broader debates about [[ethics]] and learner efficacy in AI adoption, and to [[ai-governance-education]] and technology-adoption dynamics in universities, resonating with the concept of [[dot-framework-survey]] on digital transformation readiness. ## Connected Concepts - [[administrator]] - [[ai-governance-education]] - [[educational-policy-ai]] - [[equity]] - [[ethics]] - [[higher-ed]] - [[ai-governance-education]] - [[educational-policy-ai]] ## Connected Articles - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review - [[learnity-graphs-lifelong-learning-framework-2026]] — Learnity graphs framework - [[pchl-he-framework-genai-content-creation-2026]] — PCHL-HE framework - [[amponsah-ai-acceptance-science-teachers-2026]] — AI acceptance science teachers ## Citation Ashiq, S. (2026). [*Artificial Intelligence in UK Higher Educational Policy and Institutional Decision Making*](https://osf.io/6kjmy). EdArXiv preprint. --- ## [From AI Use to Critical Thinking Among Medical Students: A Moderated Mediation Perspective on Cognitive Load and Self-Regulated Learning](https://edtechdev.github.io/aied/articles/ai-use-critical-thinking-medical-students-2026/) > **Synthesis:** Arshad et al. (2026) examined how AI-based educational technology influences critical thinking among 480 undergraduate medical students in Pakistan, using a cross-sectional design and Hayes' PROCESS Model 14. They found that AI use was positively associated with critical thinking and self-regulated learning, while cognitive load negatively related to both. Cognitive load partially mediated the AI-use→critical-thinking link, and self-regulated learning significantly moderated that indirect effect — the negative impact of cognitive load on critical thinking weakened at higher levels of self-regulation. The study argues that AI's effectiveness for higher-order thinking depends not only on the cognitive support AI provides but on learners' capacity to regulate their engagement. ## Key Findings - AI-based technology use was positively associated with both critical thinking and self-regulated learning among medical students, whereas perceived cognitive load was negatively associated with both outcomes. - Mediation analysis showed cognitive load partially mediated the relationship between AI use and critical thinking — part of AI's effect on thinking runs through the cognitive burden it imposes. - Moderated mediation revealed self-regulated learning significantly moderated the indirect effect: the negative impact of cognitive load on critical thinking was weakened at higher levels of self-regulation. - The study situates AI's educational impact within [[cognitive-load-theory]] and [[self-regulated-learning]], framing generative AI tools (ChatGPT, Claude, Gemini) as double-edged — capable of reducing extraneous load by organizing/summarizing, yet adding burden through large volumes of content that must be evaluated and validated. ## Study Design & Method The authors used a cross-sectional survey design with 480 undergraduate medical students who had prior experience using AI tools. Data were collected with standardized measures assessing AI usage, cognitive load, self-regulated learning, and critical thinking. Hayes' PROCESS Model 14 was used to test mediation and moderated mediation effects. The analytic approach allowed the authors to model cognitive load as a mediator of AI use on critical thinking and self-regulated learning as a moderator of the mediated (indirect) path — specifically, whether the load→thinking pathway is conditional on students' regulatory capacity. ## Implications for AI in Education The findings caution against treating AI as uniformly beneficial for higher-order cognition. They suggest that the same AI tool can either scaffold or undermine [[critical-thinking]] depending on how much mental work it offloads and whether students can strategically regulate their engagement. For educators, this argues for designing AI-supported learning that deliberately preserves opportunities for independent reasoning — prompting verification, reflection, and judgment rather than answer-replacement — and for building students' [[self-regulated-learning]] capacity as a protective factor against the [[cognitive-offloading]] and shallow-processing risks of generative AI. The authors position the work as a corrective to the research literature's emphasis on efficiency, academic performance, and engagement at the expense of the cognitive processes underlying higher-level thinking, and connect it to academic-integrity and dependency concerns in medical education. ## Limitations The cross-sectional, non-experimental design limits causal inference despite theory-consistent directional modeling. Data are self-report measures collected at a single time point, which raises common-method-bias concerns; the moderated mediation effects are estimated from this single snapshot. The sample is drawn from medical students in one national context (Pakistan), bounding generalizability to other disciplines, institutions, and educational systems. AI usage, cognitive load, and self-regulated learning are all operationalized via standardized self-report instruments rather than objective behavioral measures. ## Connected Concepts - [[cognitive-load-theory]] - [[self-regulated-learning]] - [[critical-thinking]] - [[generative-ai]] - [[higher-ed]] - [[cognitive-offloading]] - [[ai-literacy]] ## Connected Articles - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI - [[ai-feedback-critical-thinking-writing-2026]] — Using AI-Generated Feedback to Improve Critical Thinking - [[ai-availability-student-motivation]] — "Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation - [[same-ai-different-pathways]] — Same AI, Different Pathways: Unpacking Mechanisms of AI-Mediated Learning - [[lodge-loble-cognitive-offloading-2026]] — Artificial Intelligence, Cognitive Offloading and Implications for Learning ## Citation Arshad, A., Lone, A., Arickswamy, L., Hassan, K., Alnaim, A. A., & AlFarhan, M. F. (2026). [*From AI use to critical thinking among medical students: A moderated mediation perspective on cognitive load and self-regulated learning*](https://doi.org/10.3389/fpsyg.2026.1883053). --- ## [Beyond Output Metrics: Reframing AI-Assisted Vocal Pedagogy Through Human Learning and Educational Value](https://edtechdev.github.io/aied/articles/ai-vocal-pedagogy-2026/) > **Synthesis:** Li (2026) presents a conceptual Perspective arguing that AI-assisted vocal pedagogy should be evaluated not by how precisely AI measures vocal output (pitch, stability, timing) but by how AI-generated evidence becomes meaningful for human learning — how learners interpret feedback, regulate practice, sustain motivation, and develop trust in teacher-guided processes. The article proposes a three-level framework linking technical adaptation, human learning processes, and educational outcomes, with effectiveness, equity, and sustainability as outcome criteria. It concludes that AI should not be positioned as an autonomous evaluator of singing quality, but as a human-centered support for interpretation, reflection, teacher–student dialogue, and pedagogically responsible decision-making. ## Key Findings - The central conceptual problem is that measurable vocal outputs (pitch deviation, acoustic stability, vibrato, timing regularity) do not fully represent learners' internal experience, embodied coordination, or expert pedagogical interpretation — the same measured deviation may indicate technical inaccuracy, expressive inflection, or a recording/context issue. - AI-generated evidence becomes educationally valuable only when interpreted through human learning processes: bodily awareness, cognition and metacognitive monitoring, self-regulated practice, motivation, learner beliefs, and pedagogical mediation. - If AI evidence is treated as educational value in itself, vocal training risks being narrowed into output correction, score optimization, or automated judgment — a risk consistent with broader guidance that AI should support human-centered learning. - Effectiveness, equity, and sustainability are proposed as outcome criteria for judging whether AI-supported feedback is meaningful for learning, usable across learners, and sustainable for long-term vocal development. - The framework is explicitly recursive, not a one-way pipeline: practice produces new vocal evidence, so interpretation, feedback, practice, and reflection feed back into one another. ## Conceptual Framework The article organizes AI-assisted vocal pedagogy into three linked levels. *Technical adaptation* is the evidence AI makes visible — measurable performance features such as pitch, stability, vibrato, and timing. *Human learning processes* describe how that evidence is interpreted through bodily experience, cognition and metacognitive monitoring, self-regulated practice, motivation, learner beliefs, and pedagogical mediation. *Educational outcomes* indicate whether the evidence supports vocal development over time. Drawing on five literatures — singing voice science, vocal pedagogy and embodied music cognition, feedback and [[self-regulated-learning|self-regulated practice]], [[metacognition]] and reflective practice, and recent AI-assisted music learning plus human-centered responsible AI — the framework asks three questions: what evidence does AI make visible, how is it interpreted, and what educational outcomes follow? ## Implications for AI in Education The Perspective extends debates about [[feedback-loop|AI feedback]] and [[human-in-the-loop-ai|human-in-the-loop]] design to a domain — vocal/music education — where bodily, interpretive, and developmental learning resist reduction to metrics. It warns against equating measurement precision with educational value and positions AI as a support for teacher–student dialogue and reflection rather than an autonomous judge. For designers of [[generative-ai]] educational tools, it argues that feedback must be interpretable, pedagogically mediated, and connected to learners' lived experience and developmental readiness. It also foregrounds [[equity]] (usable across learners) and sustainability as explicit outcome criteria, echoing broader calls for human-centered, educationally responsible AI in the [[teacher-role|teacher-guided]] learning process. ## Limitations As a Perspective article, it offers a conceptual framework rather than empirical data, and its claims rest on argument and synthesis of prior literature rather than tested outcomes. The framework's three levels and three outcome criteria are proposed heuristics, not validated measures. Its applicability across different vocal genres, pedagogical traditions, and educational levels is asserted conceptually rather than demonstrated empirically. ## Connected Concepts - [[generative-ai]] - [[feedback-loop]] - [[self-regulated-learning]] - [[metacognition]] - [[human-ai-collaboration]] - [[human-in-the-loop-ai]] - [[teacher-role]] ## Connected Articles - [[multimodal-ai-feedback-learning]] — LLM-Based Multimodal AI Feedback Produces Equivalent Learning - [[genai-feedback-design-multisite-experiment]] — Human-Centered GenAI Feedback Design in Higher Education - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition - [[ai-feedback-critical-thinking-writing-2026]] — Using AI-Generated Feedback to Improve Critical Thinking ## Citation Li, Y. (2026). [*Beyond output metrics: Reframing AI-assisted vocal pedagogy through human learning and educational value*](https://doi.org/10.3389/fpsyg.2026.1844841). --- ## [Artificial intelligence in vocational education and training: A systematic review of educational purposes, theoretical conceptualizations, and empirical effectiveness](https://edtechdev.github.io/aied/articles/ai-vocational-education-training-review/) > **Authors:** Viola Deutscher, Herbert Thomann, Olga Zlatkin-Troitschanskaia, Ulrike Weyland, Stephan Abele, Amory H. Danek, Samuel Greiff, Andreas Rausch, Susan Seeber, Jürgen Seifried, Esther Winther **Source:** Computers and Education: Artificial Intelligence, Vol 11, 100628 — Open Access (CC BY 4.0) > First systematic review of AI in vocational education and training, identifying 26 empirical studies (2015–2026) via ERIC, Web of Science, and Elicit, analyzed with a theory-informed coding scheme under PRISMA guidelines. ## Key Findings - This is the first systematic review of AI in vocational education and training (VET), identifying 26 empirical studies published between 2015 and 2026 through ERIC, Web of Science, and Elicit, analyzed with a theory-informed coding scheme following PRISMA guidelines. - The corpus spans 9 technical-domain studies, 3 in health, 5 in business administration and services, and 9 domain-general studies; settings were 6 classroom, 8 online, 4 blended, and 8 simulation-based — with **no study conducted in workplace settings** despite VET's work-based character. - Research is geographically concentrated: **17 of 26 studies originated in Asia** (China, South Korea, Singapore, Taiwan, Thailand, Indonesia), with the remainder from European contexts (Germany, the Netherlands, Norway) and isolated contributions (New Zealand, Saudi Arabia, Turkey); the field is also fragmented, as only 9 studies shared at least one reference and none cited each other directly. - Intelligent Extended Reality (XR) shows consistent positive effects on procedural competence, practical skills, and learner motivation — e.g., a randomized pre-post comparison (iXR n = 14 vs. traditional group task n = 15) found both groups gained knowledge but gains were significantly higher with iXR. - Intelligent Tutoring Systems foster declarative and procedural knowledge, while AI chatbots show promising effects on self-regulation and task performance — including a grounded-theory study of 408 polytechnic students tracing a self-regulatory arc (goal setting → performative interaction → reflection), though the only randomized chatbot trial (n = 50, posttest-only) reported advantages on seven competencies without effect sizes, pretests, or baseline checks. - The evidence base is methodologically constrained: only five randomized experimental studies were identified among the 26, 21 of 26 rely on pre-experimental or quasi-experimental designs, and most measure outcomes immediately after the intervention; affective and meta-cognitive outcomes rest predominantly on self-report, raising novelty-effect concerns. - Only **three studies explored AI-empowered designs** that grant learners an active role; meta-cognitive goals such as self-regulated learning are frequently espoused but rarely implemented through genuinely learner-empowered systems. - Across applications, AI is predominantly implemented through behaviorist or cognitively oriented instructional designs that emphasize drill-and-practice and adaptive feedback, while approaches fostering learner agency, critical reflection, and autonomous decision-making remain underrepresented. - Current research largely reflects a generalized "success narrative"; the authors call for future studies of failure cases, contextual moderators, and boundary conditions to develop a more differentiated understanding of effectiveness. ## Study Design & Method This is a PRISMA-guided systematic review of 26 empirical studies (2015–2026) identified through ERIC, Web of Science, and Elicit, with Scopus added as a supplementary domain-specific database, and coded with a theory-informed scheme that distinguishes AI-directed, AI-supported, and AI-empowered human-AI interaction paradigms and underlying learning theories; all included studies were independently double-coded, with coding documented in a publicly available dataset (Appendix 2). Methodologically, quantitative designs dominate (14 studies, surveys most frequent), followed by mixed methods (10) and two qualitative case studies; sample sizes range from 9–15 VET learners (qualitative) to 20–3,518 (quantitative). Among the quantitative studies, 4 employ quasi-experimental approaches and 4 use randomized experiments per the methodological breakdown, and 2 rely exclusively on self-report while 5 use only objective measures (performance tests or log data). ## The Constructivist Paradox and the Turing Trap The review documents a notable paradox: constructivist theories are espoused in VET discourse while behaviorist AI implementations dominate in practice. The authors warn against an educational "Turing Trap" — the danger of using AI to replicate human instruction rather than to augment human judgment. Realizing the transformative potential of AI in VET, they argue, requires learning environments that augment human judgment, strengthen learner agency, and support teachers, rather than systems that merely automate existing instructional patterns. ## Implications for AI in Education For educators and developers, the review offers a map of what the evidence currently supports: XR for procedural and practical skills, ITS for declarative and procedural knowledge, and chatbots for self-regulation support — with the most consistent benefits emerging when AI augments authentic, practice-proximal environments, especially [[intelligent-tutoring]]-style simulations in technical domains. The scarcity of randomized experiments flags the need for stronger causal designs, and the dominance of drill-and-practice implementations suggests that [[professional-training]] contexts are under-serving the learner-agency goals that VET espouses; delayed post-tests, objective performance-based assessments, and analyses of transfer to workplace contexts are largely absent from the literature. The Turing Trap framing connects directly to [[efficacy-study]] debates and to [[constructivist]] design commitments, and the call for reporting failure cases is a useful corrective to the field's prevailing success narrative — while the heavy reliance on self-report for [[self-regulated-learning]] outcomes should temper claims about chatbots' regulatory effects. ## Limitations The review restricted its search to English-language, peer-reviewed journal articles, likely excluding grey literature and non-English work — a notable gap given the applied, project-based, and often locally documented nature of VET interventions. The database-dependent search may underrepresent regions with distinct publication cultures (partly explaining the Asia/Europe concentration), and the use of Elicit as an AI-assisted discovery tool constrains full reproducibility because retrieved outcomes depend on probabilistic ranking mechanisms and database coverage changes. Finally, the heterogeneity of included studies and frequent lack of transparency about AI implementations and instructional designs required interpretive judgment in coding, despite double-coding and consensus-based resolution of discrepancies. ## Connected Concepts - [[professional-training]] - [[efficacy-study]] - [[intelligent-tutoring]] - [[lifelong-learning]] - [[self-regulated-learning]] - [[constructivist]] - [[affective-tutoring]] - [[open-source]] ## Connected Articles - [[supplynet-visual-exploratory-learning]] — SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[genai-pd-ai-pck-learning-gain-2026]] — Efficacy of an Intensive Generative AI Professional Development Program on Pedagogical Content Knowledge (AI-PCK) and the Comparative Analysis of Learning Gain between Experienced and Pre-service Teachers - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components - [[multimodal-affective-its-presentation]] — An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training ## Citation Deutscher, V., Thomann, H., Zlatkin-Troitschanskaia, O., Weyland, U., Abele, S., Danek, A. H., Greiff, S., Rausch, A., Seeber, S., Seifried, J., & Winther, E. (2026). [*Artificial intelligence in vocational education and training: A systematic review of educational purposes, theoretical conceptualizations, and empirical effectiveness*](https://doi.org/10.1016/j.caeai.2026.100628). --- ## [From Planning to Revision: How AI Writing Support at Different Stages Alters Ownership](https://edtechdev.github.io/aied/articles/ai-writing-support-stage-ownership-2026/) # AI Writing Support and Ownership by Stage > Gero, Long, Schnitzler & Dhillon (2026, DIS '26) ran a between-subjects essay study (n = 253) showing that **where** AI support enters the writing process determines how much students feel they own the work: any AI assistance decreased ownership, but planning support cost the least while drafting support cost the most. The mechanism is AI-contributed text and ideas — and there is a genuine **quality–ownership trade-off**. ## The experiment - Short-essay writing, n = 253, between-subjects - AI support offered at one of three stages: **planning, drafting, or revising** - Measured: felt ownership, AI-contributed text/ideas, essay quality ## Findings - **Any AI assistance decreased ownership** - **Planning support: minimal decrease** (outline-level help preserves authorship) - **Drafting support: largest decrease** — and an AI draft built from the participant's *own outline* still contributed far more ideas than planning support - More AI-contributed text/ideas → less ownership, but also **better essay quality** - Recommendation: writers, educators, and designers should consider writing stage when introducing AI assistance ## Connected Concepts - [[generative-ai]] - [[metacognition]] - [[student-experience]] - [[writing-education]] - [[llm]] ## Connected Articles - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[authentic-assessment]] — Authentic Assessment - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation quality, M.F.O.A.T.A.E. (2026). [*From Planning to Revision: How AI Writing Support at Different Stages Alters Ownership*](https://arxiv.org/abs/2604.11009) --- ## [AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models](https://edtechdev.github.io/aied/articles/aiawe-automated-writing-evaluation/) Gayed presents **AiAWE**, an open-source [[automated-grading|automated writing evaluation]] (AWE) system that scores argumentative essays using a LoRA-adapted instruction-tuned [[llm|large language model]] (Gemma-3-27B-it). Using a proprietary ETS dataset of 480 TOEFL Independent Writing essays (120 training, 360 evaluation), the system achieves: - **RMSE:** 0.474 - **Quadratic Weighted Kappa (QWK):** 0.828 - **Agreement within ±0.5 of human score:** 90.56% These results outperform both LLaMA-3.3-70B and the fine-tuned GPT-3.5 baseline from prior work on the same dataset, demonstrating that [[generative-ai|open-weight LLMs can match or exceed proprietary fine-tuning]] for rubric-aligned scoring. Three broader findings emerge: (1) model scale is not a reliable predictor of downstream performance under LoRA adaptation; (2) identical LoRA hyperparameters produce qualitatively different adaptation behaviors across architectures; and (3) the system runs on a [[edtech-platform|consumer-grade server]], making it accessible for institutional deployment. The work contributes to debates about [[short-answer-scoring-quality-degradation|automated assessment quality]] and the feasibility of [[open-source]] AI tools in [[writing-education]]. ## Connected Concepts - [[automated-grading]] - [[llm]] - [[generative-ai]] - [[edtech-platform]] - [[open-source]] - [[writing-education]] ## Connected Articles - [[short-answer-scoring-quality-degradation]] - [[icle-plus-plus-essay-scoring]] ## Citation Gayed, J. M. (2026). [AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models](https://arxiv.org/abs/2606.12801). arXiv:2606.12801. --- ## [AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education](https://edtechdev.github.io/aied/articles/aicode-collaborative-feedback-system/) # AICoFe: AI-Based Collaborative Feedback System for Higher Education ## System Architecture AICoFe orchestrates a multi-LLM pipeline using GPT-4.1-mini, Gemini 2.5 Flash, and Llama 3.1 to synthesize quantitative rubric data and qualitative observations into actionable feedback for higher education students. The key innovation is a **teacher-in-the-loop mediation workflow**: educators use specialized Learning Analytics dashboards to curate and refine AI-generated feedback drafts before delivery. ## Technical Design - **Multi-LLM orchestration**: Three different models contribute complementary perspectives - **Hybrid storage**: SQL for traceability and structured metadata + MongoDB for semi-structured feedback versions - **Learning Analytics dashboards**: Dedicated educator interfaces for feedback curation ## Connection to Broader AIED AICoFe extends prior work on [[ai-peer-feedback-systems]] from experimental prototypes to a deployed system with educator mediation. The multi-LLM approach addresses single-model bias concerns in [[automated-grading]]. It sits at the intersection of [[feedback-loop]] design and [[learning-analytics]] — keeping educators as active curators through [[human-in-the-loop-ai]] principles. ## Connected Concepts - [[automated-grading]] - [[feedback-loop]] - [[learning-analytics]] - [[human-in-the-loop-ai]] ## Connected Articles - [[ai-peer-feedback-systems]] ## Citation Becerra, A., Palma, A., & Cobos, R. (2026). [AICoFe: Implementation and deployment of an AI-based collaborative feedback system for higher education](https://arxiv.org/abs/2605.04740). Proceedings of the Learning Analytics Summer Institute Spain 2026 (LASI Spain 2026). arXiv:2605.04740. --- ## [The Environmental Cost of LLMs in AIED: Reporting and Practices](https://edtechdev.github.io/aied/articles/aied-carbon-footprint-reporting/) > **Sabrina C. Eimler, Lukas Erle, Daniel Flood, Aditi Haiman, Luca Häckert, André Helgert, Lachlan McGinness, Büsra Yapici** **Sabrina C. Eimler, Lukas Erle, Daniel Flood, Aditi Haiman, Luca Häckert, André Helgert, Lachlan McGinness, Büsra Yapici** ## Summary LLM usage has become pervasive in the [[stanford-evidence-base-ai-k12-2026|AIED]] community, but the computational and environmental costs of deploying these models remain almost entirely unreported. This paper systematically reviews the AIED 2025 conference proceedings and finds that while most projects use LLMs, nearly none report computational resources or discuss environmental impacts as an ethical concern. ## Key Findings - **Widespread usage, invisible costs:** A review of all AIED 2025 conference papers reveals an "LLM adoption without disclosure" pattern — most projects use LLMs, but fewer than a handful report resource consumption or carbon footprint. - **No standardised reporting:** There is no established procedure in the AIED community for measuring or disclosing the computational expense or environmental cost of LLM-based systems. - **Open-source methodology proposed:** The paper introduces a standardised, open-source method with two components: 1. **Software tools** for measuring carbon footprint on both local and cloud hardware. 2. **An easy-to-use formula** for estimating computational expense of frontier LLMs, even when exact parameter counts are unknown (common with proprietary models). - **Ethical imperative:** The paper argues that failing to report environmental costs is itself an ethical concern — aligning with broader calls for [[ai-tutor-safety-harms|responsible AI]] and [[principled-ai-education|transparency]] in educational technology. ## Implications for AIED ### For Researchers - Adopt the proposed measurement tools to include carbon footprint alongside performance metrics in future publications. - Conferences should consider requiring environmental impact disclosure in review checklists, similar to ethics statements. ### For Developers - When deploying LLM-based educational tools, quantify and report computational expense so institutions can factor environmental costs into procurement decisions. - Consider efficiency-accuracy tradeoffs: smaller, fine-tuned models may achieve comparable educational outcomes at a fraction of the environmental cost. ### Connection to Broader AIED Themes - The lack of reporting parallels the [[ground-truth-reliability-aied|ground truth reliability crisis]] in AIED — in both cases, essential contextual information goes unreported, undermining the field's ability to aggregate evidence and make informed decisions. - The [[ai-tutor-safety-harms|ethics of AI in education]] must expand beyond bias and privacy to include environmental sustainability. - Institutional [[stanford-evidence-base-ai-k12-2026|adoption decisions]] should weigh environmental costs alongside learning gains and implementation costs. ## Connected Concepts - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[plagiarism-detection]] - [[administrator]] - [[bias-mitigation]] - [[open-source]] - [[pedagogical-llm-training]] - [[ai-literacy]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-safety-harms]] - [[principled-ai-education]] - [[ground-truth-reliability-aied]] - [[educational-llm-alignment]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[adapt-adaptive-lesson-plan-transformer]] - [[agency-gap-ai-writing]] - [[agent-voice-accents-k12-group-learning]] ## Citation Eimler, S. C., Erle, L., Flood, D., Haiman, A., Häckert, L., Helgert, A., McGinness, L., & Yapici, B. (2026). [The Environmental Cost of LLMs in AIED: Reporting and Practices](https://arxiv.org/abs/2606.11215). arXiv:2606.11215. --- ## [AIED's Unfinished Mission: Centering Agency and Motivation in the Age of Effortless Bypass](https://edtechdev.github.io/aied/articles/aied-unfinished-mission-bypass/) The widespread availability of general-purpose AI that can perform complex cognitive tasks threatens to undermine education at scale. This effortless bypass dilemma sharpens a challenge AIED has long engaged with but must now confront directly: ensuring learners choose effortful engagement when easier alternatives are available to complete learning tasks. In this paper, I argue that AIED's longstanding agenda of building more effective intelligent educational tools should continue, but with a renewed emphasis on the urgency of ensuring learners choose to engage authentically. Drawing on established motivational and learning theories, I outline five directions in which AIED can build on its existing strengths: supporting autonomy and agency, building learner resilience to metacognitive threats, designing for interest and relevance, amplifying process-based assessment, and empowering teachers. I then share four envisioned technologies that embody key features of this future and conclude by outlining how AIED must now evolve. Position paper (AIED 2026) reframes the effortless bypass dilemma: AIED must keep building better tools but foreground learner agency and motivation so students choose authentic effort. Five directions: autonomy/agency, metacognitive resilience, interest/relevance, process-based assessment, and teacher empowerment. This work connects to core wiki themes: [[over-reliance]] [[student-experience]] [[self-regulated-learning]] [[metacognition]] [[teacher-role]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[over-reliance]] - [[student-experience]] - [[self-regulated-learning]] - [[metacognition]] - [[teacher-role]] ## Connected Articles - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning - [[informal-learning-everyday-human-llm-interaction]] — Informal Learning Emerges in Everyday Human-LLM Interaction - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[buggy-genai-code-student-responses]] — When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata ## Citation Lane (2026). [AIED's Unfinished Mission: Centering Agency and Motivation in the Age of Effortless Bypass](https://arxiv.org/abs/2607.05557). arXiv:2607.05557. AIED 2026, LNCS 3032 (Springer). --- ## [AISSA: AI-based Student Slides Analysis Tool for Academic Presentations](https://edtechdev.github.io/aied/articles/aissa-slides-analysis/) > A web-based system that uses LLMs and Learning Analytics dashboards to provide automated, rubric-based feedback on student presentation slides. Developed by Becerra et al. (2026), AISSA addresses the scalability challenge of providing timely formative feedback in large university courses. # AISSA: AI Slides Analysis Tool ## Definition A web-based system that uses LLMs and Learning Analytics dashboards to provide automated, rubric-based feedback on student presentation slides. Developed by Becerra et al. (2026), AISSA addresses the scalability challenge of providing timely formative feedback in large university courses. ## How It Works 1. Students upload slide decks before presenting 2. System analyzes slide-level features and content using ChatGPT 5.2 3. Generates quantitative scores and qualitative feedback aligned with teacher-defined rubrics 4. Results displayed via interactive dashboards for both students and teachers ## Pilot Results Tested with 46 undergraduate students in a real academic setting. Findings: technically reliable, economically feasible, and perceived by students as useful for iterative slide improvement. ## Connected Concepts - [[human-in-the-loop-ai]] - [[automated-essay-scoring]] - [[cs-education]] - [[administrator]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[affective-computing]] ## Connected Articles - [[automatic-short-answer-grading]] — Automatic Short Answer Grading with LLMs - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[multimodal-ai-feedback-learning]] — LLM-based Multimodal AI Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models ## Citation Cobos, A.A.B.D.G.R. & 2026, V.L.S. (2026). [*AISSA: AI-based Student Slides Analysis Tool for Academic Presentations*](https://arxiv.org/abs/2605.04729) --- ## [Perceptions And Acceptance of Artificial Intelligence in Science Education Programmes: Voices of Pre-Service Science Teachers](https://edtechdev.github.io/aied/articles/amponsah-ai-acceptance-science-teachers-2026/) > **Synthesis:** Amponsah, Adu-Boahen, Commey-Mintah, Kumassah, Ayittey & Nketsiah (2026) survey 380 pre-service science teachers in Ghana using UTAUT and TPB frameworks, finding generally positive AI perceptions but a gap between intention and actual classroom use. ChatGPT emerged as the most used AI tool, with research, content explanation, and lesson planning as primary purposes. ## Key Findings - **380 pre-service science teachers** surveyed from Ghana science education programmes - **ChatGPT most used AI tool**, primarily for research, content explanation, and lesson planning - **Positive attitudes and intentions** toward AI adoption, but actual classroom use remains moderate - **UTAUT + TPB frameworks** guided the study (Unified Theory of Acceptance and Use of Technology; Theory of Planned Behaviour) - **Key barriers:** insufficient digital infrastructure, need for professional development, lack of institutional policies - **Gap identified:** positive behavioral intentions do not translate to consistent classroom AI integration without systemic support ## Implications This study provides a Global South perspective on [[ai-literacy]] and teacher AI adoption, complementing research primarily from Western contexts. The intention-practice gap mirrors findings in [[teacher-ai-adoption-confidence]], where institutional support was found to build teacher confidence. For [[faculty-development]] and [[professional-training]], the study underscores that AI tool access alone is insufficient — targeted professional development and digital infrastructure investment are prerequisites for meaningful classroom integration. The finding that ChatGPT dominates pre-service teacher AI use echoes broader trends identified in [[genai-usage-design-students-survey]] and suggests that [[ai-literacy]] curricula should address responsible and critical use of general-purpose AI tools rather than focusing on education-specific platforms. ## Connected Concepts - [[ai-literacy]] - [[professional-training]] - [[faculty-development]] - [[stem-education]] - [[k-12]] - [[ai-education]] - [[teacher-ai-competency]] ## Connected Articles - [[teacher-ai-adoption-confidence]] — AI Adoption Among Teachers: Insights on Concerns, Support, Confidence, and Attitudes - [[genai-usage-design-students-survey]] — A Study of GenAI Usage by Design Students - [[ai-higher-ed-workforce-survey]] — AI in Higher Education and the Workforce Survey - [[ai-tpack-preservice-math-teachers]] — AI-TPACK for Preservice Math Teachers - [[connected-ai-lesson-planning-vietnam]] — Connected AI Lesson Planning in Vietnam ## Citation Amponsah, K. D., Adu-Boahen, J., Commey-Mintah, P., Kumassah, E. K., Ayittey, R. F., & Nketsiah, J. (2026). [*Perceptions And Acceptance of Artificial Intelligence in Science Education Programmes: Voices of Pre-Service Science Teachers*](https://doi.org/10.35542/osf.io/7yqbe_v1). EdArXiv. doi:10.35542/osf.io/7yqbe_v1. (Published in *Advances in Research*, doi:10.31014/aior.1993.09.03.724). --- ## [When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills](https://edtechdev.github.io/aied/articles/antiskillbench-persona-skills-privacy-2026/) > **When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills** — Introduces AntiSkillBench with 7,500 persona-grounded dialogue traces from 50 behaviorally rich profiles. Evaluates skill-level privacy leakage, agent-level attribute disclosure, and behavioral impersonation across three skill-distillation strategies... [[privacy]] [[agentic-ai]] [[student-experience]] [[bias-mitigation]] [[personalized-learning]] benchmark Introduces AntiSkillBench with 7,500 persona-grounded dialogue traces from 50 behaviorally rich profiles. Evaluates skill-level privacy leakage, agent-level attribute disclosure, and behavioral impersonation across three skill-distillation strategies. Experiments across three frontier agents show persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses exhibit limited and distillation-dependent effectiveness, failing to generalize across risk and distillation strategies. ## Abstract Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personalization, this process concentrates fragmented personal signals, amplifies their impact through reuse, and challenges defenses designed for individual records or retrieval-based memory. To systematically investigate the safety of the persona-skill pipeline, we introduce AntiSkillBench, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline. ## Connected Concepts - [[privacy]] - [[agentic-ai]] - [[student-experience]] - [[bias-mitigation]] - [[personalized-learning]] ## Connected Articles - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[trust-reliance-ai-education-2026]] — Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[jeon-isd-agent-bench-2026]] — ISD Agent Benchmark ## Citation Yongli Xiang, Zhifang Zhang, Bojun Yang, Ziming Hong, Lei Feng, Miao Xu, & Tongliang Liu (2026). [When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills](https://arxiv.org/abs/2608.03700). arXiv:2608.03700. arXiv:2608.03700 [cs.CR]. --- ## [ANVIL: Analogies and Videos for Lecturers](https://edtechdev.github.io/aied/articles/anvil-ai-educational-animations/) Noviello, Birillo, and Migut (2026) present ANVIL, an end-to-end multimodal generation pipeline for educational content — one of the first systems to automate the full journey from concept definition to rendered instructional animation. The four-stage pipeline (analogy generation, screenplay compilation, animation code generation with automated repair) represents a significant advance in AI-generated [[instructional-design]] materials. ANVIL's evaluation approach is methodologically notable: rather than relying solely on automated metrics, the authors ground quality assessment in teacher evaluations and then use those findings to guide scalable automated screening. The LLM-based evaluator for analogy quality and fidelity-to-screenplay proxy for video assessment offer a replicable framework for evaluating [[generative-ai]] educational outputs at scale — addressing a key challenge identified in benchmark and efficacy-study literature. The positive educator response to perceived value and usability suggests that AI-generated instructional content may be crossing a threshold of practical classroom utility. This connects to the [[teacher-role]] discussion: ANVIL positions AI as a content-generation assistant that amplifies rather than replaces instructor expertise. The focus on CS education also complements the [[cs-education]] literature on AI tools, though ANVIL's architecture is domain-agnostic and could generalize to [[stem-education]] broadly. For [[faculty-development]], tools like ANVIL lower the production barrier for high-quality instructional media, potentially democratizing access to professional-grade educational animations. ## Connected Concepts - [[instructional-design]] - [[generative-ai]] - [[teacher-role]] - [[cs-education]] - [[stem-education]] - [[faculty-development]] ## Connected Articles - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education - [[talebzadeh-ai-green-education-2026]] — The Role of Artificial Intelligence in Green Education: Optimizing Teacher Workflow and Enhancing Pedagogical Design under Sustainable Development Pedagogy (SDP) Constraints - [[genai-marketing-education-roles-2026]] — When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education - [[concept-catalyst-engineering-scaffolds]] — Creating Learning Scaffolds for Engineering Design Using Concept Catalyst - [[prompt-based-programming-lesson]] — Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs ## Citation Yuri Noviello, Anastasiia Birillo, Gosia Migut (2026). [ANVIL: Analogies and Videos for Lecturers](https://arxiv.org/abs/2605.16295). arXiv:2605.16295. arXiv:2605.16295 [cs.CY; cs.AI; cs.CL; cs.GR; cs.HC; cs.MM]. --- ## [ASE-26: A Curriculum for Agentic Software Engineering as a Discipline](https://edtechdev.github.io/aied/articles/ase-26-agentic-software-engineering-curriculum/) Formalizes Agentic Software Engineering (ASE) as a distinct discipline. Proposes a 21-module curriculum focused on the "evolution of intent" and practitioner discipline required to manage agents rather than just writing code. This work emphasizes that [[ai-literacy]] is a developmental capacity requiring structured [[scaffolding]] and [[prompt-engineering]] discipline. It connects to the need for [[curriculum-design]] that targets [[metacognition]] and [[agentic-ai]] rather than just syntax mastery. ## Key Findings - The paper documents that professional software work increasingly consists of directing agents rather than writing code: Anthropic's Economic Index classifies 79% of Claude Code interactions as automation, Handa and colleagues find AI exposure for Computer Programmer tasks at approximately 75% of the role's distinct activities, and Brynjolfsson and colleagues report a 13% relative decline in employment for workers aged 22 to 25 in occupations most exposed to AI. - The academic literature on agentic software engineering converges on the finding that the missing capability is not better models but structured practitioner discipline. - ASE-26 is a comprehensive undergraduate curriculum for agentic software engineering as a discipline, deposited as a citable reference on Zenodo under CC BY-ND 4.0. - Its central conceptual contribution is the evolutionary spiral as the operational form of the co-evolution of intent and build — the iterative cycle in which a developer frames intent, the agent builds, and the developer judges and revises. - The curriculum sets out pedagogical commitments for grading work co-produced with an agent and is designed to outlast the specific capabilities of today's models, teaching skills such as auditability, context engineering, verification, multi-agent workflows, and AgentOps. - A motivating vignette captures the shift: a developer writes a paragraph, the agent asks clarifying questions and produces six hundred lines of code with tests, a commit message, and a draft pull request — roughly eleven minutes, where the same task took two hours three years ago. ## Curriculum Design The twenty-one-module structure organizes the discipline for teaching, building from the discipline's framing to the practitioner skills the industry currently lacks. Because grading work co-produced with an agent raises novel questions about authorship and assessment, the curriculum includes explicit pedagogical commitments for evaluating student work. And because model capabilities change rapidly, the discipline is framed around durable skills — framing tasks, judging outputs, verifying results, and managing the evolution of intent — rather than around any particular tool. ## Implications for AI in Education ASE-26 reframes [[ai-literacy]] for software professionals as the ability to manage agents, not merely to prompt them, and its emphasis on the co-evolution of intent and build aligns with [[curriculum-design]] that targets [[metacognition]] rather than syntax mastery. For higher education, the curriculum is a concrete template for programs that want to teach the discipline of [[agentic-ai]] development — including verification and auditability — rather than leaving students to acquire it informally. The labor-market evidence the paper marshals gives urgency to this curriculum agenda, and its grounding in durable practitioner discipline speaks to the broader question of what should be taught when AI can generate code. ## Connected Concepts - [[ai-literacy]] - [[scaffolding]] - [[prompt-engineering]] - [[curriculum-design]] - [[metacognition]] - [[agentic-ai]] ## Connected Articles - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing - [[guided-llm-scaffolding-independent-learning]] — Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm ## Citation Mikael Gorsky (2026). [ASE-26: A Curriculum for Agentic Software Engineering as a Discipline](https://arxiv.org/abs/2606.01152). arXiv:2606.01152. --- ## [Assessment in Team Problem-Solving Exercises in Computing Education](https://edtechdev.github.io/aied/articles/assessment-team-problem-solving-computing-education/) Tabletop exercises (TTXs) let learner teams rehearse high-stakes workplace tasks such as cybersecurity incident response, but their open-ended, collaborative nature makes [[formative-assessment]] difficult: teams often receive delayed or incomplete feedback. This full research-to-practice paper compares assessment methods that exploit the action and communication logs captured by TTX platforms to evaluate how well teams meet learning objectives. The work situates team problem-solving assessment within [[cs-education]] and broader [[stem-education]] contexts, showing how logged interaction data can drive faster, richer [[feedback-loop]]s than manual grading. By operationalising teamwork behaviours as measurable signals, it connects to [[learning-analytics]] and the [[student-experience]] of collaborative crisis-response training, with implications for [[higher-ed]] computing courses where TTXs are increasingly used but rubric reliability remains a barrier. ## Connected Concepts - [[formative-assessment]] - [[cs-education]] - [[stem-education]] - [[feedback-loop]] - [[learning-analytics]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[debugtracker-classroom-debugging]] — DebugTracker: Lightweight Process Evidence for Classroom Debugging - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments ## Citation Valdemar Švábenský, Jan Vykopal, Sukrit Leelaluk, Pavel Čeleda, et al. (2026). [Assessment in Team Problem-Solving Exercises in Computing Education](https://arxiv.org/abs/2607.19209). arXiv:2607.19209. arXiv:2607.19209 [cs.CY]. --- ## [ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots](https://edtechdev.github.io/aied/articles/astra-atco-training-simulator/) ASTRA uses autonomous AI sim-pilots for scalable air traffic control training; reduces dependency on human role-players while maintaining realistic scenario complexity. ## Synthesis ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots investigates astra uses autonomous ai sim-pilots for scalable air traffic control training; reduces dependency on human role-players while maintaining realistic scenario complexity. This work connects to existing research on intelligent-tutoring-systems by demonstrating that Abstract:Air Traffic Control Operators (ATCOs) are vital in ensuring the safe, orderly, and efficient flow of air traffic, yet training capacity is constrained by reliance on specialized human trainers known as simpilots, who must role-play both pilots and ATCOs in a simulated airspace. Existing aut.... ## Connected Concepts - [[reinforcement-learning]] - [[lifelong-learning]] - [[affective-tutoring]] - [[knowledge-tracing]] - [[socratic-ai-dialogue]] - [[open-source]] - [[pedagogical-llm-training]] - [[affective-computing]] ## Connected Articles - [[supplynet-visual-exploratory-learning]] — SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation - [[medeasy-ai-standardized-patients]] — MedEasy: Designing AI Standardized Patients for Clinical Consultation Training - [[ai-psychotherapy-training-avatars]] — Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[curiobot-llm-tutoring-exploratory-learning]] — Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior - [[ai-vocational-education-training-review]] — Artificial intelligence in vocational education and training: A systematic review of educational purposes, theoretical conceptualizations, and empirical effectiveness ## Citation Ethan Chew, Enjia Wu, Iruss Eng Wei Yeow, Ian Weiqin Lim, Ranen Sim, Brandon Koh Ziheng, Kaleb Nim, Caden Toh Jun Yi, Wei Dong Soin, Darius Kai Keat Koh, Galen King Yu Tay, Prannaya Gupta, Jonathan Ee Fang Koong, Yong Zhi Lim (2026). [ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots](https://arxiv.org/abs/2606.18319). arXiv:2606.18319. --- ## [Analysis and Prediction of At-Risk Students Using Machine Learning Algorithms](https://edtechdev.github.io/aied/articles/at-risk-students-ml-prediction/) Gheisari and Salarian (2026) apply supervised machine learning classification to identify at-risk students before they withdraw from higher education programs. The study evaluates Logistic Regression, Random Forest, Support Vector Machines (SVM), and K-Nearest Neighbors (KNN) using academic performance, demographic data, and enrollment records. Logistic Regression and linear SVM achieved the highest predictive accuracy, demonstrating ML's capability to detect at-risk students for proactive intervention. This [[learning-analytics]] research contributes to [[student-modeling]] for dropout-reduction in [[higher-ed]] contexts, providing a data-driven foundation for strategic retention decisions and connecting to the broader [[ai-adoption-training-public-sector]] discussion on AI-supported institutional decision-making. ## Connected Concepts - [[learning-analytics]] - [[student-modeling]] - [[higher-ed]] ## Connected Articles - [[ai-adoption-training-public-sector]] ## Citation Soheila Gheisari, Hamid Salarian (2026). [Analysis and Prediction of At-Risk Students Using Machine Learning Algorithms](https://arxiv.org/abs/2606.20617). arXiv:2606.20617. arXiv:2606.20617 (cs.CY; cs.LG) --- ## [Authentic Assessment](https://edtechdev.github.io/aied/articles/authentic-assessment/) > Wiggins (1990) proposed AA as a counterbalance to standardised tests: direct examination of "student performance on worthy intellectual tasks." > Authentic assessment (AA) has evolved from workplace-task replication toward a multi-dimensional framework encompassing professional, digital, personal, and social authenticity. The recent challenge by generative AI—which threatens the validity of any task that can be replicated à la Wiggins (1990)—makes AA's broader forms essential. Zhan, Boud & Du (2025) propose a six-dimensional design model that centres student agency and social collaboration, directly relevant to how AI assessment tools should be designed. ## The Evolution of Authenticity ### 1990s Origins: Worthy Intellectual Tasks Wiggins (1990) proposed AA as a counterbalance to standardised tests: direct examination of "student performance on worthy intellectual tasks." ### Late 1990s HE Uptake: Workplace Replication Joughin (1998) argued authenticity should reflect the extent to which assessment replicates professional practice or real life. This view dominated for two decades. ### 2020s Critique: Beyond Replication McArthur (2023) contends AA must enable students to "**influence the future and transform society**" rather than merely replicate existing tasks. Ajjawi et al. (2024) broaden authenticity to contextual, task, and personal forms that reflect student experience. ### Generative AI as Existential Challenge Generative AI makes traditional workplace-replication AA newly vulnerable: any task that a language model can credibly simulate in a take-home setting loses its validity as an assessment of original student competence. The field must pivot toward forms of authenticity (digital literacy, real-time collaboration, social contribution, individual meaning-making) that AI cannot credibly counterfeit. ## Six-Dimensional Framework (Zhan et al., 2025) This scoping review of 37 empirical AA studies (2000–2024) proposes six design dimensions, moving beyond earlier frameworks (Gulikers et al., 2004; Villarroel et al., 2018): 1. **Authenticity in assessment** — multiple meanings: assessment authenticity (portfolios, projects, concept maps), professional authenticity (workplace scenarios), digital authenticity (Twitter, podcasts, YouTube, LMS), self-authenticity (student identity, well-being), and social authenticity (citizenship, sustainability, ethics). Only **3 of 37 studies** addressed social authenticity — a critical gap. 2. **Cognitive challenges** — knowledge construction (n=29), professional skills (n=22), and 21st-century skills (n=29, led by critical thinking n=17, communication n=13). Digital literacy: only **n=5**. 3. **Assessment criteria** — rubric use was common (n=22) but most students were **passive recipients** rather than co-authors. Only 3 studies co-designed rubrics with students; only 7 involved students as assessors via self/peer assessment. 4. **Feedback** — formative feedback dominated (n=23), summative was common (n=12), but **sustainable feedback** (transferable to future contexts) appeared in only 4 studies. This mirrors the field-wide problem that AI tools also replicate: reactive, momentary feedback rather than lifelong evaluative judgement. 5. **Student agency** — choices about what/how/when/where to submit appeared in only **n=8 studies**. Self-reflection was more common but often **assigned/graded**, making it potentially **performative** (instrumental rather than genuine). 6. **Social collaboration** — mostly individual tasks (n=18) or group tasks (n=16), with few mixing both (n=3). Peer collaboration strategies (peer assessment, peer discussion) appeared in n=16 studies; teacher–student collaboration in n=16, though only 3 designed *equitable* teacher–student partnership (roles were usually feedback-giver, monitor, facilitator — a power imbalance); external industry/community connections in only n=5. Social construction of assessment meaning was under-theorized but present. ## AI-Specific Implications ### What AI Assessment Tools Get Wrong Current AI assessment systems — MCQ generators ([[formative-assessment|CODE-GEN]]), essay scorers (MASS), short-answer graders — focus on **efficiency and standardization**, replicating the very limitations Zhan et al. identify: - **Rubric-centric:** AI systems typically generate pre-defined rubrics without student co-design, replicating the "passive recipient" problem - **Momentary feedback:** AI formative feedback is abundant but rarely designed as **sustainable evaluative judgement** - **Professional authenticity bias:** Most AI-generated assessments simulate workplace or academic tasks, neglecting personal and social authenticity - **Choicelessness:** AI assessment systems rarely allow students to define assessment parameters, output formats, or evaluation criteria ### The Four-Step Collaborative Design Framework Zhan et al. propose a cyclical design model that AI tools could operationalize: | Step | Action | AI Enabler | AI Risk | |------|--------|------------|---------| | 1. Decide goals | Students + educators **co-negotiate** purpose and authenticity | LLM-facilitated dialogue tools | Over-optimizing for what's easy to grade | | 2. Create context | Design real-world scenarios | RAG-augmented scenario generation | Hallucinating false domain contexts | | 3. Design criteria | **Co-design rubrics** with students | Collaborative rubric editors | Imposing opaque algorithmic criteria | | 4. Plan feedback | Future-oriented, sustainable feedback | LLM personalization based on learner profiles | Surveillance-level behavior tracking | ## Connections to AI Education Research ### Self-Regulated Learning Student agency in AA (choice, self-reflection, co-design) is isomorphic to the [[self-regulated-learning|forethought → performance → self-reflection]] cycle. However, when self-reflection is **graded**, it becomes **performative** — students write to impress assessors rather than to learn. AI journaling tools face the same instrumentalization risk. ### Metacognitive Calibration [[metacognition|Metacognition]] is required for students to evaluate their own work against co-designed rubrics. When AI provides the rubric, generates the feedback, *and* monitors progress, the student's metacognitive practice is displaced — the very suppression risk identified in [[ai-tutor-safety-harms|SafeTutors]] and [[llm-fallacy-misattribution|LLM Fallacy]] research. ### Pedagogical Training Theory-grounded training (see [[pedagogical-llm-training|ISD-Agent-Bench, EduQwen]]) should explicitly align with the six-dimensional framework. A model trained to reward "guiding over answering" still falls short if it does not understand **sustainable feedback**, **co-designed rubrics**, or **social authenticity**. ### Adaptive Systems [[adaptive-learning|Adaptive systems]] that personalize only *content difficulty* miss the personalization of *assessment authenticity*. DeepTutor's multi-resolution memory and MAIC's archetype agents begin to address this, but neither incorporates student co-design of assessment parameters. ## Open Questions 1. **AI-proof assessment types:** Which forms of authentic assessment are robust to generative AI? In-vivo demonstrations, social contribution portfolios, co-created artefacts with auditable provenance chains, and assessments requiring real-time embodied interaction may be more resilient than take-home essays or MCQs. 2. **Student co-design at scale:** Zhan et al. show co-design is rare (3/37 studies). Can AI tools *enable* rubric co-design at classroom or MOOC scale, or does the paradox of machine-mediated human agency undermine the authenticity itself? 3. **Sustainable feedback via LLM:** Can a language model deliver feedback that students apply months later? The CDPK and ISD benchmarks test pedagogical knowledge transfer to models, not feedback sustainability transfer to students. 4. **Social authenticity deficit:** Only 3 studies addressed social issues (citizenship, sustainability, ethics). How can AI assessment tools help students contribute to societal transformation rather than merely simulate it? ## Connected Concepts - [[adaptive-learning]] - [[cognitive-load-theory]] - [[desirable-difficulties]] - [[faculty-development]] - [[formative-assessment]] - [[human-in-the-loop-ai]] - [[metacognition]] - [[pedagogical-llm-training]] - [[self-regulated-learning]] - [[zone-of-proximal-development]] - [[ai-ed-evaluation]] - [[ai-education]] - [[higher-ed]] ## Connected Articles - [[agentic-workflows-education]] — Agentic Workflows in Education - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[collaborative-ai-tutoring]] — Collaborative AI Tutoring - [[educational-llm-alignment]] — Educational LLM Alignment - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[moral-panic-genai-classroom]] — Navigating the moral panic: encouraging appropriate use of GenAI in the classroom rather than condemning innovation a... - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[principled-ai-education]] — Principled AI in Education - [[student-cheat-sheets-make-or-take]] — Make or Take: How Students Navigate Self-Created and Instructor-Provided Cheat Sheets - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development ## Citation Zhan, Y., Boud, D., & Du, Z. (2025). [*Designing for authentic assessment: a scoping review*](https://doi.org/10.1007/s10734-025-01588-9). *Higher Education*. --- ## [From authentic products to authenticated processes: authentic assessment in AI-rich higher education](https://edtechdev.github.io/aied/articles/authentic-products-authenticated-processes-2026/) > Generative AI has not created the need for authentic assessment — it has made weaknesses in assessment design harder to ignore. Polished products can now be generated or substantially mediated by tools, so **product resemblance is an increasingly unreliable signal of capability**. Tsiligkiris calls this risk **construct substitution**: an AI-generated or AI-mediated product is attributed to the st > Tsiligkiris (2026) reframes authentic assessment as an *evidential and validity-oriented design problem* in AI-rich higher education. His central distinction — **authentic products vs. authenticated processes** — argues that assessment validity under generative AI depends not on realistic outputs alone but on architectures that make human judgement, verification, and responsibility visible. A systematic conceptual review of 37 sources yields a six-dimension framework for redesigning assessment briefs at module and programme level. ## The core argument Generative AI has not created the need for authentic assessment — it has made weaknesses in assessment design harder to ignore. Polished products can now be generated or substantially mediated by tools, so **product resemblance is an increasingly unreliable signal of capability**. Tsiligkiris calls this risk **construct substitution**: an AI-generated or AI-mediated product is attributed to the student, producing an inference about capability that reflects the tool's performance rather than the student's learning. The evidence is concrete: fully AI-generated submissions passed through a live university examination system largely undetected (Scarfe et al., 2024), and experienced markers do not reliably detect GenAI-authored work (Kofinas, Tsay & Pike, 2025). Crucially, the evidential question survives any AI policy: whether AI use is **prohibited, permitted, or required**, the assessment must still generate evidence that warrants the inference being drawn — either about unaided capability, or about the capacity to direct, critique, and take responsibility for AI-supported work. ## Six-dimension framework 1. **Contextual fidelity and consequential relevance** — realistic problems, roles, constraints, audiences, artefacts; credibility to disciplinary or civic practice *plus* an intelligible reason to care (stakeholder, public output, policy brief), not surface imitation. 2. **Cognitive demand and evaluative judgement** — analyse, evaluate, synthesise, interpret ambiguity, make defensible trade-offs. Evaluative judgement (Sadler 1989; Tai et al. 2018) matters more when machines can generate plausible first drafts. 3. **Process transparency and assessment integrity** — reasoning, iteration, feedback use, and verification visible enough to support warranted inference (Boud 2000; Kane 2013). Architectures: staged submissions, annotated decision rationales, oral defence, feedback-use statements, process records. 4. **Student agency and bounded choice** — topic/modality/case/medium choice within a common architecture, bounded by clear standards to preserve comparability and fairness. 5. **Inclusivity and representational fairness** — realism can privilege particular communication styles, professional norms, and cultural capital; needs transparent criteria, scaffolding, equivalent routes to demonstrate achievement, and attention to whose realities are represented. 6. **AI-aware validity and ethical practice** — specify how AI relates to intended outcomes, permitted uses, and the evidence supporting defensible attribution of capability; evidential requirement (does the task still capture the construct?) plus ethical requirement (disclose, critique, verify, take responsibility). ## Operational tool The framework becomes a **review instrument for assessment briefs**: teams examine each dimension for evidence generated, validity risks, and redesign priorities — using a 4-point indicative alignment scale (weak → partial → substantial → strong). Not every assessment must maximise all six dimensions; across a programme, tasks may emphasise different ones. It complements (rather than replaces) the [[ai-ed-evaluation|AI Assessment Scale]] by treating AI permissions as part of a defensible assessment argument, alongside fairness, agency, cognitive demand, and process evidence. ## Connected Concepts - [[ai-ed-evaluation]] - [[ai-literacy]] - [[assessment-validity]] - [[formative-assessment]] - [[generative-ai]] - [[higher-ed]] - [[llm]] ## Connected Articles - [[authentic-assessment]] — Authentic Assessment - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform ## Citation Tsiligkiris, V. (2026). [*From authentic products to authenticated processes: a systematic conceptual review of authentic assessment in AI-rich higher education*](https://doi.org/10.1080/02602938.2026.2695376). *Assessment & Evaluation in Higher Education*. --- ## [The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences](https://edtechdev.github.io/aied/articles/automated-formative-assessments-a-level-sciences/) This quasi-experimental mixed-methods longitudinal study (N=142) deploys a fully automated marking pipeline for handwritten mock examinations in A-Level sciences, removing the human-marking bottleneck that normally caps the frequency of formative mocks. High-frequency, automatically-marked [[formative-assessment]] cycles were associated with improved student outcomes, providing field evidence for the pedagogical payoff of [[automated-grading]] at upper-secondary level. The handwritten-work pipeline links to [[llm-handwritten-math-grading]] and [[automatic-short-answer-grading]], while the human-in-the-loop trade-offs echo [[hybrid-e-assessment-semi-automated-grading]]. ## Connected Concepts - [[formative-assessment]] - [[automated-grading]] ## Connected Articles - [[llm-handwritten-math-grading]] - [[automatic-short-answer-grading]] - [[hybrid-e-assessment-semi-automated-grading]] ## Citation Matey Yordanov, Mikhail Bychkov, Andrei Kuchma (2026). [The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences](https://arxiv.org/abs/2607.23566). arXiv:2607.23566. --- ## [Automated Grading of Linux/Bash Examinations Using Large Language Models](https://edtechdev.github.io/aied/articles/automated-grading-linux-bash-examinations-large-language-models/) **Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira (2026)** This paper presents an [[llm]]-based grading system for Linux/bash command-line examinations, applying a four-level cognitive taxonomy to assess student work in programming courses. The system addresses the scalability challenge of rising enrolments by providing [[automated-grading]] that captures both partial correctness and conceptual understanding. The approach demonstrates high agreement with human graders, suggesting a viable path toward [[formative-assessment]] at scale. Results indicate that LLMs can evaluate command-line proficiency more nuancedly than traditional rule-based autograders, offering detailed [[ai-feedback-quality]] feedback for students. ## Connected Concepts - [[llm]] - [[automated-grading]] - [[formative-assessment]] - [[ai-feedback-quality]] ## Connected Articles - [[evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations]] — Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations ## Citation Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira (2026). [Automated Grading of Linux/Bash Examinations Using Large Language Models](https://arxiv.org/abs/2607.02432). arXiv:2607.02432. --- ## [A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges](https://edtechdev.github.io/aied/articles/automated-presentation-coaching/) This survey provides the first systematic review of automated presentation coaching systems, organizing them along a five-dimensional task taxonomy: segmental pronunciation, lexical stress, suprasegmental prosody, pacing, and content faithfulness. The authors review systems spanning pronunciation tutors, fluency and prosody coaches, multimodal trainers, and conference Q&A practice tools, identifying coverage gaps in pacing and content feedback dimensions. The survey connects to [[intelligent-tutoring|intelligent tutoring systems]] by examining how real-time [[feedback-loop|feedback delivery]] mechanisms in coaching systems mirror educational feedback principles. The taxonomy serves as a framework for [[professional-training|professional skills training]], particularly for [[language-learning|second language learners]], and highlights the role of [[scaffolding|scaffolded feedback]] in skill development. Key open challenges include the scarcity of annotated presentation corpora, achieving accent-fair feedback across diverse L1 backgrounds, and delivering real-time multimodal feedback without overwhelming the learner — all of which connect to broader [[personalized-learning|personalized learning]] and [[ai-literacy|AI literacy]] concerns in education technology design. ## Connected Concepts - [[intelligent-tutoring]] - [[feedback-loop]] - [[professional-training]] - [[language-learning]] - [[scaffolding]] - [[personalized-learning]] - [[ai-literacy]] ## Connected Articles - [[prompt-coach-agentic-tutor-prompt-engineering]] — Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents ## Citation Wen Liang, Li Siyan, Zackary Rackauckas, Julia Hirschberg (2026). [A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges](https://arxiv.org/abs/2606.27380). arXiv:2606.27380. Computation and Language (cs.CL). --- ## [Automatic Short Answer Grading with LLMs](https://edtechdev.github.io/aied/articles/automatic-short-answer-grading/) > Automatic Short Answer Grading (ASAG) is never perfect. Upper bounds on accuracy arise from: > Zero-shot LLMs perform strongly on ASAG without task-specific fine-tuning, but **model-based confidence signals alone are insufficient** for reliable uncertainty quantification. A hybrid framework combining epistemic (model-derived) and aleatoric (dataset-derived) uncertainty significantly improves selective grading and calibration.^[[cong-confidence-asag-2026]] ## Why Confidence Matters Automatic Short Answer Grading (ASAG) is never perfect. Upper bounds on accuracy arise from: - Linguistic variability in student responses - Ambiguity in scoring criteria - Inherent disagreement among human graders Reliable confidence estimates are therefore essential for **human-in-the-loop review**—flagging uncertain cases for teacher attention rather than auto-grading everything.^[[cong-confidence-asag-2026]] ## Three Model-Based Confidence Strategies All generated with `gpt-oss-20b` at temperature 0.1: | Strategy | Mechanism | Edge Case Failure | |---|---|---| | **Verbalizing** | Prompt model to self-report confidence in [0,1] | Overconfident on ambiguous but plausible phrasings | | **Latent** | Softmax log-likelihood of predicted label token | Underconfident when label token probabilities are flat | | **Consistency** | Sample 5 predictions at varying temperatures; measure agreement | Fails when model is consistently wrong | All three, when used in isolation, miss **aleatoric uncertainty**—ambiguity intrinsic to the data itself.^[[cong-confidence-asag-2026]] ## Aleatoric Uncertainty via Semantic Clustering To capture data-level ambiguity: 1. **Embed** student responses with `all-MiniLM-L6-v2` 2. **Cluster** via agglomerative hierarchical clustering (Ward linkage) on a 10% calibration subset 3. **Quantify** within-cluster label heterogeneity via normalized Shannon entropy 4. **Assign** test responses to nearest cluster centroid High-entropy clusters indicate inherently ambiguous responses (e.g., partially correct, underspecified, or stylistically unusual).^[[cong-confidence-asag-2026]] ## Hybrid Confidence Fusion Feature vector per response: `[s^verb, s^lat, s^cons, s^alea, token_length]` - **Classifier:** Random Forest (500 trees) trained on human labels - **Calibration:** Platt scaling via 5-fold CV to yield calibrated probability p ∈ [0,1] **Result:** Hybrid confidence with aleatoric uncertainty outperforms all single-source approaches on selective prediction metrics (AUROC, AUARC) and reliability metrics (ECE, MCE, Brier).^[[cong-confidence-asag-2026]] ## Implications for Human-in-the-Loop Assessment - **Selective grading:** Flag low-confidence responses for human review; auto-grade high-confidence responses - **Teacher dashboards:** Display confidence alongside grades to support triage - **Fairness:** Aleatoric uncertainty may reveal systemically ambiguous scoring criteria that need rubric revision ## Relationship to Tutoring-Specific Design ASAG confidence estimation is complementary to [[tutoring-specific-vs-general-ai|tutoring-specific feedback]]: while tutoring tools generate formative scaffolds, grading tools must manage summative uncertainty. Both require explicit epistemic humility—acknowledging when the model should defer to human judgment.^[[cong-confidence-asag-2026]] ## Connected Concepts - [[automated-essay-scoring]] - [[human-in-the-loop-ai]] - [[ai-feedback-quality]] - [[cs-education]] - [[administrator]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[ai-ed-evaluation]] ## Connected Articles - [[cong-confidence-asag-2026]] - [[tutoring-specific-vs-general-ai]] ## Citation Cong, L., Hahn, S., Gombert, S., Camus, L., Drachsler, H., & Kroehne, U. (2026). [*Confidence Estimation in Automatic Short Answer Grading with LLMs*](https://arxiv.org/abs/2605.00200). arXiv:2605.00200. --- ## [Awareness of Technological Isomorphism: AI in Elementary Math](https://edtechdev.github.io/aied/articles/awareness-technological-isomorphism/) Introduces a novel core concept, **"Awareness of Technological Isomorphism,"** defined as a student's metacognitive realization that their own mathematical cognitive operations (observing trends, inducing patterns, making predictions) share an underlying logical structure with AI technical operations (pattern recognition, predictive modeling). This awareness facilitates [[transfer-of-learning|cognitive transfer from disciplinary mathematics to AI comprehension]]. The paper proposes a **three-stage pedagogical pathway** — Perception → Comprehension → Creation — validated through a case study on compound line graphs with fifth-grade students in China. This framework bridges elementary [[stem-education|mathematics education]] with [[ai-literacy|AI literacy]], offering a replicable model for deep convergence of disciplinary instruction and AI understanding. The concept is distinguished from computational thinking: it focuses on the isomorphic interface between domain knowledge and AI operations rather than programming skills. **Key contribution:** Provides a grounded, theory-based framework for helping elementary students see structural parallels between their own reasoning and AI, moving beyond general computational thinking toward metacognitive awareness of structural isomorphism. Empirical validation through a concrete fifth-grade lesson on data trends and prediction. ## Connected Concepts - [[stem-education]] - [[ai-literacy]] ## Connected Articles - [[transfer-of-learning]] ## Citation Li, L., & Cao, Y. (2026). Awareness of Technological Isomorphism: Integrating AI into Elementary Mathematics Teaching on Data and Prediction — A Case Study of the Compound Line Graph. [arXiv:2606.09598](https://arxiv.org/abs/2606.09598). --- ## [AICoFE: AI-Powered Feedback System](https://edtechdev.github.io/aied/articles/becerra-aicofe-feedback-2026/) > **AICoFE** (AI-based Collaborative Feedback) is a multi-LLM feedback generation system for higher education that combines independently fine-tuned language models with **teacher-in-the-loop mediation**, producing diverse feedback perspectives while preserving pedagogical authority through Learning Analytics dashboards. ## Key Findings 1. **Multi-LLM diversity improves feedback quality.** AICoFE uses three independently fine-tuned LLMs (GPT-4.1-mini, Gemini 2.5 Flash, Llama 3.1) to generate diverse feedback perspectives from the same input data — quantitative rubric scores, validated qualitative observations, rubric level descriptions, and instructional materials. Each model produces independent drafts that the teacher can inspect and curate. 2. **Teacher mediation is central, not an afterthought.** Rather than automating feedback end-to-end, the system positions educators as active curators. The teacher dashboard enables instructors to compose final feedback by **selecting individual sentences or paragraphs** from AI outputs, with a visual legend indicating the proportion of content contributed by each LLM. This preserves pedagogical judgment while reducing feedback workload. 3. **Dual-purpose transparency supports both practice and research.** The system tracks the extent of teacher modification and curation, serving simultaneously as (a) a reflection tool for individual instructors examining their curation patterns, and (b) a dataset for large-scale analysis of the teacher mediation role in AI-assisted feedback. 4. **Role-specific dashboards close the feedback loop.** The student dashboard provides access to video recordings of presentations, self-evaluation rubrics, visual comparisons of self vs. external evaluations, teacher-curated AI feedback, and the ability to rate perceived agreement and usefulness — completing a full assessment-for-learning cycle. ## Implications AICoFE addresses a critical tension in [[ai-feedback-quality]]: the trade-off between efficiency and pedagogical authority. By adopting a [[human-in-the-loop-ai]] architecture rather than full automation, it demonstrates that AI can accelerate feedback workflows without displacing the teacher's interpretive role. This aligns with emerging best practices in [[ai-feedback-quality]] research, which increasingly emphasizes teacher agency over raw automation. The multi-LLM pipeline is a practical response to the observation that different models excel at different dimensions of feedback — tone, specificity, actionability. Rather than selecting a single "best" LLM, AICoFE treats model diversity as a feature, surfacing complementary perspectives that the teacher can synthesize. This approach parallels work in [[ai-peer-feedback-systems]] that leverages multiple AI-generated perspectives for richer formative assessment. The [[learning-analytics]] component of AICoFE is notable for its role-specific design: teacher and student dashboards serve fundamentally different functions within the same platform, and the system's transparency features (source attribution, curation tracking) support both reflective practice and [[learning-analytics]] at scale. For [[higher-ed]] institutions facing growing assessment loads, AICoFE models a path where AI-assisted feedback scales without sacrificing the relational and interpretive dimensions of effective [[formative-assessment]]. ## Connected Concepts - [[formative-assessment]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[learning-analytics]] - [[ai-ed-evaluation]] - [[student-experience]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams ## Citation Becerra, Á., Palma, A., & Cobos, R. (2026). [*AICoFe: AI-Based Collaborative Feedback System for Higher Education*](https://arxiv.org/abs/2605.04740). arXiv:2605.04740. --- ## [Pragmatic users and skeptical nonusers: A qualitative typology of ChatGPT adoption in physics education](https://edtechdev.github.io/aied/articles/becker-chatgpt-typology-physics-2026/) > **Synthesis:** Becker, Bauer, Schrader, Bitzenbauer & Veith (2026) analyze 1,189 survey responses from physics students using qualitative content analysis and latent class analysis, identifying two distinct user profiles: 70% are "Pragmatic Users" who use ChatGPT for scaffolding despite awareness of inaccuracies, and 30% are "Skeptical Non-Users" who avoid it over overreliance concerns. Both groups make calculated risk-utility trade-offs — challenging one-size-fits-all AI policies and calling for differentiated pedagogy. ## Summary This study develops a data-driven typology of physics students' ChatGPT use through qualitative content analysis of 1,189 survey responses and latent class analysis (LCA). Two profiles emerge: Pragmatic Users (70%) who are aware of AI's inaccuracies but use it for conceptual clarification and problem-solving scaffolds, and Skeptical Non-Users (30%) who avoid the tool over concerns about overreliance and its potential to hinder independent problem-solving. Both groups demonstrate calculated risk-utility trade-offs rather than uncritical adoption or blanket rejection — challenging institutional one-size-fits-all AI policies. ## Key Findings - Qualitative content analysis of 1,189 responses + latent class analysis - Two profiles: 70% Pragmatic Users, 30% Skeptical Non-Users - Pragmatic Users use ChatGPT for conceptual clarification, finding starting points, scaffolds — despite awareness of inaccuracies - Skeptical Non-Users avoid ChatGPT over overreliance/independent problem-solving concerns - Both groups make calculated trade-offs; neither is uncritical - Calls for differentiated pedagogy respecting both profiles ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[student-experience]] - [[physics-education]] - [[rag]] - [[scaffolding]] ## Connected Articles - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] - [[critical-genai-use-predictors]] ## Citation Becker, E., Bauer, A., Schrader, J., Bitzenbauer, P., & Veith, J. (2026). [Pragmatic users and skeptical nonusers: A qualitative typology of ChatGPT adoption in physics education](https://doi.org/10.1103/gd1w-1637). --- ## [Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery](https://edtechdev.github.io/aied/articles/behaviorally-adaptive-visual-diversion-assessment-2026/) > **Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery** — Proposes BAVD, a theoretical framework for adaptive visual diversion in digital assessment that resists screen-capture cheating while accommodating learners with visual-processing accommodations. Formulates the model using coupled dynamical systems (... [[assessment]] [[accessible-learning]] [[privacy]] [[academic-integrity]] [[equity]] [[adaptive-learning]] Proposes BAVD, a theoretical framework for adaptive visual diversion in digital assessment that resists screen-capture cheating while accommodating learners with visual-processing accommodations. Formulates the model using coupled dynamical systems (Diversion Field Generator, Rendering Tensor, Behavior Tensor, Multi-dimensional Entropy Model). Establishes theoretical properties for content fidelity, rendering stability, entropy boundedness, and closed-loop adaptation stability. Explicitly addresses the trade-off between accessibility and capture resistance. ## Abstract Institutions increasingly rely on browser lockdown, webcam monitoring, and behavioral analytics to secure high-stakes digital assessments, yet these mechanisms are commonly designed and evaluated independently and often overlook learner accessibility. This paper introduces Behaviorally-Adaptive Visual Diversion (BAVD), a theoretical framework in which a synthetic, non-semantic visual field is composited with assessment content and adaptively modulated according to observed candidate behavior. The underlying assessment content is never altered; only its visual presentation is modified to reduce the usefulness of unauthorized screen capture or screen sharing while remaining minimally intrusive for legitimate candidates. ## Connected Concepts - [[assessment]] - [[accessible-learning]] - [[privacy]] - [[academic-integrity]] - [[equity]] - [[adaptive-learning]] ## Connected Articles - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-literacy-career-adaptability-business-2026]] — AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[ai-assessment-scale-reform]] — A bit of chaos and madness: The AI Assessment Scale and the work of assessment reform ## Citation Gupta Lovi Raj, Kamalpreet Kaur, Dama Sriram, & Parali Prajithaa (2026). [Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery](https://arxiv.org/abs/2608.03531). arXiv:2608.03531. arXiv:2608.03531 [cs.AI]. --- ## [Beyond Detection: redesigning authentic assessment in an AI-mediated world](https://edtechdev.github.io/aied/articles/beyond-detection-authentic-assessment-ai-2025/) > Detection-led responses face well-documented limits: validity and fairness failures (bias against non-native writers), notable error rates, erosion of trust, and distraction from assessment design. Detection should be a **limited, situational tool — not a strategy of first resort**. The constructive question is not "how do we prevent students from using AI?" but "how do we enable them to use it th # Beyond Detection: authentic assessment in an AI-mediated world > Kickbusch, Ashford-Rowe, Kemp, Boreland & Huijser (2025) argue the dominant institutional response to generative AI in assessment — surveillance and AI detection — **misdiagnoses the problem**: in an AI-mediated world, authenticity cannot be policed into existence; it must be redesigned. They reconceptualise authenticity as constructed where AI is expected, declared, and scrutinised, and offer discipline-agnostic "design for learning" patterns that position AI as a collaborator rather than a cheating application. ## The case against detection Detection-led responses face well-documented limits: validity and fairness failures (bias against non-native writers), notable error rates, erosion of trust, and distraction from assessment design. Detection should be a **limited, situational tool — not a strategy of first resort**. The constructive question is not "how do we prevent students from using AI?" but "how do we enable them to use it thoughtfully, responsibly, and effectively in contexts that mirror their future work?" Excluding AI from assessment creates an *inauthentic* scenario: the authentic professional justifies when, how, and why they use tools, and critically evaluates their outputs. ## Authenticity as a four-dimensional continuum Not a binary but a continuum across four intersecting dimensions: 1. **Task–context alignment** with contemporary professional practice — judgement, decision-making, and problem-solving under uncertainty, not superficial workplace replication 2. **Foregrounding professional judgement and ethics** — sustainable assessment (Boud & Soler 2016), collaboration (Boud & Bearman 2024), UNESCO 2023 capability framing 3. **Visibility of process** — iteration, critique, rationale; polished outputs can mask superficial understanding, so assessment must reveal the "messiness" of authentic professional work 4. **Appropriate use of tools** (including AI) within human decision-making — tools as enablers of higher-order capability, not substitutes for it Stage-appropriate authenticity: early units get constrained, well-scaffolded tasks; later units open complexity, uncertainty, and stakeholder engagement. ## Design patterns ("design for learning moves") - **Critique, adapt, verify AI outputs**: business students interrogate a chatbot-generated market analysis; pre-service teachers evaluate AI-produced lesson plans for inclusivity and pedagogical soundness; journalism students edit an AI news brief to identify bias; health students appraise AI diagnostic recommendations - **Process transparency artefacts**: process logs, AI prompt records, drafts showing iterations — "behind the scenes" evidence submitted alongside the final output - **Reflective commentaries**: explain key decisions, justify tool use, account for changes, with explicit criteria for depth, criticality, and ethical awareness - **Oral defences / annotated portfolios / recorded walkthroughs**: probe reasoning in real time, mirroring professional practices like pitching and peer review - **Self-critique and peer feedback** for feedback literacy (Boud & Molloy 2012) - **Progressive release across a programme**: transparency artefacts + short defences → collaboration and negotiated briefs → capstones with external stakeholders and negotiated criteria ## Challenges and institutional responsibilities - **Equity**: unequal access to tools deepens divides; institutional provision (fenced AI deployments) reduces back-channel inequality; authentic formats can create *new* barriers (workload, carer/employment constraints) — mitigate with workload modelling, staged scaffolding, modality choice - **Ethics and bias**: tools reproduce cultural stereotypes and can be fluent yet unfaithful (Bender et al. 2021); institutions should run privacy/data-protection impact assessments (PIA/DPIA) for assessment AI, vet tools against privacy/bias/accessibility criteria, and standardise prompt-log conventions that evidence process without exposing personal data - **Load and feasibility**: process artefacts and defences raise workload; needs modelling and scaffolds - **Staff development**: design-led collaboration rather than superficial tool training ## Connected Concepts - [[ai-literacy]] - [[assessment-validity]] - [[metacognition]] - [[self-regulated-learning]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[authentic-assessment]] — Authentic Assessment - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis ## Citation Kickbusch, S., Ashford-Rowe, K., Kemp, A., Boreland, J., & Huijser, H. (2025). Beyond Detection: Redesigning Authentic Assessment in an AI-Mediated World. [Education Sciences](https://doi.org/10.3390/educsci15111537), 15(11), 1537. DOI --- ## [Beyond MOOCs: How technical and structural factors shape learner engagement, retention and inclusivity across online learning platforms](https://edtechdev.github.io/aied/articles/beyond-moocs-how-technical-and-structural-factors-shape-learner-engagement-reten/) > **Synthesis:** This study examines the critical influence of technical and structural factors on learner Engagement, Retention and Inclusivity (ERI) in MOOCs and other large-scale online learning platforms. Using a novel mixed-methods 2TS method, analysis of over 226,000 user reviews from six platforms (Coursera, edX, Udemy, Alison, uLesson, Khan Academy) found that technical instability, limited offline functionalities and unclear pricing structures negatively impact learner engagement. Regional payment restrictions disproportionately disadvantage learners in low-resource settings. ## Key Findings This study examines the critical influence of technical and structural factors on learner Engagement, Retention and Inclusivity (ERI) in MOOCs and other large-scale online learning platforms. Using a novel mixed-methods 2TS method, analysis of over 226,000 user reviews from six platforms (Coursera, edX, Udemy, Alison, uLesson, Khan Academy) found that technical instability, limited offline functionalities and unclear pricing structures negatively impact learner engagement. Regional payment restrictions disproportionately disadvantage learners in low-resource settings. ## Connected Concepts - [[motivation]] - [[learning-analytics]] - [[rag]] - [[intelligent-tutoring]] - [[adult-learning]] - [[socratic-method]] - [[math-education]] - [[higher-ed]] ## Connected Articles - [[learning-by-chatting-genai-impact]] - [[genai-meta-analysis-programming-learning]] - [[ai-in-the-wild-college]] - [[agents-that-teach-incidental-learning]] ## Citation Ogunsakin, R., Okonu, D., & Nwokeocha, P. (2026). [*Beyond MOOCs: How technical and structural factors shape learner engagement, retention and inclusivity across online learning platforms*](https://doi.org/10.1111/bjet.70066). *British Journal of Educational Technology.* --- ## [From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs](https://edtechdev.github.io/aied/articles/bloom-aligned-educational-control-llms/) Introduces a Bloom-aligned framework for measuring 'educational control' in LLMs: the ability to preserve a task's instructional intent while shifting its cognitive demand toward higher-order Bloom levels, offering a metric for evaluating whether AI assistance scaffolds or shortcuts learning. The work connects to broader debates about how [[generative-ai]] systems reshape [[student-experience]] and the conditions under which AI support [[scaffolding|scaffolds]] rather than undermines learning. It has direct implications for [[pedagogy-ai-mistakes]] and the risk of [[over-reliance]] when assistants absorb too much of the cognitive load. Findings also bear on [[ai-literacy]] and [[self-regulated-learning]], and on how institutions should govern [[student-experience]] and [[academic-integrity]]. Practitioners in [[higher-ed]] and [[teacher-role|teachers]] can use the evidence to calibrate when to deploy [[llm]]-based help and how to pair it with [[feedback-loop|feedback]] that preserves learning gains. ## Connected Concepts - [[generative-ai]] - [[student-experience]] - [[scaffolding]] - [[over-reliance]] - [[ai-literacy]] - [[self-regulated-learning]] - [[academic-integrity]] - [[higher-ed]] - [[teacher-role]] - [[llm]] - [[feedback-loop]] ## Connected Articles - [[pedagogy-ai-mistakes]] ## Citation S. Bekkouch, T. Constantinou, M. Ovaere, et al. (2026). [From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs](https://arxiv.org/abs/2607.08009). arXiv:2607.08009. --- ## [Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training](https://edtechdev.github.io/aied/articles/book-level-synthetic-textbook-organization/) Studies how organizing synthetic content into coherent book-level documents affects language model training, moving beyond local rewriting. Presents a scalable synthesis pipeline that retrieves source material, clusters it into topical units, and plans hierarchical textbook structures. Shows book-level organization significantly outperforms isolated content generation for educational knowledge acquisition in LLMs. ## Key Findings - Prior work on synthetic textbooks treated the benefit as a property of generated content or local rewriting style; this paper studies a different factor — whether related content is organized into coherent book-level documents — and provides controlled evidence that organization matters. - The synthesis pipeline retrieves source material from a pre-training corpus, clusters it into topical units, plans hierarchical tables of contents, and assembles source-grounded sections into complete books, yielding 686K textbooks (32B tokens) across 15,000+ disciplines. - Replacing natural books in a mid-training mix with this corpus improves downstream performance by +1.09 on average. - A content-matched Split condition that treats each section as an independent document shows a +1.02 mean gain for Full, isolating document packaging as the active factor. - A length-matched RandomConcat control that joins sections from different books performs below Full, ruling out document length alone as the explanation. - A retrieval-pool-matched Rephrase condition that independently rewrites individual documents without clustering, TOC planning, or book assembly yields a +1.17 gain for Full, demonstrating the value of structured synthesis. - On Llama3-8B, Full likewise outperforms both RandomConcat and Natural Books, supporting book-level organization as a useful axis for synthetic pre-training data design. ## Why Organization Matters The authors argue that organization is not incidental. On the generation side, a book-level plan provides the scaffold needed to turn scattered retrieved materials into long, structured, and coherent textbook data rather than a collection of locally plausible passages. On the training side, preserving structure determines whether planned adjacent sections remain in a shared document with continuous positions and shared intra-document attention, or are split into independent examples with resets between sections. ## Implications for AI in Education For educational applications of [[llm]]s, the results indicate that the structure of training data is a design axis in its own right: pedagogically organized, book-level documents improve knowledge acquisition over isolated or locally rewritten content. The pipeline is scalable and source-grounded, offering a practical recipe for generating educational corpora at scale — of interest to researchers building synthetic textbooks for mid-training and to efforts that use [[generative-ai]] to produce learning materials. The controlled-comparison methodology also models how to isolate specific data-design factors, a discipline relevant to [[educational-nlp]] research on training data for education. ## Connected Concepts - [[llm]] - [[generative-ai]] - [[educational-nlp]] - [[pedagogical-agent]] - [[open-source]] - [[affective-computing]] - [[lifelong-learning]] - [[self-regulated-learning]] ## Connected Articles - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[generativism-learning-theory]] — Generativism: Toward a Learning Theory for the Age of Generative Artificial Intelligence - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents ## Citation Jiawen Tao, Miao Peng, Yaoming Li, Xiaokun Yuan, Mengzhou Wu (2026). [Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training](https://arxiv.org/abs/2607.28109). arXiv:2607.28109. cs.AI. --- ## [Bots and Blocks: Presenting a Project-Based Approach for Robotics Education](https://edtechdev.github.io/aied/articles/bots-blocks-project-based-robotics-education-2026/) > **Synthesis:** Geger, Briechle, and Rausch (2026) propose a project-based learning approach for teaching robotics in higher education, arguing that classic study programs often fail to prepare students for industry work because of a lack of practical experience caused by solely theoretical lecturing. They present a framework for an agile, semester-spanning project where students learn to work with robots, part of the applied computer science degree program "Digital Technologies." The paper presents the framework and an exemplary project featuring the development of a disassembly software ecosystem for hardware robots. ## Key Findings - Classic study programs often **fail to prepare students for industry work** because of a lack of practical experience from solely theoretical lecturing. - A **project-based learning (PBL) approach** is proposed for teaching both practical and theoretical robotics skills interactively. - The approach uses an **agile, semester-spanning project** in which students learn to work with robots, within the applied computer science program "Digital Technologies." - The paper presents the framework and an **exemplary project: the development of a disassembly software ecosystem for hardware robots.** ## Study Design & Method This is a **design/case presentation** of a project-based learning framework for robotics education. The authors describe an agile, semester-spanning project within the applied computer science degree program "Digital Technologies" in which students work with robots. They present the pedagogical framework and an exemplary project — the development of a disassembly software ecosystem for hardware robots — illustrating how the approach integrates practical and theoretical robotics skills. ## Implications for AI in Education The paper demonstrates how [[project-based-learning]] can address the theory-practice gap in [[higher-ed]] robotics education by giving students hands-on experience with robots over a sustained, agile project. It connects to [[educational-robotics]], [[active-learning]], [[computational-thinking]], and [[programming-education]]. For educators, it offers a model for structuring semester-long robotics projects that prepare students for industry, emphasizing practical experience alongside theory. ## Limitations The paper presents a framework and a case example rather than a rigorous comparative evaluation of learning outcomes; empirical evidence of effectiveness is not comprehensively reported in the abstract. The approach is specific to one applied computer science degree program and context, so generalizability to other programs and levels is limited. ## Connected Concepts - [[educational-robotics]] - [[project-based-learning]] - [[higher-ed]] - [[active-learning]] - [[computational-thinking]] - [[programming-education]] ## Connected Articles - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM: LLMs and Robotic Simulation ## Citation Geger, T., Briechle, D., & Rausch, A. (2026). *Bots and blocks: Presenting a project-based approach for robotics education*. arXiv:2603.14529. --- ## [WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators](https://edtechdev.github.io/aied/articles/bridging-instructional-design-framework-math/) Castillo Ventura et al. (2026) address the gap between instructional design of digital mathematics resources and their pedagogical use in classrooms. Their work-in-progress framework translates learning theory principles into observable pedagogical variables structured as metadata, enabling teacher-support systems to characterize digital resources for mathematics education. Drawing on a literature review documenting limited operationalization of learning theories in educational technology, the authors propose a multidimensional structure and focus on one dimension — the conceptual structure of content — showing how prior knowledge, representation, and conceptual construction can be represented as metadata. While early-stage (work-in-progress), this framework has potential relevance for designing [[teacher-role]]-centered tools in [[stem-education]] and for integrating [[ai-literacy]] considerations into digital resource selection. ## Connected Concepts - [[teacher-role]] - [[stem-education]] - [[ai-literacy]] ## Connected Articles - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[reshaping-cs-education-genai]] — Reshaping Undergraduate Computer Science Education in the Generative AI Era ## Citation Castillo Ventura, E., Solis Lastra, J. U., & Alves Franco Brandão, A. (2026). [*WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators*](https://arxiv.org/abs/2606.20934). arXiv:2606.20934. --- ## [A New Direction for Students in an AI World: Prosper, Prepare, Protect](https://edtechdev.github.io/aied/articles/brookings-ai-students-report/) > A yearlong global "premortem" by the Brookings Center for Universal Education (2026) examining generative AI's risks and benefits for students. Based on 500+ interviews across 50 countries, 400+ studies reviewed, and a Delphi panel. # Brookings Report: AI and Students (Prosper, Prepare, Protect) ## Definition A yearlong global "premortem" by the Brookings Center for Universal Education (2026) examining generative AI's risks and benefits for students. Based on 500+ interviews across 50 countries, 400+ studies reviewed, and a Delphi panel. ## Central Finding **At this point in AI's trajectory, the risks of utilizing generative AI in children's education overshadow its benefits.** This is because risks strike at foundational child development and may block realization of AI's potential benefits. ## Two Paths 1. **AI-enriched learning** — well-designed AI tools with sound pedagogy can offer significant benefits 2. **AI-diminished learning** — overreliance threatens learning ability, social-emotional wellbeing, teacher-peer relationships, and student safety/privacy ## Three Action Pillars + 12 Recommendations - **Prosper:** Design AI that expands student potential; co-create with educators and communities; use tools that teach not tell - **Prepare:** Holistic AI literacy for all stakeholders; prepare teachers to teach with and through AI; clear vision for ethical use - **Protect:** Comprehensive regulatory frameworks; privacy and safety in procurement; break engagement addiction; support families ## Connected Concepts - [[regulation]] - [[student-experience]] - [[ai-literacy]] - [[k-12]] - [[llm]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[transfer-of-learning]] — AI Learning Transfer - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[tutoring-specific-vs-general-ai]] — Tutoring-Specific vs. General-Purpose AI in Education - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries ## Citation Institution, S.B. (2026). [A New Direction for Students in an AI World: Prosper, Prepare, Protect](https://www.brookings.edu/articles/a-new-direction-for-students-in-an-ai-world-prosper-prepare-protect/) --- ## [When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code](https://edtechdev.github.io/aied/articles/buggy-genai-code-student-responses/) As generative AI becomes central to software development, CS education is shifting toward prompt-centered workflows where students describe intended behavior in natural language to elicit code. But professional practice demands careful review of GenAI output that may look correct yet harbor subtle faults — a challenge in CS1, where current models solve tasks correctly and dull students' incentive to inspect generated code. Padurean et al. (2026) investigate how prompt-centered programming activities can be designed to deliberately surface buggy GenAI code, forcing students to practice verification rather than blind trust. The work speaks directly to the [[over-reliance]] problem and the [[cognitive-offloading]] temptation of strong code models, and complements [[code-review-genai-cs1]], which found oral code review preserves learning outcomes under rising AI use. By intentionally injecting faults, the approach reframes [[generative-ai]] in [[cs-education]] as a metacognitive training ground rather than an answer engine, reinforcing [[ai-literacy]] and [[metacognition]]. Implications for [[student-experience]] and [[formative-assessment]] design suggest that 'wrong-on-purpose' perturbations are a scalable way to build debugging habits in an era where [[reshaping-cs-education-genai]] is already underway. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[generative-ai]] - [[cs-education]] - [[ai-literacy]] - [[metacognition]] - [[student-experience]] - [[formative-assessment]] ## Connected Articles - [[code-review-genai-cs1]] - [[reshaping-cs-education-genai]] ## Citation Victor-Alexandru Padurean, Kaitlin Riegel, Alkis Gotovos, Jyotika Mahapatra, Ahana Ghosh, Adish Singla (2026). [When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code](https://arxiv.org/abs/2607.05068). arXiv:2607.05068. arXiv:2607.05068 [cs.SE, cs.CY]. --- ## [Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education](https://edtechdev.github.io/aied/articles/calibrating-trustworthiness-llm-education-2026/) > **Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education** — Longitudinal co-design with learning engineers building an LLM-powered digital textbook. Co-constructed five trustworthiness metrics with 20 measures tailored to pedagogical use. Designed visualizations mapping trustworthiness violations onto LLM res... [[llm]] [[ai-ed-evaluation]] [[over-reliance]] [[human-in-the-loop-ai]] [[instructional-design]] [[edtech-platform]] Longitudinal co-design with learning engineers building an LLM-powered digital textbook. Co-constructed five trustworthiness metrics with 20 measures tailored to pedagogical use. Designed visualizations mapping trustworthiness violations onto LLM responses. Making trustworthiness explicit increased inter-rater reliability and helped learning engineers resolve conflicting objectives and produce more consistent judgments. Proposes design guidelines for future LLM evaluation tools that enable pedagogically-aligned learning tools. ## Abstract LLMs are reshaping educational technology, yet evaluating their responses for pedagogical alignment remains underexplored, relying heavily on the expertise of learning engineers building the technology. Through a longitudinal co-design process with learning engineers developing an LLM-powered digital textbook, we co-constructed five trustworthiness metrics comprising 20 measures tailored to pedagogical use; designed visualizations that map trustworthiness violations onto LLM responses; and evaluated how these tools help learning engineers make A/B comparisons of LLM responses. ## Connected Concepts - [[llm]] - [[ai-ed-evaluation]] - [[over-reliance]] - [[human-in-the-loop-ai]] - [[instructional-design]] - [[edtech-platform]] ## Connected Articles - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education - [[cong-confidence-asag-2026]] — Confidence-Aware Automatic Short Answer Grading - [[jeon-isd-agent-bench-2026]] — ISD Agent Benchmark ## Citation Adam Coscia, Sujata Duwal, Langdon Holmes, Scott Crossley, & Alex Endert (2026). [Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education](https://arxiv.org/abs/2608.04006). arXiv:2608.04006. arXiv:2608.04006 [cs.HC] (under review). --- ## [The care-full craft of feedback in an age of generative AI](https://edtechdev.github.io/aied/articles/care-full-feedback-genai/) ## Summary A conceptual/position paper arguing that feedback in an age of GenAI must be understood as **"matters of care"** — ethical, relational practices rather than information transmission. It builds on a ten-principle **Manifesto for Feedback in the Age of GenAI** (Winstone et al. 2025, Copenhagen Feedback Symposium) and distils **four core values** for integrating GenAI into a multimodal feedback landscape: (1) feedback processes should support **meaning-making**, (2) build **educative relationships**, (3) be **trustworthy**, and (4) be respected as a **professional craft**. ## Core arguments - **Feedback as more than comments.** Feedback information ("bits and bytes") is only input; without student sense-making and uptake it does not constitute feedback. GenAI comment-generators risk replicating Sadler's "dangling data" critique — and may entrench an information-centric view the field spent decades moving past. - **"AI slop" and "human slop."** GenAI's mass-produced, speedy content (Madsen & Puyt 2025's "AI slop") has a human analogue: comment banks, quick marks, and standardised templates that already weaken feedback practice. Quantity and speed ≠ quality. - **Relational recognition.** Only human feedback can be *recognitive* — "the mutual acknowledgement of agency, vulnerability, and shared humanity" (Corbin, Tai & Flenady 2025); GenAI feedback is "extra-recognitive." Students in Otaki et al. (2026) described GenAI interactions as "each interaction feels like a new beginning," lacking the continuous timeline of human relationships. - **The safe-space paradox.** GenAI feedback reduces perceived personal risk (asking "stupid" questions, exposing weaknesses) — a genuine benefit, especially in power-hierarchical cultures — but may come at the cost of priming students for the friction of professional feedback encounters. - **Epistemic trust.** Trust is dynamic and relational ("a verb, not a noun"): students ascribe less trust to AI-generated than human feedback (Ruwe & Kuklick 2026; Henderson et al. 2025 — see [[genai-teacher-feedback-comparison]]), algorithm aversion makes AI trust decline sharply after errors, and GPT-4 limitations (outdated data, over-generalisation, hallucinations) strike at expertise, integrity, and benevolence — the dimensions of epistemic trust. - **Feedback as professional craft.** Tuck's ethnography shows marking is non-linear, context-sensitive, and entangled with "readings of students" — including their "back stories." Multimodal feedback (written + dialogic + digital) is where this craft lives; GenAI lacks genuine dialogue. Risk: outsourcing feedback production erodes the craft (skill atrophy, "unthinking") and even human-in-the-loop approval can internalise AI bias. ## Agenda for the future — four respects - **Respect for scholarship:** design GenAI integration starting from known feedback challenges (e.g. ipsative feedback across modules) rather than from comment generation; research beyond self-report. - **Respect for equity:** equitable access to meaningful feedback encounters; scaffold feedback literacies and evaluative judgement to avoid amplifying Matthew effects; respect conscientious objectors to GenAI. - **Respect for professional craft:** distinguish what GenAI cannot replicate (relationship-rich, multimodal, dialogic feedback); protect against de-prioritising care-full feedback because it is labour-intensive. - **Respect for human connection:** preserve situated, meaningful, care-full elements of feedback encounters; study how trust, connection, and engagement evolve as GenAI enters the ecosystem. ## Connected Concepts - [[equity-in-ai-education]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[teacher-role]] - [[ai-education]] - [[ethics]] - [[generative-ai]] ## Connected Articles - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative a... - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers' practices and perceptions - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education ## Citation Winstone, N. E., Gravett, K., Bearman, M., Noble, C., Jensen, L. X., Jones, A., & Nicola-Richmond, K. (2026). The care-full craft of feedback in an age of generative AI. [*Assessment & Evaluation in Higher Education*](https://doi.org/10.1080/02602938.2026.2643333), 51(5), 911–927 --- ## [Artificial intelligence assisted design of a novel cooperative learning technique for higher education](https://edtechdev.github.io/aied/articles/ccct-cooperative-learning-technique/) > **Synthesis:** Artificial intelligence assisted design of a novel cooperative learning technique for higher education ## Key Findings - A mixed-methods explanatory sequential study with **67 sophomore prospective teachers** at Hakkari University (Turkey) in the spring 2024–2025 semester: an experimental group of **31** Guidance and Psychological Counseling students (19 female, 12 male) versus a control group of **36** Primary School Mathematics Teaching students, in a theoretical "Curriculum Development in Education" course. - The intervention was designed by prompting **five AI chatbots (ChatGPT, Copilot, DeepSeek, Gemini, Qwen)** with course documents and cooperative learning principles in a single, non-iterative prompt; the researcher selected **Qwen's "Curriculum Concept Constellation Technique (CCCT)"** using a four-criterion rubric (novelty, alignment with cooperative learning principles, course context fit, clarity and practicality) — the other four proposals were judged syntheses or adaptations of existing methods. - CCCT has students collaboratively map curriculum concepts as "stars" into visual "constellations," working in heterogeneous teams of 4–6 with assigned roles (Star Finder, Constellation Designer, Metaphor Maker, Visual Artist, Navigator) across a two-week cycle ending in a gallery walk and group reflection. - **Academic achievement:** the experimental group's post-test (M = 51.45, SD = 6.08) significantly outperformed the control (M = 43.89, SD = 10.76), p = 0.001, **g = 0.839**; only the experimental group improved significantly from pre-test (p = 0.005, g = 0.738) while the control showed a weak effect (g = 0.119). - **Co-regulation:** experimental group rose from M = 61.32 to 65.45 (p = 0.043, **g = 0.512**); **attitudes toward cooperativeness** rose from M = 68.03 to 75.19 (p = 0.007, **g = 0.751**). - Qualitative analysis yielded five themes — fostering student agency and accountability, social cohesion and interpersonal growth, cognitive and pedagogical engagement, collaborative learning dynamics, and structural/logistical challenges (unequal participation, time constraints). ## Study Design & Method The study used an explanatory sequential mixed-methods design in three phases. Phase I developed the technique: two documents (course content; cooperative learning foundations and existing techniques) were given to five AI chatbots asked to propose a new, original cooperative learning technique; proposals were evaluated by the researcher against a rubric (novelty/distinctiveness, alignment with the five essential elements of cooperative learning, fit with the theoretical course and two-session timeframe, clarity and practicality). Phase II implemented the selected CCCT over two weeks, with pre- and post-measurement: a 15-item multiple-choice achievement test (KR-20 = 0.70), a 20-item cooperativeness scale (α = 0.80), and a 19-item co-regulated learning questionnaire (α = 0.89). Pre-tests showed no significant baseline differences; the experimental group was arranged into six heterogeneous teams based on pre-test scores, gender, GPA, and prior cooperative learning experience. Phase III collected semi-structured interviews (four open-ended questions) with experimental-group students. ## Key Results - **CCCT mechanics:** groups identify 5–7 key concepts ("stars") such as John Dewey's philosophy, behaviorism, societal needs, and developmental psychology, connect them into "constellations," and invent metaphors for the relationships (e.g., "Behaviorism is the gravitational force pulling all other ideas toward measurable outcomes"); role allocation, constellation creation, a gallery walk, and reflection structure the two weeks. - **Mechanism accounts:** participants described CCCT making abstract theoretical content more accessible and memorable (e.g., "When the lesson was conducted with this technique, I actually got more efficiency and it was more fun," P13), attributing gains to interaction, visual elements, shared responsibility, and holistic understanding — consistent with the authors' framing in cognitive elaboration, dual coding, and constructivist theories. - **Human–AI division of labor:** the authors stress the process was "AI-generated" rather than "co-design" — chatbots were prompted once and the researcher retained all selection, adaptation, and implementation decisions, with human oversight judged necessary to contextualize AI output. ## Implications for AI in Education The study positions generative AI as a **scalable ideation tool for pedagogical innovation**: one-shot prompting of multiple chatbots followed by principled, human rubric-based selection produced a novel, implementable cooperative learning technique that incorporates all five essential elements of cooperative learning and generated meaningful learning gains. For practitioners, this offers a low-cost template for generating course-tailored instructional methods; the CCCT itself demonstrates a concrete way to teach abstract theoretical content through metaphorical, visual, role-based collaboration — relevant to [[collaborative-learning]] and [[scaffolding]] in [[higher-ed]]. The authors caution that AI output requires systematic evaluation and continued human oversight, and that effectiveness should be compared against established cooperative learning techniques before claiming additive value. ## Limitations - Small sample from two departments of a single university limits generalizability; unmeasured disciplinary differences between intact classes (counseling students' possible inclination toward metaphorical, reflective thinking vs. mathematics students' logical, sequential orientation) may partly explain effects. - The control condition was traditional lecture-based instruction rather than an established cooperative learning technique, so the study does not show CCCT is superior to existing cooperative learning methods. - The two-week intervention duration makes a novelty effect ("more fun," "more enjoyable") impossible to rule out; longitudinal studies are needed. - No formal systematic review (e.g., PRISMA) or bibliometric analysis was used to establish the pedagogical gap motivating the new technique, and qualitative data relied on self-report, which may be subject to social desirability bias. ## Connected Concepts - [[higher-ed]] - [[collaborative-learning]] - [[scaffolding]] - [[math-education]] - [[prompt-engineering]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[affective-tutoring]] ## Connected Articles - [[ai-collaborative-learning-skills-impacts]] — Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education - [[ai-chatbot-collective-efficacy-collaborative-learning]] — AI chatbot design principles to enhance the collective efficacy in collaborative learning - [[llm-critical-thinking-teamwork-review]] — Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review - [[ai-collaborative-learning-systematic-review]] — A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Tutal, Ö. (2026). [*Artificial intelligence assisted design of a novel cooperative learning technique for higher education*](https://doi.org/10.1038/s41598-026-50770-1). --- ## [Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits](https://edtechdev.github.io/aied/articles/chat-debugging-human-ai-collaboration-circuits/) > **Synthesis:** This exploratory study investigates how undergraduates use [[llm|LLMs]] to debug malfunctioning analog circuits under exam conditions, identifying both promising [[human-ai-collaboration|collaborative patterns]] and critical limitations. Through thematic analysis of student chat logs, the authors find that off-the-shelf LLMs offer considerable domain knowledge and sensible debugging suggestions, yet struggle with 2D/3D image-based reasoning and display unjustified confidence. Students, in turn, show deficits in fundamental concepts and [[computational-thinking|critical thinking]] during AI-assisted debugging. ## Study Design - **Context:** Undergraduate students debugging predetermined buggy analog circuits on breadboards and PCBs - **Conditions:** Exam setting with time pressure - **Data:** Voluntarily shared chat logs from student-LLM conversations - **Analysis:** Thematic analysis of interaction patterns ## Key Findings ### What Worked - LLMs provided **sensible debugging suggestions** grounded in domain knowledge - Students developed **multimodal usage patterns** — combining text descriptions, code snippets, and circuit descriptions - AI-assisted debugging showed potential for **scaffolding learning** by guiding students through systematic troubleshooting ### What Didn't Work | Limitation | Description | |-----------|-------------| | 2D/3D image reasoning | LLMs failed to interpret circuit board images and diagrams | | Unjustified confidence | LLMs presented incorrect diagnoses with high confidence | | Student fundamental gaps | Students lacked core concepts needed to evaluate AI suggestions | | Critical thinking deficits | Students over-relied on AI outputs without verification | ## Implications for AI in Engineering Education This study contributes to understanding AI's role in [[cs-education|engineering education]] by revealing a dual challenge: 1. **Tool limitations:** Current LLMs are not yet reliable for tasks requiring spatial reasoning about physical circuits 2. **Pedagogical challenge:** Simply providing AI access does not replace the need for strong fundamental knowledge — students must develop the critical thinking skills to evaluate AI outputs The findings support a [[scaffolding]] approach where AI tools complement rather than replace instructor-guided learning in hands-on engineering contexts. ## Connected Concepts - [[llm]] - [[human-ai-collaboration]] - [[computational-thinking]] - [[cs-education]] - [[scaffolding]] ## Connected Articles - [[generativism-learning-theory]] — Generativism: Toward a Learning Theory for the Age of Generative Artificial Intelligence - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[llm-design-problems-hot-pjbl]] — LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning - [[spritz-ai-disciplinary-mediation-student-teams-2026]] — Exploring AI-Supported Disciplinary Mediation in Student Project Teams' Text-Based Communication ## Citation Hu, J., & Ash, A. (2026). [*Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits*](https://arxiv.org/abs/2608.02955v1). arXiv:2608.02955v1. --- ## [WIP: Chat-Debugging: Large Language Model as a Hardware Debugging Assistant](https://edtechdev.github.io/aied/articles/chat-debugging-llm-hardware-education-2026/) > **Synthesis:** Work-in-progress exploring LLMs as debugging assistants for physical hardware lab courses. Proposes 'Chat-Debugging' where students interact with an LLM to diagnose circuit faults. Aims to reduce frustration and improve debugging skill development. Initial prototype tested in an undergraduate hardware course; preliminary results suggest LLM assistance helps students identify faults faster and provides just-in-time scaffolding without giving away solutions. [[llm]], [[scaffolding]], [[cs-education]], [[stem-education]], and [[student-experience]]. Work-in-progress exploring LLMs as debugging assistants for physical hardware lab courses. Proposes 'Chat-Debugging' where students interact with an LLM to diagnose circuit faults. Aims to reduce frustration and improve debugging skill development. Initial prototype tested in an undergraduate hardware course; preliminary results suggest LLM assistance helps students identify faults faster and provides just-in-time scaffolding without giving away solutions. ## Connected Concepts - [[llm]] - [[scaffolding]] - [[cs-education]] - [[stem-education]] - [[student-experience]] ## Connected Articles - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[llm-diverse-explanations-programming]] — Exploring the Value of Diverse LLM Explanations in Introductory Programming - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[reshaping-cs-education-genai]] — Reshaping Undergraduate Computer Science Education in the Generative AI Era - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning ## Citation Andrew Ash, & John Hu (2026). [WIP: Chat-Debugging: Large Language Model as a Hardware Debugging Assistant](https://arxiv.org/abs/2608.02420). arXiv:2608.02420. IEEE Frontiers in Education Conference (FIE) 2026. --- ## [ChatGPT Critical and Creative Thinking: Systematic Review](https://edtechdev.github.io/aied/articles/chatgpt-critical-creative-thinking-review/) > Li, Cui & Hagedorn (2026) PRISMA-review **67 empirical studies (2022–2025)** on ChatGPT and university students' [[critical-thinking|critical]] and creative thinking: effects are contingent on **pedagogical framing**, not the tool itself ([[generative-ai]]). Li, Cui, and Hagedorn (2026) conducted a PRISMA-guided systematic review of **67 empirical studies** (2022–2025) examining how ChatGPT influences university students' critical and creative thinking. Using a dual-lens framework — convergent (critical thinking) and divergent (creative thinking) processes — the review reveals that ChatGPT's cognitive effects are fundamentally contingent on **pedagogical framing**, not the tool itself. ## Theoretical Framework The review draws on an integrated constellation of theories spanning cognitive, sociocultural, and technological dimensions: | Theory | Role in Framework | |--------|-------------------| | **AI Literacy** (Long & Magerko, 2020) | Foundational moderator — shapes prompt quality, epistemic vigilance, and verification practices | | **Self-Regulated Learning** (Zimmerman, 2002) | Explains how structured engagement promotes planning, monitoring, and evaluative judgment | | **Cognitive Load Theory** (Sweller, 2011) | Clarifies when ChatGPT reduces extraneous load (beneficial) vs. enables cognitive offloading (harmful) | | **Distributed Cognition** (Hutchins, 1995) | Frames ChatGPT as an interactive cognitive artifact supporting iterative reasoning cycles | | **Cultural Historical Activity Theory** (Engeström, 1987) | Explains why outcomes differ across pedagogical ecosystems — rules, division of labor, task objects | | **Boundary Object Theory** (Star & Griesemer, 1989) | Captures ChatGPT's interpretive flexibility across disciplines and tasks | | **Connectivism** (Siemens, 2005) | Analyzes ChatGPT as a generative node in distributed learning networks | This constellation treats ChatGPT as an **interactive cognitive artifact within a socio-technical learning system** — neither inherently beneficial nor detrimental, but contingent on how learners and instructors mobilize its affordances. ## Study Characteristics The 67 studies reflect the evolving nature of ChatGPT research: - **Design:** 39% quantitative, 30% qualitative, 31% mixed-methods - **Disciplines:** STEM 36%, Teacher Education 31%, Language/Writing 27%, Interdisciplinary 6% - **Geography:** Asia 58%, Europe 18%, North America 10%, Oceania 4%, Africa 4% - **Publication trend:** 10% in 2023, 58% in 2024, 31% in 2025 (through April) - **Focus:** 34% CT only, 16% CrT only, 49% both - **Top venues:** Education and Information Technologies (4), Frontiers in Education (3), JITE: Research (3), Computers & Education (3) ### Methodological Note: Assessment Asymmetry A critical methodological finding: **CrT was more often assessed with direct performance tasks** (TTCT, expert-rated artifacts), while **CT relied more on indirect self-report measures**. This asymmetry may partly explain why some studies report stronger evidence for CrT gains than CT gains, independent of actual cognitive effects. ## Key Findings ### Critical Thinking (56 studies) **Affordances** (when ChatGPT was embedded in structured, scaffolded designs): 1. **Metacognitive engagement** (n=27) — Guided prompting, comparative analysis, and reflective writing enabled students to monitor thinking and exercise evaluative judgment 2. **Argumentative structuring** (n=22) — ChatGPT as dialogic scaffold or counterargument generator, especially with argument mapping and rubric-guided evaluation 3. **Verification and error detection** (n=19) — Fact-checking and triangulation behaviors emerged when students were explicitly taught to identify hallucinations 4. **Self-regulated learning** (n=17) — Structured prompts and revision cycles promoted planning, monitoring, and strategic adjustment 5. **Disciplinary reasoning** (n=15) — Complex analysis emerged organically when ChatGPT was a co-developer or critique target in authentic disciplinary tasks **Limitations** (in unstructured contexts): 1. **Cognitive offloading/overreliance** (n=21) — Most common risk, especially among novice users and non-native speakers 2. **Surface-level engagement** (n=18) — Uncritical acceptance of AI outputs without appraisal 3. **Erosion of argument development** (n=14) — AI replaced the cognitive struggle integral to constructing ideas 4. **Metacognitive offloading** (n=12) — Surface-level edits without deeper planning or reflection 5. **Epistemic boundary limits** (n=10) — ChatGPT struggled with advanced rationality, logical consistency, and sustained Socratic dialogue ### Creative Thinking (44 studies) **Affordances:** 1. **Ideation and divergent thinking** (n=31) — Most frequently reported affordance; ChatGPT as brainstorming partner surfacing unique insights 2. **Structural and expressive scaffolding** (n=24) — Assisted with structuring content, experimenting with tone, stylistic expression 3. **Dialogic engagement and perspective-shifting** (n=18) — Functioned as co-designer in argument, debate, and role-based simulations 4. **Affective and motivational activation** (n=16) — Reduced creative anxiety; perceived as "brainstorming buddy" 5. **Instructionally mediated gains** (n=21) — Significant gains in originality, fluency, and elaboration when embedded in flipped classrooms or scaffolded creative modules (d=0.55–0.69 for key measures) **Limitations:** 1. **Creative passivity** (n=20) — Diminished inclination to explore original ideas; substitution of cognitive effort 2. **Loss of voice and affective authenticity** (n=15) — AI outputs lacked individual style and emotional nuance 3. **Suppression of iterative exploration** (n=13) — Repetitive use, uncritical adoption, minimal conceptual recombination 4. **Risk-related inhibition** (n=11) — Privacy concerns and fear of underperformance suppressed creative risk-taking 5. **Instructional deficit** (n=14) — Without reflective prompts or structured interaction, ChatGPT functioned as passive answer provider ### Co-Occurrence Patterns The 33 studies examining both CT and CrT revealed three trajectories: | Pattern | N | Description | Conditions | |---------|---|-------------|------------| | **Synergistic Enhancement** (CT↑, CrT↑) | 18 | Simultaneous gains in both domains | Inquiry-oriented tasks, scaffolded reflection, dialogic interaction, iterative refinement | | **Asymmetrical Development** (CrT↑, CT↓) | 8 | Creative fluency improved but critical engagement declined | Unstructured use, emphasis on ideation over evaluation, minimal critical framing | | **Joint Cognitive Erosion** (CT↓, CrT↓) | 4 | Both domains stagnated or declined | Passive/unscaffolded usage, task-completion focus, no metacognitive prompts | The **synergistic pattern** aligns with [[critical-thinking-genai-scaffolding]] — when tasks require both generation and evaluation, AI supports both. The **asymmetrical pattern** is the most common failure mode: creativity flourishes at critical thinking's expense. The **joint erosion pattern**, while least common (n=4), is the most concerning — occurring when ChatGPT is used purely as a convenience tool with no pedagogical framing. ## Discussion: ChatGPT as Cognitive Mediator The review's central insight: ChatGPT functions as a **cognitive mediator** whose outcomes are contingent on how its affordances are mobilized. The most consistent divider was not ChatGPT's presence but the **instructional ecology** surrounding it: - When tasks required **verification, justification, and iterative revision**, students treated AI outputs as provisional representations to interrogate - When such norms were **weak**, fluent outputs lowered perceived task difficulty and encouraged premature closure ChatGPT's **semantic fluency** emerged as both a strength and constraint — enabling rapid ideation while risking the masking of epistemic gaps. This maps onto [[metacognition|metacognitive suppression risks]]: reduced perceived effort may encourage cognitive offloading rather than strategic load management. The **boundary object** function — ChatGPT's interpretive flexibility across disciplines and tasks — supported cognitive adaptability and creative recombination, but also amplified surface-level synthesis when verification norms were absent. This connects to [[higher-ed|institutional norms]] and the [[institutional-change-framework-ai|broader activity systems]] in which AI tools operate. ## Six Pedagogical Recommendations 1. **Embed structured cognitive scaffolding** — Stepwise activities: prompt design → output evaluation → iterative refinement. Frame ChatGPT as dialogic partner, not solution provider. This aligns with [[critical-thinking-genai-scaffolding|the six-process scaffolding framework]] and [[human-in-the-loop-ai|human-in-the-loop architectures]]. 2. **Explicitly teach AI literacy for epistemic vigilance** — Integrate modules on prompt refinement, hallucination recognition, bias detection, and contextual interpretation. Addresses the [[ai-literacy-assessment-misalignment|self-report vs. performance gap]] in AI evaluation skills. 3. **Design tasks that co-activate CT and CrT through recursive inquiry** — Open-ended case studies, argumentative writing with multi-perspective AI dialogue, project-based tasks requiring both generation and analytical reflection. This is the practical implementation of the dual-lens framework. 4. **Implement reflection protocols for cognitive regulation** — Guided prompts after each interaction: "What was most useful/misleading?", "How did this shape your thinking?", "What would you change in your next prompt?" Reinforces [[metacognition|metacognitive monitoring]]. 5. **Leverage ChatGPT as a connective node for interdisciplinary thinking** — Cross-domain tasks that draw on ChatGPT's broad knowledge while critically examining disciplinary assumptions. Supports [[ai-learning-companions-framework|dialogic partner]] and connectivist learning. 6. **Position feedback as a multi-source process** — Triangulate AI feedback with peer review, instructor input, and self-assessment. Creates [[feedback-loop|multi-source feedback loops]] that mitigate overreliance. ## Limitations of the Review - English-language, peer-reviewed journal articles only — excludes conference proceedings (LAK, AIED, L@S) and non-English research - Dominance of Asian institutions (58%) and early-adopter settings limits generalizability - Most studies were cross-sectional/short-term; no longitudinal tracking of cognitive habit formation - Assessment asymmetry: CrT measured with performance tasks, CT with self-reports — apparent robustness differences may reflect measurement, not reality - Rapidly evolving technology — findings tied to specific ChatGPT versions; living systematic reviews needed - Publication bias likely favors positive findings in this emerging field ## Implications for the Wiki This review is a **keystone synthesis** connecting multiple threads in the AI education evidence base: - **[[critical-thinking-genai-scaffolding]]** shares the core premise — pedagogical design determines whether GenAI helps or harms thinking — and the six-process framework maps directly onto this review's scaffolding recommendations - **[[metacognition]]** identifies the same suppression risk and regulatory mechanisms the review documents at scale across 67 studies - **[[ai-literacy-assessment-misalignment]]** explains why students struggle to calibrate AI evaluation — the review confirms this as a critical moderator - **[[human-in-the-loop-ai]]** provides implementation architectures for the scaffolding strategies recommended - **[[ai-learning-companions-framework]]** offers design paradigms aligned with "dialogic partner" and "boundary object" concepts - **[[feedback-loop]]** operationalizes the multi-source feedback recommendation - **[[faculty-development]]** is essential — educators need training to implement these scaffolds - **[[student-experience]]** captures the learner perspective on usage patterns - **[[higher-ed]]** and [[universities-ai-era-rethinking]] provide the institutional context - **[[institutional-change-framework-ai]]** frames how activity systems must adapt The review's core insight — that ChatGPT's cognitive effects are contingent on pedagogy, not inherent to the technology — reinforces a pattern visible across the wiki: **AI in education succeeds or fails based on how it is implemented, not what it can do.** ## Connected Concepts - [[ai-literacy-assessment-misalignment]] - [[dot-framework-survey]] - [[faculty-development]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[metacognition]] - [[student-experience]] - [[ai-literacy]] - [[generative-ai]] - [[scaffolding]] ## Connected Articles - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[critical-genai-use-predictors]] — GenAI Knowledge, Epistemic Orientation, and Intellectual Values Predict Undergraduate Students' Critical GenAI Use - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI - [[institutional-change-framework-ai]] — A Framework for Institutional Change in the Age of AI - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education ## Citation Li, C., Cui, H., & Hagedorn, L. S. (2026). [*The cognitive impact of ChatGPT in higher education: A systematic review of critical and creative thinking outcomes*](https://www.sciencedirect.com/science/article/pii/S2666920X26000330). *Computers and Education: Artificial Intelligence*. --- ## [Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative artificial intelligence](https://edtechdev.github.io/aied/articles/chatgpt-feedback-engagement-genai/) A qualitative study of **16 undergraduates** at a Hong Kong teacher-education university who used **ChatGPT 3.5** to obtain feedback on IELTS writing tasks. Data came from unobtrusive screen-recorded observations plus stimulated-recall interviews. The study extends the traditional tripartite model of feedback engagement (cognitive, affective, behavioural) to a **four-dimensional model adding metacognitive engagement**, and asks what feedback literacy students need in a GenAI context. ## Key findings by engagement dimension ### Cognitive engagement — active, comparison-driven - Three strategies: **selective attention** (n = 8, focusing on areas of self-assessed weakness), **extracting key information** (n = 4, filtering lengthy ChatGPT output), and **comparison** (n = 10 — comparing original vs revised versions, ChatGPT comments across time, against IELTS criteria, and against teachers' previous comments). - Comparison is "a hub" for internal feedback (Nicol 2021): in GenAI contexts students can instantaneously compare multiple sources and versions — something hard in traditional feedback environments. ### Metacognitive engagement — weaker, signalling "metacognitive laziness" - Only **5 of 16** monitored their revision process; 6 reflected after finishing; **none** used goal-setting or planning strategies. - Echoes Fan et al.'s (2025) *metacognitive laziness*: ChatGPT may boost short-term task performance while impeding self-regulation and knowledge transfer. Most revisions targeted **local** aspects of L2 writing (grammar, vocabulary, sentence structure), which may not transfer to future writing tasks. ### Affective engagement — calm, but selectively trusting - Majority (n = 13) described interactions as calm, relaxed, stress-free — minimal emotional resistance (unlike typical teacher-feedback contexts). - **Trust split:** trusted ChatGPT on **language forms** (n = 11) but doubted its **examples/evidence** (n = 12); some doubted its legitimacy as an IELTS examiner. - Negative affect arose when ChatGPT misunderstood prompts or repeated similar suggestions. ### Behavioural engagement — superficial patterns - **308 prompts** total (9–40 per student); more prompts on **local aspects** (n = 104) than global (n = 55); 11.3% were direct copies of the draft; only 1.6% probed ChatGPT's credibility. - Interactions were mostly **one-off** (n = 133 one-round vs 34 two-round vs 18 three-round). - **High uptake:** 56.3% of ChatGPT comments honestly followed, 27.6% adapted, 16.1% rejected (83.9% overall uptake) — but revisions were superficial and local, and some students avoided directly using AI content due to academic-integrity beliefs. ## Five capacities for student feedback literacy in a GenAI context 1. **Prompt engineering** — the quality of GenAI feedback is largely determined by prompt clarity (e.g. the CLEAR framework; Lo 2023). 2. **Evaluative judgement** — discerning useful feedback from plausible-but-unreliable output. 3. **Emotional reflexivity** — balancing trust and doubt by understanding GenAI's capabilities and limits (Bearman & Ajjawi 2023). 4. **Ethical decision-making** — deciding how, when, and why to use GenAI feedback so work remains authentic (academic integrity). 5. **Metacognitive skills** — setting feedback goals, planning prompts, self-monitoring interactions, and reflecting on the whole process. ## Implications - Feedback literacy and engagement are bidirectional and mutually reinforcing; the model proposed here (Figure 2) shows the four engagement dimensions interplaying with these five literacy capacities. - GenAI feedback can lower the emotional barrier to feedback seeking (important in power-hierarchical cultures — see [[genai-teacher-feedback-comparison]]'s "less risky" finding), but without metacognitive scaffolding students drift toward **superficial, high-uptake, low-transfer** use — a core [[over-reliance]] risk. ## Connected Concepts - [[over-reliance]] ## Connected Articles - [[genai-teacher-feedback-comparison]] ## Citation Zhan, Y., & Yan, Z. (2026). Students' engagement with ChatGPT feedback: Implications for student feedback literacy in the context of generative artificial intelligence. [*Assessment & Evaluation in Higher Education*](https://doi.org/10.1080/02602938.2025.2471821), 51(5), 821–834 --- ## [ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills](https://edtechdev.github.io/aied/articles/chatgpt-hints-human-tutor-learning-gains-2024/) > Pardos & Bhandari (2024) report a randomized efficacy study (N=274) comparing ChatGPT-generated hints to human tutor-authored hints and a no-help control across four mathematics subject areas. Only the ChatGPT condition produced statistically significant learning gains versus control, with no significant difference between ChatGPT and human-authored hints — and ChatGPT's 32% raw hint-error rate was reducible to near zero (algebra) or 13% (statistics) using the self-consistency hallucination-mitigation technique. Authoring help content for educational technologies is labor-intensive and costly — a full-time employee may take a year to produce a textbook's worth of material. If LLMs can generate hints with sufficiently low error and sufficient learning efficacy, they could alleviate the most time- and cost-intensive component of tutoring-system development and enable scaling to many domains. This study evaluates whether ChatGPT-generated worked-solution hints can do so. ## Study design A **3 × 4 between-subjects design** with 274 participants (Mechanical Turk workers, all MTurk Masters with at least a high-school degree). Participants were randomly assigned to one of three hint conditions — **control** (no hints), **human tutor**, or **ChatGPT** — paired with one of four mathematics subjects drawn from OpenStax CC BY textbooks: Elementary Algebra, Intermediate Algebra, College Algebra, or Statistics. Each participant completed a 3-item pre-test, a 5-item acquisition phase in the OATutor platform, and a 3-item post-test (same questions as pre-test). ChatGPT hints were single worked-solution hints generated by prompting ChatGPT 3.5 with each problem's text; human hints came from the existing OATutor content library authored by UC Berkeley undergraduates. ## Key findings ### RQ1 — Hint quality and hallucination mitigation - **32% of ChatGPT-generated hints failed quality checks** (75 problems; 24 disqualified) due to containing incorrect answers and/or incorrect solution steps. No hints contained inappropriate language or grammatical errors. Disqualification ranged from 25% (Elementary Algebra) to 47% (Intermediate Algebra). - **Self-consistency drastically reduced errors:** prompting 10 times per problem and returning the modal answer reduced the error rate to **near 0% for the three algebra subjects** and **13% for statistics**. - Human inter-rater agreement was high (Fleiss' κ = 0.857–0.929, "almost perfect"). Manual quality checking averaged 37.6 seconds per hint. ### RQ2 — Learning efficacy - **ChatGPT hints produced the largest learning gain: 17.00%** (pre 43.51% → post 60.52%, p<0.001) — statistically significant versus the control's 1.85% gain (p=0.011). - **Human tutor hints produced an 11.62% gain** (p=0.001), not statistically separable from the ChatGPT condition (p=0.416). - The **control produced a non-significant 1.85% gain** (p=0.192), establishing that gains were attributable to the hint conditions rather than mere test-retest recall. - ChatGPT gains were higher than human-authored gains in **all four subjects**, though not significantly separable. - **No significant time-on-task difference** between the ChatGPT and human conditions (both higher than control). ## Implications for AI in education The findings suggest that **LLM-generated worked solutions can be as effective as human-authored tutoring content** while being produced in a fraction of the time (roughly 1/20th), opening the door to autonomous generation of effective mathematics tutoring content from arbitrary educational resources. However, the authors are explicit about caution: at a 32% raw error rate, **ChatGPT should not be used to give feedback the way a teacher or TA would** unless in a domain verified to have near-zero error. Where error mitigation cannot achieve near-zero rates, designers should frame LLM feedback as coming from an "imperfect robot" or peer-like source so students consider it critically. The error-reduction via [[hallucination-risk|self-consistency]] is central, and the findings ground the [[genai-performance-vs-learning|performance-vs-learning]] distinction by demonstrating genuine learning gains, not just performance. ## Connected Concepts - [[generative-ai]] - [[llm]] - [[ai-tutoring]] - [[intelligent-tutoring]] - [[scaffolding]] - [[math-education]] - [[learning-gains]] - [[hallucination-risk]] - [[adaptive-learning]] - [[feedback-loop]] ## Connected Articles - [[oatutor-open-source-adaptive-tutor-2023]] — OATutor: Open-Source Adaptive Tutoring System - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[genai-performance-vs-learning]] — Distinguishing Performance Gains from Learning - [[ai-generated-feedback-higher-ed]] — AI Feedback in University Education - [[generative-ai-guardrails-harm-learning]] — GenAI Without Guardrails Can Harm Learning - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators - [[access-not-enough-ai-tutoring-2026]] — Access Is Not Enough: AI Tutoring - [[adaptive-pretesting-retention]] — Adaptive Pretesting and Retention ## Citation Pardos, Z. A., & Bhandari, S. (2024). [*ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills*](https://doi.org/10.1371/journal.pone.0304013). *PLOS ONE*, 19(5), e0304013. https://doi.org/10.1371/journal.pone.0304013 --- ## [Little Impact of ChatGPT Availability on High School Student Test Score Performance](https://edtechdev.github.io/aied/articles/chatgpt-impact-high-school-tests/) ## Research Design This paper uses a clever identification strategy: measure the **seasonal drop in ChatGPT activity during non-school summer months** (2023 and 2024). Areas with larger summer dropoffs have heavier school-related AI use. The author then examines whether higher AI-use areas show different test score trends. ## Key Finding: Null Result **No meaningful impact of ChatGPT availability on high school test score averages in either direction.** Whether students use AI to cheat, to learn, or both, the aggregate effect on standardized test performance is negligible — at least in the short term and as AI is actually used. ## Interpretation The null result has two non-exclusive interpretations: (1) AI-assisted cheating does not harm test performance in aggregate, or (2) negative effects of AI misuse are offset by positive uses of AI as a learning aid. Either way, this challenges both utopian and dystopian narratives about AI in classrooms. It aligns with the [[genai-performance-vs-learning]] distinction — performance boosts from AI may not translate to test score improvements. It connects to [[over-reliance]] research showing that AI assistance can produce apparent competence without underlying understanding. ## Methodological Contribution The seasonal variation identification strategy is innovative for causal inference in education technology research, where RCTs are often infeasible at scale. ## Connected Concepts - [[over-reliance]] ## Connected Articles - [[genai-performance-vs-learning]] ## Citation Huntington-Klein, N. (2026). [Little impact of ChatGPT availability on high school student test score performance](https://arxiv.org/abs/2605.08812). arXiv:2605.08812. --- ## [Pedagogical Promise and Peril of AI: A Text Mining Analysis of ChatGPT Research Discussions in Programming Education](https://edtechdev.github.io/aied/articles/chatgpt-programming-education-text-mining/) ## Analysis This book chapter presents a **text mining analysis** of how scholarly literature frames ChatGPT's role in programming education. Using term frequency analysis, phrase pattern extraction, and topic modeling, the authors identify four dominant themes: pedagogical implementation, student-centered learning, AI infrastructure, and assessment design. The central finding is a **dual positioning**: ChatGPT is consistently framed as both a learning aid (enhancing explanation, feedback, efficiency) and a pedagogical risk (overreliance, unreliable outputs, academic integrity). This connects to [[academic-integrity]], [[over-reliance]], and [[hallucination-risk]] by documenting these as dominant framings in the research literature. Notably, research is skewed toward classroom practice while systematic assessment design and institutional governance remain underexplored. ## Connected Concepts - [[academic-integrity]] - [[over-reliance]] - [[hallucination-risk]] ## Connected Articles - [[shame-guilt-ai-regulation-computing-education]] — Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education - [[eduguard-safe-rag-llm-tutor]] — EduGuard: A Safe RAG-Based LLM Tutor for Programming Education - [[generative-ai-reduced-study-time-math]] — Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build - [[chatgpt-impact-high-school-tests]] — Little Impact of ChatGPT Availability on High School Student Test Score Performance - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom ## Citation Grume et al. (2026). [Pedagogical Promise and Peril of AI: A Text Mining Analysis of ChatGPT Research Discussions in Programming Education](https://arxiv.org/abs/2605.00361). arXiv:2605.00361. Pedagogical Innovations in CS Education (IGI Global). --- ## [Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework](https://edtechdev.github.io/aied/articles/child-safety-genai/) > **Haein Kong** — HEAL Workshop at CHI 2026, submitted 1 Jul 2026 **Haein Kong** — HEAL Workshop at CHI 2026, submitted 1 Jul 2026 Proposes an evaluation framework for child-specific harms in generative AI; applied to education domain, Llama Guard models struggle to detect unsafe user prompts from children. ## Key Findings - Most existing AI safety evaluation frameworks and benchmarks focus on general (adult) user populations and overlook risks unique to younger users, despite evidence of heavy youth engagement — a recent national survey found 72% of U.S. adolescents have used AI companions. - The paper proposes an evaluation framework that integrates expert-guided risk factors with real-world AI incident data, identifying hazard categories from expert guidelines and AI incident databases and using them to construct a synthetic test set for model evaluation. - Applied to the education domain, three Llama Guard models were evaluated on their ability to detect unsafe user prompts; the models struggled to identify education-related unsafe prompts. - The evaluation pipeline includes incident-grounded prompt generation — producing harmful education-related prompts framed from a teen or student perspective based on reported incidents — alongside generation of safe prompts for contrast. - Future work is framed as extending the evaluation to additional risk categories and incorporating domain experts throughout the evaluation pipeline. ## Study Design & Method The framework combines two evidence sources: hazard categories derived from expert guidelines and categories mined from AI incident databases. These inform a synthetic test set in which harmful and safe education-domain user prompts are generated from incident descriptions, with the user assumed to be a teen or student. The resulting test set is used to evaluate safety classifiers — here, three Llama Guard models — on their detection of unsafe user prompts, with assessments scored as safe or unsafe. This design lets the authors measure child-specific safety performance in a region where existing general-population benchmarks leave a gap. ## Implications for AI in Education The results carry a direct warning for AI-based learning environments: general-purpose safety classifiers do not reliably catch education-related unsafe prompts from children, so [[pedagogical-safety]] cannot be assumed from standard model safeguards. Schools and edtech providers deploying [[generative-ai]] tools need child-specific evaluation, incident-grounded testing, and human oversight rather than reliance on off-the-shelf safety models. The framework's structure — expert guidance plus incident data plus synthetic testing — is itself a template that educational institutions and researchers can reuse to evaluate tools for younger users, with implications for [[privacy]] and [[equity]] in who is protected by default safety practices. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[gaze-informed-ai-children]] — Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[llm-unlearning-math-privacy]] — Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education ## Citation Haein Kong (2026). [Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework](https://arxiv.org/abs/2607.00395). arXiv:2607.00395. HEAL Workshop at CHI 2026, submitted 1 Jul 2026 --- ## [Anchor Is the Key: Toward Accessible Automated Essay Scoring with Large Language Models Through Prompting](https://edtechdev.github.io/aied/articles/choi-anchor-aes-prompting-2025/) > **Synthesis:** Choi, Tate, Ritchie, Nixon & Warschauer (2025) investigate the most practical approach to LLM-based automated essay scoring — prompting — and find that providing anchor papers (example essays with scores) significantly improves LLM-human agreement, bringing it close to human-human scoring reliability. GPT-4o mini achieves comparable results to GPT-4o at substantially lower cost, making accessible, teacher-friendly AES feasible. ## Key Findings - **Anchor papers are critical:** Including exemplar essays in prompts dramatically improved scoring accuracy, reaching near-human reliability levels - **Cost-effective models work:** GPT-4o mini achieved comparable results to GPT-4o at a fraction of the cost, removing barriers for educator adoption - **Prompting beats fine-tuning for accessibility:** Unlike resource-intensive optimization methods, structured prompting is practical for teachers to use directly - **Rubrics + anchors outperform rubrics alone:** Combining grading rubrics with anchor papers yielded the best results - **Secondary school essays tested:** Study used argumentative essays from secondary students, demonstrating relevance for [[k-12]] contexts ## Implications This study advances [[automated-essay-scoring]] by shifting focus from resource-intensive model optimization to accessible prompting strategies. For teachers, the finding that GPT-4o mini with anchor papers approaches human reliability means practical AES is within reach — no expensive compute, no large pre-scored essay banks. The anchor paper approach connects to [[prompt-engineering]] best practices and suggests a pathway for [[writing-education]] where teachers can calibrate AI scoring to their own assessment standards rather than relying on black-box systems. The work complements [[psyscore-essay-scoring-zpd-feedback]] research on psychometrically-aware scoring and [[icle-plus-plus-essay-scoring]] work on fine-grained trait scoring, showing that prompt design alone — particularly anchor inclusion — can achieve strong holistic scoring. For [[ai-literacy]], this empowers educators to understand and control AES rather than treating it as an opaque tool. ## Connected Concepts - [[k-12]] - [[automated-essay-scoring]] - [[prompt-engineering]] - [[writing-education]] - [[ai-literacy]] ## Connected Articles - [[psyscore-essay-scoring-zpd-feedback]] - [[icle-plus-plus-essay-scoring]] ## Citation Choi, J., Tate, T., Ritchie, D., Nixon, N., & Warschauer, M. (2025). [*Anchor Is the Key: Toward Accessible Automated Essay Scoring with Large Language Models Through Prompting*](https://doi.org/10.35542/osf.io/cbhgz_v1). EdArXiv. doi:10.35542/osf.io/cbhgz_v1. --- ## [AI-Generated Lesson Plans in Civic Education](https://edtechdev.github.io/aied/articles/civic-education-ai-lesson-plans/) > An analysis of 310 AI-generated lesson plans (2,230 individual activities) produced by ChatGPT (GPT-4o), Gemini (1.5 Flash), and Copilot (GPT-4 based) for all 53 Massachusetts eighth-grade civics standards. Each standard received two prompts: a basic "write a lesson plan" and a "highly interactive" variant. > Large-scale empirical evaluation of AI-generated civics lesson plans reveals that without teacher revision, AI tools overwhelmingly produce lower-order thinking activities and monocultural content — fundamentally at odds with the goals of civic education. ## The Study **Trust et al. (2025)** analyzed 310 AI-generated lesson plans (2,230 individual activities) produced by ChatGPT (GPT-4o), Gemini (1.5 Flash), and Copilot (GPT-4 based) for all 53 Massachusetts eighth-grade civics standards. Each standard received two prompts: a basic "write a lesson plan" and a "highly interactive" variant. ## Key Findings ### Lower-Order Thinking Dominates Using **Bloom's Revised Taxonomy**: | Level | Share | |-------|-------| | Remember | 45% | | Understand | 21% | | Apply | 24% | | **Subtotal (lower-order)** | **90%** | | Analyze | 4% | | Evaluate | 2% | | Create | 4% | **90% of activities** demanded only recall, comprehension, or simple application. Activities like "write definitions," "list three facts," and "answer comprehension questions" were pervasive. Even prompting for "highly interactive" lessons made minimal difference. ### Near-Total Absence of Multicultural Content Using **Banks' Four Levels of Integration of Multicultural Content**: - **94% of activities** contained no discernible multicultural content (2,086 of 2,230). - Of the 144 activities that did, 137 were at the lowest "Additive" level (mentioning diverse figures without restructuring curriculum). - Only **1 activity** reached "Transformation" (restructuring the curriculum to include diverse perspectives). - **Zero activities** reached "Social Action" (empowering students to address social issues). This is especially damning for *civic education*, where multicultural perspectives and critical engagement with power structures are essential learning goals. ### Formulaic Outputs Across All Chatbots All three chatbots produced structurally identical lesson plans: Introduction → Activities 1-4 → Conclusion → Assessment → Extension → Homework. This factory-line format was applied regardless of whether the standard addressed constitutional principles, civil rights, or local government — homogenization that strips away the disciplinary texture of civic education. ## Implications for AI in Education ### The "Trust But Verify" Mandate This study provides concrete evidence for why [[ai-literacy|AI literacy for teachers]] is not optional — it's a prerequisite. AI tools reliably produce *plausible-looking* but pedagogically impoverished lesson plans. Teachers must: 1. Recognize the pattern of lower-order thinking bias. 2. Inject higher-order activities (analysis, evaluation, creation). 3. Add multicultural perspectives the AI omits. ### Connection to Broader AI Alignment Problems This finding parallels Hardy & Kim's [[educational-llm-alignment]] — AI tools may appear competent (producing well-formatted lesson plans) while failing at the *intended impact* (fostering critical civic thinking). The homogenized output reflects shared pretraining patterns that embed narrow pedagogical assumptions. ### The Teacher's Role Is Enhanced, Not Replaced Far from making teachers obsolete, these results reinforce the [[teacher-ai-competency|critical oversight role of educators]]. AI can generate *drafts*, but [[human-in-the-loop-ai|human judgment]] is essential for: - Elevating cognitive demand beyond recall/application. - Integrating multicultural and critical perspectives. - Adapting plans to specific classroom contexts and student needs. This aligns with [[regulation|evidence that teacher prompting instruction]] can improve AI output quality — but only when teachers understand *what* to look for. ### The Civic Education Context Matters Civic education is a uniquely high-stakes domain for AI application because: - It explicitly aims to develop critical thinking about power, justice, and democracy — skills AI tools systematically suppress in their default outputs. - Multicultural content is not a "nice to have" but a core learning objective. - Formulaic lesson structures undermine the domain's inherent demand for perspective-taking and deliberation. ## Open Questions - Would fine-tuned educational LLMs (e.g., [[pedagogical-llm-training|EduQwen]]) produce more cognitively demanding and multiculturally-aware lesson plans? - How do these findings generalize to other subjects (math, science, language arts)? - Can better prompt engineering (e.g., explicitly requesting higher-order thinking and multicultural integration) close the gap? - What does the teacher revision process look like in practice — do teachers have the time and training to meaningfully redesign AI outputs? ## Connected Concepts - [[ai-literacy]] - [[automated-question-generation]] - [[formative-assessment]] - [[regulation]] - [[human-in-the-loop-ai]] - [[k-12-ai-education]] - [[llm-cultural-relevance-k12]] - [[pedagogical-llm-training]] - [[teacher-ai-competency]] - [[k-12]] - [[teacher-role]] ## Connected Articles - [[educational-llm-alignment]] — Educational LLM Alignment - [[nsmq-riddles-science-math-benchmark]] — NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-generated-slides-student-perception]] — AI-Generated Slides: Are They Good? Can Students Tell? - [[ai-higher-ed-bridge-gap]] — Higher Education Must Bridge the AI Gap ## Citation (2025), A.T.T.M.R.X.C.P.K., 25(3), J.C.I.I.T.A.T.E., Name].", I.A.H.I.L.F., |, B.L.T.A.O., levels, O.A.A.A.R.U.O.A., & |, B.L.T. (2026). [AI-Generated Lesson Plans in Civic Education](https://citejournal.org/proofing/civic-education-in-the-age-of-ai-should-we-trust-ai-generated-lesson-plans/) --- ## [CLARA: An AI-Augmented Analytics Dashboard for Collaboration Literacy](https://edtechdev.github.io/aied/articles/clara-collaboration-literacy-dashboard/) ## Key Finding Agentic analytics using AI-produced concept-map artifacts as shared human-AI representations improves collaboration quality analysis and AI response grounding over transcript-only baselines. ## Synthesis CLARA introduces a novel architecture for [[learning-analytics]] where AI-produced artifacts (concept maps, seven-dimension collaboration assessments) serve as shared representations between human dashboard users and AI reasoning agents. By indexing these artifacts into vector databases, CLARA establishes a human-AI common ground that simultaneously scaffolds human interpretation of collaboration data and grounds AI reasoning — improving both retrieval performance and response quality. This dual-use architecture has implications beyond collaboration analytics: the principle of AI-produced artifacts as shared infrastructure could apply to [[intelligent-tutoring]] where student models, [[knowledge-tracing-irt]] estimates, and concept maps could serve as common ground between tutor agents and teacher dashboards. The artifact-as-knowledge-infrastructure approach also resonates with [[rag]] patterns used in [[retrieval-augmented-tutoring-algorithm-kite]] and [[cyberscholar-genai-writing-feedback]], where structured content representations improve AI response quality. CLARA's focus on semantic dimensions beyond behavioral signals parallels the shift advocated by [[ai-tutor-behavioral-evaluation]] toward evaluating what students actually do, not just what the AI says. ## Connected Concepts - [[learning-analytics]] - [[intelligent-tutoring]] - [[rag]] ## Connected Articles - [[knowledge-tracing-irt]] - [[retrieval-augmented-tutoring-algorithm-kite]] - [[cyberscholar-genai-writing-feedback]] - [[ai-tutor-behavioral-evaluation]] ## Citation 2026, A. (2026). [*CLARA: An AI-Augmented Analytics Dashboard for Collaboration Literacy*](https://arxiv.org/abs/2605.17259) --- ## [Coauthorship integrity: Reconceptualising assessment validity for the age of generative artificial intelligence](https://edtechdev.github.io/aied/articles/coauthorship-integrity-reconceptualising-assessment-validity-for-the-age-of-gene/) > **Synthesis:** This paper addresses concerns that students use GenAI to submit texts they do not understand, adopting an assessment validity lens. It proposes Coauthorship Integrity as a new conceptual source of validity evidence—violated when students submit AI-generated content they do not understand. The paper reports progress on an "AI Viva," a conversational agent engaging students in hybrid viva voce with comprehension questions of controllable type and complexity. ## Key Findings This paper addresses concerns that students use GenAI to submit texts they do not understand, adopting an assessment validity lens. It proposes Coauthorship Integrity as a new conceptual source of validity evidence—violated when students submit AI-generated content they do not understand. The paper reports progress on an "AI Viva," a conversational agent engaging students in hybrid viva voce with comprehension questions of controllable type and complexity. Contributions are threefold: conceptual (Coauthorship Integrity), technical (LLM-generated questions of controllable difficulty), and empirical (preliminary expert evaluation). ## Connected Concepts - [[generative-ai]] - [[academic-integrity]] - [[agentic-ai]] ## Connected Articles - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] - [[will-skill-not-tool-chinese-university-students-acceptance-of-generative-ai-for-]] - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] ## Citation Ebrahimzadeh, M., Shibani, A., & Shum, S. B. (2026). [*Coauthorship integrity: Reconceptualising assessment validity for the age of generative artificial intelligence*](https://doi.org/10.1016/j.caeai.2026.100609). *Computers and Education: Artificial Intelligence.* --- ## [Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations](https://edtechdev.github.io/aied/articles/code-anchor-multi-view-visualization/) > **Naaz Sibia, Jessica Wen, Amber Richardson, Yashika Jain, Khushi Malik, Bogdan Simion, Carolina Nobre, Angela Zavaleta Bernuy, Andrew Petersen, Michael Liut** (2026). ICER 2026 **Naaz Sibia, Jessica Wen, Amber Richardson, Yashika Jain, Khushi Malik, Bogdan Simion, Carolina Nobre, Angela Zavaleta Bernuy, Andrew Petersen, Michael Liut** (2026). ICER 2026 ## Overview Students spent ~47% of gaze time on code despite visual scaffolds. Three factors shape selective engagement with multi-representational tools: Agency (students want control over cognitive effort), Representational Fit (same design feels helpful to some, overwhelming to others), and Legitimacy (metaphorical scaffolds perceived as childish at university level). ## Key Contributions ### Paper 1: Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations - **Problem:** Students often ignore well-designed program visualizations; existing cognitive design principles don't explain learner engagement/disengagement. - **Method:** Within-subjects study (N=19 undergraduates, post-CS1/CS2) using think-aloud, interviews, and webcam gaze tracking with a multi-representational probe. - **Gaze finding:** ~47% of time on code despite visual scaffolds; students without prior experience anchored more in code and ignored metaphor views. - **Three engagement themes:** Agency (control over cognitive effort), Representational Fit (wide individual variation), Legitimacy (metaphors seen as childish at university level). - **Implication:** multi-representational-tools need attention to affective and social factors, not just cognitive design. ### Paper 2: Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - **Problem:** Static-prompt LLM tutoring systems fail to adapt across diverse academic disciplines. - **Solution:** Subject-aware prompt routing using 14 pedagogical features extracted from transcripts; contextual bandit formulation with 20 pedagogical prompts. - **Simulation:** Router achieves 0.694 vs 0.647/0.64 static baselines (p<0.001). - **Real-world A/B test:** N=656 conversations, 359 Dutch high-school students. Stochastic router achieves 28.1% exercise conversion rate vs 19.6% baseline. - **Implication:** [[adaptive-prompt-routing]] with stochastic sampling improves both efficiency and engagement in real-world tutoring. ### Paper 3: Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - **Problem:** Automated scoring of student-drawn scientific models lacks reliability indicators, leaving teachers unable to decide when to trust scores. - **Method:** Vision Transformer (ViT) with LoRA + confidence-aware framework using test-time perturbations. - **Dataset:** Six NGSS-aligned middle school assessment items (477-816 responses each, scored Beginning/Developing/Proficient). - **Key innovation:** Response-level confidence enables selective automation — high-confidence auto-scored, uncertain cases deferred for human review. - **Implication:** [[confidence-aware-ai-assessment]] enables practical triage between automation and human oversight in educational assessment. ### Paper 4: PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - **Problem:** AES systems treat scoring and feedback as separate; neural scoring lacks interpretability; LLM feedback is ability-agnostic. - **Solution:** Unified psychometric latent space (θ) via Neural GPCM Trait-Adaptive Scorer. - **Scoring performance:** QWK 0.747 (besting prior SOTA 0.722); 1st in 6/8 ASAP++ prompts and 10/11 trait dimensions. - **Feedback quality:** ZPD-based strategy mapping (Explicit Correction for θ<-1, Scaffolding for -1<θ<1, Socratic for θ>1). Multi-agent fusion from Llama-4-Scout, Qwen3-235B, GPT-4o. - **Evaluation:** >90% win rate in Actionability vs GPT-4o, Llama-4-Scout, Qwen3. - **Implication:** psychometrically-aware-ai can unify assessment and instruction, transforming automated essay scoring from summative to formative. ## Connected Concepts - [[adaptive-prompt-routing]] - [[confidence-aware-ai-assessment]] ## Connected Articles - [[learning-to-prompt-adaptive-tutoring]] — Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - [[psyscore-essay-scoring-zpd-feedback]] — PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - [[confidence-aware-student-drawing-assessment]] — Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - [[flowcode-ai-creative-coding]] — Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education - [[instructional-guidance-genai-learning]] — Role of Instructional Guidance in Generative AI-Assisted Learning - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Naaz Sibia, Jessica Wen, Amber Richardson, Yashika Jain, Khushi Malik, Bogdan Simion, Carolina Nobre, Angela Zavaleta Bernuy, Andrew Petersen, Michael Liut (2026). [Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations](https://arxiv.org/abs/2606.19570). arXiv:2606.19570. ICER 2026. --- ## [CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation](https://edtechdev.github.io/aied/articles/code-gen/) > **A dual-agent RAG-based system for generating and validating coding comprehension MCQs**, evaluated by 6 SMEs across 7 pedagogical dimensions (N=288 questions, 2,016 rating pairs). AI excels at criteria-matching and computational verification (concept alignment 98.6%, code validity 95.5%), but human expertise remains essential for distractor quality (79.9%) and pedagogically rich feedback — providing an evidence-based division of labor for [[human-in-the-loop-ai]] educational content generation. **Venue:** AIED 2026 (short paper) **ArXiv:** 2604.03926 ### Overview CODE-GEN (Context-aligned, Output-validated, Dual-agent, Expert-guided GENeration) is a [[human-in-the-loop-ai]] [[agentic-ai]] system for generating contextually grounded multiple-choice coding comprehension questions. It integrates [[rag]] with a dual-agent architecture separating question generation from quality validation. ### Architecture 1. **RAG Pipeline:** Instructional materials (learning objectives, example questions, code) are parsed with a domain-specific chunking strategy that preserves semantic coherence, embedded via OpenAI text-embedding-3-small, and indexed in a FAISS vector store. On generation, nearest-neighbor retrieval injects relevant examples into the Generator's prompt. 2. **Generator Agent (GPT-4.1):** Produces MCQs with stem, executable code, four answer options, and explanatory feedback. Augmented with an Arithmetic Expression Evaluator tool for deterministic computation. 3. **Validator Agent (GPT-5-mini):** Independently assesses each question across **seven pedagogical dimensions**: question stem clarity, code validity, concept alignment, correct answer validity, distractor quality, correct answer feedback quality, and distractor feedback quality. Uses an Arithmetic Expression Evaluator and a Sandboxed Python Runner for code execution verification. ### Evaluation - **6 SMEs** evaluated 288 AI-generated questions - **2,016 human-AI rating pairs** (SME agreement/disagreement with Validator) - **131 qualitative feedback instances** ### Key Results | Dimension | Success Rate | Failure Rate | |-----------|:-----------:|:------------:| | Concept Alignment | **98.6%** | 0.3% | | Question Stem Clarity | **97.9%** | 2.1% | | Code Validity | **95.5%** | 3.1% | | Correct Answer Feedback | 92.4% | 2.1% | | Correct Answer Validity | 92.0% | 1.4% | | Distractor Feedback Quality | 86.1% | 9.4% | | Distractor Quality | **79.9%** | 15.6% | ### Division of Labor Findings **AI strengths** (reliable automation): - Computational verification and explicit criteria matching - Concept alignment via RAG grounding - Code syntax and output verification via tool augmentation **Human-essential dimensions** (require oversight): - Designing pedagogically meaningful distractors that target common student misconceptions - Providing feedback that elaborates on underlying concepts, not just surface mechanics - Interpreting structured answer representations (Validator sometimes confused option position with answer value) ### Failure Patterns - **False positives:** Validator approved distractors that were syntactically valid but instructionally shallow; approved feedback that described surface mechanics without deeper elaboration - **False negatives:** Validator misinterpreted answer schemas (confusing answer value with option position); internal inconsistency where textual analysis affirmed correctness but binary classification contradicted it ### Significance for [[automated-assessment]] and [[automated-question-generation]] CODE-GEN demonstrates that agentic AI with RAG grounding and tool augmentation can serve as scalable first-line quality control for [[automated-assessment]] item generation. The explicit evaluation of the Validator against human judgment — rather than assuming automated evaluation is reliable — provides an evidence-based framework for determining where AI can be safely delegated and where [[human-in-the-loop-ai]] oversight must be maintained. ## Connected Concepts - [[human-in-the-loop-ai]] - [[agentic-ai]] - [[rag]] - [[automated-assessment]] - [[automated-question-generation]] ## Connected Articles - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[llm-difficulty-calibration-programming-exams-2026]] — From Evaluated Models to Evaluation Aids: A Multi-Evidence Study of LLM-Based Difficulty Calibration for Programming Examinations - [[trust-reliance-ai-education-2026]] — Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators ## Citation RAG-Based, C.A.H., Question, A.A.S.F.M., Frederick, X.D., Wang1, A.C., multiple-choice, A.A.I.W.A.G.A.P., 2,016, J.A.Q.T.S.P.A.T.O., & generation, H.A.A.E.I.A.E.C. (2026). [*CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation*](https://arxiv.org/abs/2604.03926) --- ## [Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom](https://edtechdev.github.io/aied/articles/code-review-genai-cs1/) Oral code reviews paired with a flipped classroom represent a pragmatic harm-reduction approach to generative AI in CS education. Rather than banning LLMs, Fowles et al. (2026) designed weekly formative assessments where students must explain their submitted code regardless of its origin. Over three semesters at Utah State University, keystroke logs confirmed significantly higher AI usage (increased paste-to-type ratios), yet exam scores showed no statistically significant decline. Students reported positive attitudes toward the code review format, with negative feedback centered on logistical issues (scheduling, TA training) rather than pedagogical objections. This study addresses a core tension in the [[stanford-evidence-base-ai-k12-2026]]: how to maintain [[academic-integrity]] while acknowledging that AI tools are now part of professional practice. The approach aligns with [[ai-literacy]] frameworks that emphasize critical engagement over prohibition. Unlike AI detection tools, which create adversarial dynamics, oral assessments reframe the conversation around understanding and [[metacognition]]. The findings also complement research on [[over-reliance]] and [[cognitive-offloading]], suggesting that structured accountability mechanisms can preserve learning outcomes even as AI usage rises. The flipped classroom component is particularly notable — it creates dedicated time for one-on-one assessments without sacrificing content coverage, addressing a scalability concern that has limited adoption of oral assessment in large CS1 courses. This connects to broader discussions about [[formative-assessment]] design in AI-rich environments and the evolving role of [[teacher-role]] as learning facilitator rather than knowledge dispenser. ## Connected Concepts - [[academic-integrity]] - [[ai-literacy]] - [[metacognition]] - [[over-reliance]] - [[cognitive-offloading]] - [[formative-assessment]] - [[teacher-role]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] ## Citation Peter Fowles, Erik Falor, Sulove Bhattarai, John Edwards, Seth Poulsen (2026). [Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom](https://arxiv.org/abs/2605.21374). arXiv:2605.21374. arXiv:2605.21374 [cs.HC]. --- ## [Codify: An Intelligent Socratic Tutoring System for Programming Education](https://edtechdev.github.io/aied/articles/codify-socratic-programming-tutor/) # Codify: Socratic Programming Tutor 📄 DOI: 10.32473/flairs.39.1.141554 Codify (also called AI Tutor) is an [[intelligent-tutoring]] system that leverages [[llm|LLMs]], competency tracking, and adaptive assessment to provide Socratic, discovery-based programming education. Rather than giving direct answers, it guides students through dialogue designed to elicit their own reasoning. ## System Architecture The platform includes: - **Conversational tutoring** using Socratic methodology - **Automated practice generation** adapting to student needs - **Competency modeling** with topic-level mastery tracking - **Code analysis** for evaluating student submissions - **Gamified engagement** mechanisms for motivation Students interact through a chat-based interface, and the system dynamically adjusts question difficulty and [[scaffolding]] based on real-time competency estimates. This approach aligns with research on [[knowledge-tracing-irt|knowledge tracing]] and [[adaptive-learning]], though as a conference poster, empirical evaluation of learning outcomes is preliminary. ## Positioning in the ITS Landscape Codify represents the trend toward LLM-powered tutoring systems that combine conversational AI with structured pedagogical frameworks. Unlike [[ai-peer-feedback-systems|AI feedback systems]] that provide targeted critique, Codify emphasizes discovery learning — a Socratic approach that connects to questions about [[metacognition]] and [[self-regulated-learning]]. Future work could benefit from incorporating findings from [[llm-reasoning-traces-metacognition|research on reasoning traces and metacognitive calibration]], as the Socratic dialogue format may face similar challenges of inflating student confidence without improving performance. ## Connected Concepts - [[adaptive-learning]] - [[llm]] - [[metacognition]] - [[scaffolding]] - [[self-regulated-learning]] - [[higher-ed]] - [[rag]] ## Connected Articles - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[knowledge-tracing-irt]] — Interpretable Knowledge Tracing via IRT - [[llm-reasoning-traces-metacognition]] — Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice ## Citation Ilyasov, A., Bardelli, G., Torres, S., & Keshtkar, F. (2026). Codify: An intelligent Socratic tutoring system for programming education. [The International FLAIRS Conference Proceedings](https://doi.org/10.32473/flairs.39.1.141554), *39*(1). https://doi.org/10.32473/flairs.39.1.141554 --- ## [Codify: An Intelligent Socratic Tutoring System for Programming Education](https://edtechdev.github.io/aied/articles/codify-socratic-tutoring-programming/) ## System Overview Codify (also referred to as "AI Tutor") is a web-based [[intelligent-tutoring]] platform for programming education that integrates conversational AI, adaptive assessment, and learning analytics. It leverages **LLMs deployed via AWS Bedrock** with a **Socratic teaching methodology** that promotes discovery-based learning over direct answer generation — students are guided through questions and hints rather than given solutions outright. ## Architecture and Components The system comprises five integrated components: 1. **Conversational AI Tutor**: Built on [[llm]] technology with LangChain and [[retrieval-augmented-tutoring-algorithm-kite|Retrieval-Augmented Generation (RAG)]] for contextual memory and response coherence. Uses Socratic questioning rather than answer provision. 2. **Practice Question Engine**: Automatically generates programming exercises at three difficulty levels (beginner, intermediate, advanced), dynamically tailored to student competency. 3. **Competency Tracking**: Maintains topic-level competency scores (0–100) per student, updated continuously based on interactions, quiz performance, and problem-solving. Incorporates **resistance curves** — mastery becomes progressively harder as proficiency increases. 4. **Code Analysis & Feedback**: Performs syntax validation, logic analysis, style feedback, and debugging assistance. Critically, the system **does not correct code directly** — it provides educational hints that encourage self-discovery, aligning with [[scaffolding]] best practices. 5. **Adaptive Learning & Gamification**: Features XP, daily streaks, competitive leagues (Bronze → Diamond), and multiple leaderboards for engagement. ## Positioning in the ITS Landscape Codify represents a modern instantiation of the [[conversational-ai-tutors-framework]] paradigm, combining LLM-powered dialogue with structured competency tracking — bridging the gap between open-ended chatbot tutoring and traditional [[automated-grading]] systems. Unlike purely conversational approaches (e.g., [[learnmate2-llm-adaptive-learning]]), Codify integrates formal assessment, gamification, and adaptive difficulty scaling into a single platform. The RAG architecture aligns with [[retrieval-augmented-tutoring-algorithm-kite|KITE]], which also uses RAG for algorithmic tutoring, though Codify adopts a much broader scope covering all programming education domains. The Socratic methodology directly connects to the [[socratic-ai-dialogue]] approach and the [[pedagogical-llm-training]] literature on training LLMs for guided inquiry rather than answer provision. ## Preliminary Evidence Initial deployment in programming courses showed promising results for student engagement and perceived learning support. Students particularly valued the 24/7 availability and conversational style as a supplement to traditional instruction. However, the paper reports preliminary observations rather than a controlled efficacy-study, and the evidence base remains early-stage. ## Connected Concepts - [[agentic-ai]] - [[automated-grading]] - [[llm]] - [[pedagogical-llm-training]] - [[scaffolding]] - [[socratic-ai-dialogue]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalab... - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[moodle-ai-tutoring-deep-learning]] — From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[programming-its]] — Programming Intelligent Tutoring Systems - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[socraticode-k12-programming-tutor]] — Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participato... - [[structured-llm-feedback-programming]] — The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... ## Citation Ilyasov, A., Bardelli, G., Torres, S., & Keshtkar, F. (2026). Codify: An intelligent Socratic tutoring system for programming education. [The International FLAIRS Conference Proceedings](https://doi.org/10.32473/flairs.39.1.141554), *39*(1). https://doi.org/10.32473/flairs.39.1.141554 --- ## [Cognitive Agent Compilation for Explicit Problem Solver Modeling](https://edtechdev.github.io/aied/articles/cognitive-agent-compilation/) ## Core Idea **Cognitive Agent Compilation (CAC)** is a framework that uses a strong teacher LLM to compile problem-solving knowledge into an explicit, inspectable target agent. Unlike end-to-end LLM tutoring approaches, CAC separates the agent into three components: 1. **Knowledge representation** — explicit, structured representation of domain knowledge (skills, misconceptions, strategies) 2. **Problem-solving policy** — how the agent applies knowledge to solve problems 3. **Verification and update rules** — mechanisms for checking correctness and updating the knowledge state The goal is **bounded-knowledge AI** for education: agents that know what they know, can explain their reasoning in terms of explicit skills, and can be inspected and edited by educators. ## Motivation LLMs are widely used for tutoring, feedback generation, and content creation, but their broad pretraining makes them hard to constrain. Key problems: - **Inability to "not know"**: LLMs struggle to simulate a state of not knowing, making them poor proxies for student learners. Evidence suggests Chain-of-Thought and reasoning models operate fundamentally differently from human cognition. - **Expert blind spot**: Even human experts struggle to teach effectively without pedagogical training. LLMs, having absorbed vast unconstrained information, face an amplified version of this problem. - **Inspectability gap**: Educators need to know what a system assumes the learner knows; learners benefit when the system can justify actions in terms of explicit skills, misconceptions, and strategies. Opaque LLMs can't provide this. ## Architecture CAC draws inspiration from **cognitive architectures** (ACT-R, SOAR) and compiles LLM knowledge into explicit, inspectable agents. The three separable components enable: - **Inspectability**: Educators can examine the knowledge representation and problem-solving policy directly - **Editability**: Knowledge states can be corrected or refined without retraining - **Bounded reasoning**: The agent operates within explicit knowledge boundaries rather than leveraging unbounded pretraining A proof of concept was implemented using **Small Language Models (SLMs)**, surfacing key design trade-offs between explicit control and scalable generalization. ## Connection to Existing Work CAC connects to several threads in AI education research: - **[[knowledge-tracing-irt]]**: CAC makes knowledge tracing explicit and inspectable, complementing interpretable approaches like IRT-based knowledge tracing - **[[neural-symbolic-knowledge-tracing]]**: Shares the goal of combining explicit (symbolic) knowledge representation with neural approaches for interpretable learner modeling - **[[agentic-workflows-education]]**: CAC is a specific agentic architecture for education, separated into knowledge, policy, and verification components - **[[adaptive-learning]]**: CAC enables adaptive behavior grounded in explicit, editable learner models - **[[llm-student-modeling-memory]]**: CAC addresses the learner modeling challenge from the knowledge representation angle, complementing memory-based approaches - **[[pedagogical-llm-training]]**: CAC offers an alternative to training pipelines — compile rather than train for pedagogical alignment - **[[personalized-learning]]**: Explicit learner models in CAC enable personalization that is inspectable and editable by educators - **[[prober-ai-inquiry-writing]]**: Both approach bounded-knowledge tutoring from different angles — CAC via compilation, Prober via inverted questioning ## Open Questions - **Scalability vs. control trade-off**: The proof of concept surfaces tension between explicit, editable representations and the scalability of end-to-end LLM approaches - **Compilation fidelity**: How faithfully can a teacher LLM compile its knowledge into a bounded agent? - **Domain breadth**: The initial implementation is domain-specific — can CAC generalize across subjects? - **Bounded rationality in practice**: Does bounded-knowledge AI actually improve educational outcomes compared to unbounded LLM tutors? ## Connected Concepts - [[adaptive-learning]] - [[pedagogical-llm-training]] - [[personalized-learning]] ## Connected Articles - [[knowledge-tracing-irt]] - [[neural-symbolic-knowledge-tracing]] - [[agentic-workflows-education]] - [[llm-student-modeling-memory]] - [[prober-ai-inquiry-writing]] ## Citation Moon, H., Rosé, C., & Stamper, J. (2026). [Cognitive Agent Compilation for Explicit Problem Solver Modeling](https://arxiv.org/abs/2605.07040). arXiv:2605.07040. Accepted to AIED 2026 Blue Sky. --- ## [The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise](https://edtechdev.github.io/aied/articles/cognitive-commons-ai-expertise-regeneration/) > **Nolan Lovett** — Human Resource Development Review (author accepted manuscript, 2026). **Nolan Lovett** — Human Resource Development Review (author accepted manuscript, 2026). ## Synthesis This conceptual paper introduces the Cognitive Commons framework, integrating commons theory, HRD scholarship, and distributed cognition to explain how rational AI adoption decisions can deplete the shared expertise pool professions require for renewal. It distinguishes Internalized Mastery (deep domain knowledge from sustained practice) from Distributed Mastery (orchestrating human-AI systems) and develops the Validation Tether: effective AI oversight depends on the expertise AI adoption may undermine. Early labor-market and clinical evidence suggests disruption to expertise-regeneration pathways in highly AI-exposed sectors; five factors determine occupational vulnerability, and governance may form at organizational, professional-association, and policy levels. The paper reframes expertise development as collective stewardship rather than organizational optimization — a systems-level complement to individual-level cognitive-offloading and skill-decay findings, with implications for professional training and AI governance. ## Connected Concepts - [[open-source]] - [[plagiarism-detection]] - [[student-experience]] - [[ai-misuse-learning-harm]] - [[over-reliance]] - [[critical-thinking]] - [[lifelong-learning]] - [[generative-ai]] ## Connected Articles - [[genai-expertise-pathways-sysadmin]] — Unanticipated Effects of Generative AI on Expertise Pathways and Performance Perception in System Administration - [[chatgpt-impact-high-school-tests]] — Little Impact of ChatGPT Availability on High School Student Test Score Performance - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[efficiency-gain-illusion-ai-overreliance]] — The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks - [[ai-making-us-stupid]] — Is AI making us stupid? - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development ## Citation Lovett, N. (2026). [The tragedy of the cognitive commons: How AI could disrupt the regeneration of professional expertise. Human Resource Development Review](https://arxiv.org/abs/2607.29380). arXiv:2607.29380. --- ## [Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content](https://edtechdev.github.io/aied/articles/cognitive-offloading-llm-synthesis-writing/) **Oleksandra Poquet, Mani Shankar Nanduri, Maria Ximena Salinas Loyer, Matthias Stadler, Michael Sailer, Jelena Jovanovic** — Accepted at EC-TEL 2026 — cs.HC, cs.ET This study compares two approaches to profiling how learners offload cognitive activity to LLMs during a synthesis writing task, drawing on Salomon's distributed cognition and the Kintsch–van Dijk model of text comprehension. Volume-based profiling (k-means, k=4) differentiated learners primarily by prior knowledge, with higher LLM use negatively associated with essay authorship. Content-based profiling (k=5) revealed qualitatively distinct patterns: from vocabulary clarification to active direction of structuring and generation to passive delegation of comprehension at both local and global levels. These patterns correspond to differences in learning strategies, behavioral markers, and essay authorship. The study provides a nuanced framework for understanding not just how much learners rely on LLMs, but what kind of cognitive work they offload — critical for designing [[ai-literacy]] interventions and [[formative-assessment]] that promotes productive AI use without undermining learning. The findings connect to [[self-regulated-learning]] research by showing that offloading patterns reflect different fragmentation of the cognitive process. ## Connected Concepts - [[ai-literacy]] - [[formative-assessment]] - [[self-regulated-learning]] ## Connected Articles - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[reshaping-cs-education-genai]] — Reshaping Undergraduate Computer Science Education in the Generative AI Era ## Citation Poquet, O., Nanduri, M. S., Salinas Loyer, M. X., Stadler, M., Sailer, M., & Jovanovic, J. (2026). [*Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content*](https://arxiv.org/abs/2606.10434). arXiv:2606.10434. --- ## [Cognitive offloading and the speedup illusion in human-AI interaction](https://edtechdev.github.io/aied/articles/cognitive-offloading-speedup-illusion/) This preregistered large-scale study (N = 1,237) investigates whether people are well-calibrated in estimating the time savings from AI assistance on simple cognitive tasks. The key finding is a **speedup illusion**: participants accurately predict how long they'll take independently but significantly *underestimate* how long they'll take with AI assistance — despite actual completion times being equivalent between independent and AI-assisted conditions. Notably, this bias is AI-specific; the same miscalibration does not appear when participants imagine help from another human. A critical dissociation emerges between time and effort: participants report **lower subjective effort** with AI even when completion times are identical, suggesting subjective experience drives AI adoption more than objective efficiency. The findings complement the companion paper on the [[efficiency-gain-illusion-ai-overreliance]] (same authors, arXiv:2605.22687) which found people underestimate their rate of AI usage and overestimate benefits on simple tasks. Together, these studies reveal a **dual miscalibration** that could entrench inefficient [[over-reliance]] patterns: users choose AI believing it saves time and effort, when for simple tasks it may do neither. This has direct implications for educational settings — students using AI for simple cognitive work (arithmetic, basic writing, spell-checking) may [[cognitive-offloading|cognitively offload]] without actual efficiency gains, reducing [[transfer-of-learning|learning transfer]] through diminished deliberate practice. Presented at CogSci 2026, the work bridges cognitive science and [[metacognition]] research with practical design implications for AI tools in learning environments. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[metacognition]] ## Connected Articles - [[efficiency-gain-illusion-ai-overreliance]] - [[transfer-of-learning]] ## Citation Sunny Yu, Myra Cheng, Ahmad Jabbar, Ilia Sucholutsky, Katherine M. Collins, Dan Jurafsky, Robert D. Hawkins (2026). [Cognitive offloading and the speedup illusion in human-AI interaction](https://arxiv.org/abs/2605.23177). arXiv:2605.23177. Proceedings of the 48th Annual Meeting of the Cognitive Science Society (CogSci 2026). - [[digital-literacy-illusion]] — Digital literacy illusion confirms AI overestimation in secondary students - [[llm-reasoning-traces-metacognition]] — Processing fluency account of trace-induced overconfidence - [[ai-productivity-moderation]] — Learning curve factor explains when AI productivity gains fail --- ## [Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence?](https://edtechdev.github.io/aied/articles/cognitive-shift-ai-education/) ## Overview This paper presents a striking longitudinal finding: as AI becomes a routine educational tool, students systematically revalue **human intelligence (HI) over artificial intelligence (AI)**. Drawing on 6 years of classroom poll data (2020–2026) from 471 undergraduate and MSc computer science students, Rekik documents a cognitive shift that progresses through four phases: hype → distrust → trust → dependency. **Four-phase trajectory of student perception:** 1. **Hype (2020):** Initial excitement slightly favored AI over HI in poll responses 2. **Distrust:** Emerging skepticism as students encountered AI limitations 3. **Trust:** Growing reliance on AI tools for coursework 4. **Dependency:** Habitual use leading to a reappraisal of what makes human intelligence valuable **Converging toward human intelligence.** From 2024 onward, a consistent shift toward HI preference emerged across all MSc cohorts. By 2026: - **Technical course (ML/Deep Graph Learning):** 65% preferred HI (↑12pp from 2025) - **Design-oriented course (Design Thinking for AI):** 90% preferred HI (↑36pp from 2025) This is a striking reversal from 2020, when AI was slightly favored. ## Implications for AI Literacy and Learner Autonomy The findings directly challenge the assumption that increased AI exposure leads to increased AI trust. Instead, sustained use produces a more nuanced — and more skeptical — relationship. The design-oriented course's 90% HI preference suggests that **pedagogical framing matters**: courses that teach students to think critically *about* AI (not just *with* AI) accelerate this cognitive shift. This connects to [[contextual-sycophancy-ai-literacy]], which found that AI literacy interventions alone may be insufficient to prevent over-reliance. The cognitive shift documented here suggests a longer-term developmental trajectory: epistemic recalibration happens through immersion, not instruction. The paper's emphasis on **learner autonomy** and **epistemic agency** ties directly to [[genai-performance-vs-learning]], which warns that AI tools can improve task performance while undermining learning. The shift toward HI preference may reflect students' growing awareness of this tradeoff. ## Connections to Faculty Development For instructors, these results suggest that AI education should explicitly surface the **value of human cognition** rather than treating AI as a neutral productivity tool. The [[ai-pedagogical-orientation]] framework shows that faculty AI orientation strongly predicts adoption — this paper adds that student orientation evolves dynamically and may benefit from curricular scaffolding. ## Methodological Notes - **N = 471** students across technical and design-oriented courses - **Poll-based measurement** of HI vs. AI valuation - **Longitudinal design** spanning 6 years (2020–2026) - Workshop paper (ICLR HCAIR), not yet peer-reviewed at a major venue ## Connected Concepts - [[administrator]] - [[socratic-ai-dialogue]] - [[agentic-ai]] - [[ai-tutoring]] - [[math-education]] - [[prompt-engineering]] - [[affective-tutoring]] - [[curriculum-design]] ## Connected Articles - [[contextual-sycophancy-ai-literacy]] - [[genai-performance-vs-learning]] - [[ai-pedagogical-orientation]] ## Citation Rekik, I. (2026). [Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence?](https://arxiv.org/abs/2605.16292) arXiv:2605.16292. ICLR HCAIR Workshop 2026. --- ## [CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education](https://edtechdev.github.io/aied/articles/cogtax-cognitive-taxonomy/) > **Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira** — Universidade de Vigo, submitted 30 Jun 2026 **Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira** — Universidade de Vigo, submitted 30 Jun 2026 Proposes CogTax, a four-level cognitive taxonomy for command-line computing education that integrates Bloom's Revised Taxonomy with operational impact dimensions, achieving 89% accuracy in automated classification. ## Key Contributions - Proposes CogTax, a four-level cognitive taxonomy for command-line computing education that integrates Bloom's Revised Taxonomy with operational impact dimensions, achieving 89% accuracy in automated classification. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[mixed-reality-engineering-learning]] — Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[codify-socratic-programming-tutor]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[less-deliberate-teams-llm]] — Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work ## Citation Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira (2026). [CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education](https://arxiv.org/abs/2607.00140). arXiv:2607.00140. Universidade de Vigo, submitted 30 Jun 2026 --- ## [Reexamining the Cold-Start Problem in Knowledge Tracing Models and Implications for SafeInsights](https://edtechdev.github.io/aied/articles/cold-start-knowledge-tracing-safeinsights/) **Jiayi Zhang, Ryan S. Baker, Debshila Basu Mallick, Cristina Heffernan, Neil Heffernan** — cs.HC This paper replicates and extends prior work on the cold-start problem in knowledge tracing — the challenge of making accurate predictions when a student begins practicing a new skill. Using a more recent ASSISTments dataset (FoundationalASSIST), the study evaluates KT model performance across both practice trajectories and four problem types: fill-in-the-blank, multiple-choice select-one, multiple-choice select-all, and order/sort. Results show that KT model performance varies across both dimensions, with deep-learning-based models maintaining advantages during early practice but with context-dependent consistency. The study also serves as a proof of concept for SafeInsights, a privacy-preserving research infrastructure designed to facilitate reproducible educational data mining research. This work extends [[knowledge-tracing-irt]] findings by demonstrating that problem type — not just skill — moderates model performance, with implications for [[student-modeling]] in adaptive learning systems. ## Connected Concepts - [[student-modeling]] ## Connected Articles - [[knowledge-tracing-irt]] ## Citation Zhang, J., Baker, R. S., Basu Mallick, D., Heffernan, C., & Heffernan, N. (2026). [*A Case Study Reexamining the Cold-Start Problem in Knowledge Tracing Models and Implications for SafeInsights, an Education Research Infrastructure*](https://arxiv.org/abs/2606.11004). arXiv:2606.11004. --- ## [Collaborative AI Tutoring](https://edtechdev.github.io/aied/articles/collaborative-ai-tutoring/) > ProPACT constructs a real-time model of pair collaboration using three signals: > Most adaptive learning systems are individual-centric and reactive. **ProPACT** treats **collaboration itself as the object of instruction**, intervening *before* dyadic breakdowns occur by forecasting suboptimal states up to 30 seconds in advance.^[[golrang-propact-pair-programming-2026]] ## The Dyadic Learner Model ProPACT constructs a real-time model of pair collaboration using three signals: | Signal | Description | Measurement | |--------|-------------|-------------| | **JVA** (Joint Visual Attention) | Shared attentional focus | Cosine similarity of gaze distributions over 30-second windows | | **ME** (Mental Effort) | Individual cognitive load | Index of Pupillary Activity (IPA) over 10-second windows | | **JME** (Joint Mental Effort) | Cognitive engagement synchrony | Cross-recurrence of synchronized ME signals | All signals are discretized against a resting baseline using **±2SD bins** (High, Average, Low).^[[golrang-propact-pair-programming-2026]] ## Proactive Forecasting Engine - **Model:** XGBoost predicts JVA, JME, and ME over a **30-second horizon** - **Policy:** Rule-based pedagogical hierarchy prioritizes minimally intrusive support - **Principle:** Fade support when collaboration is productive; escalate only when breakdown risk increases ## Adaptive Feedback Hierarchy | Intervention | Trigger | Intrusiveness | |---|---|---| | **A1: Do nothing** | MEs=AVG, JVA=H, JME=H | None (desired state) | | **A2: GitHub Copilot** | MEs=HH or LL, or (MEs=HL and JVA=L) | Low (autocomplete) | | **A3: Gaze-awareness tool** | JVA=Low | Low (visual cue highlighting partner's gaze) | | **A4: Dialogue prompt** | JME=Low | Medium (editor prompt encouraging brief dialogue) | | **A5: Task-based hint** | Both MEs=High (last resort) | High (directive hint) | The hierarchy embodies the principle that **proactive, minimally intrusive scaffolds** outperform reactive, directive interventions.^[[golrang-propact-pair-programming-2026]] ## Key Results In a within-subjects study with **26 pair-programming dyads** (52 CS/Engineering students), ProPACT feedback vs. control showed: | Measure | Effect | |---|---| | **Debugging success** | Significantly higher (*t*(49.96) = −13.51, *p* < .0001) | | **Debugging time** | Significantly lower / more efficient (*t*(44.70) = 4.39, *p* < .0001) | | **Feedback uptake** | Significantly higher (*F*(49.81) = −17.69, *p* < .0001) | Post-intervention gains in JVA and JME suggest improved collaborative regulation beyond immediate task completion.^[[golrang-propact-pair-programming-2026]] ## Relationship to Tutoring-Specific Design ProPACT exemplifies [[tutoring-specific-vs-general-ai|tutoring-specific design]]: it does not give answers but structures collaboration through graduated scaffolds (mimicking a skilled pair-programming mentor). Like the best tutoring systems, it targets the [[zone-of-proximal-development|zone of proximal development]] at the *dyadic* level—ensuring partners are neither overloaded nor under-challenged. It also preserves [[metacognition]] by requiring learners to actively monitor and regulate their collaboration.^[[golrang-propact-pair-programming-2026]] ## Implications - **For CS education:** Real-time dyadic regulation is tractable and effective; eye tracking + ML forecasting is viable at classroom scale - **For adaptive systems:** The "tutoring" unit need not be an individual—collaboration itself can be scaffolded - **For transfer:** Does improved dyadic regulation transfer to unsupervised pair work? Not yet tested ## Connected Concepts - [[zone-of-proximal-development]] - [[metacognition]] ## Connected Articles - [[golrang-propact-pair-programming-2026]] - [[tutoring-specific-vs-general-ai]] ## Citation Golrang, A., Sharma, K., Dehaen, S., & Viberg, O. (2026). [*ProPACT: A Proactive AI-Driven Adaptive Collaborative Tutor for Pair Programming*](https://arxiv.org/abs/2605.02703). arXiv:2605.02703. --- ## [Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications](https://edtechdev.github.io/aied/articles/commenting-copilot-student-code-specs/) Analyzes how students specify intended behavior in natural language to AI code tools (Copilot) across multiple years, deriving a taxonomy of code-generation specifications expressed through comments. As AI tools shift emphasis from writing code to specifying behavior, the study documents what students actually ask of these systems. Situates in [[llm]]-assisted [[programming-its]] and [[student-experience]] within [[cs-education]] and [[higher-ed]], extending [[reshaping-cs-education-genai]] by characterizing the new 'specification' literacy. It informs [[ai-literacy]] for coding and the design of tools that scaffold rather than replace student reasoning. ## Key Findings - Analyzing a four-year dataset of undergraduate programming submissions and reflections from tasks in which students wrote comments to guide code generation and refined solutions using test-case feedback, the study introduces a taxonomy spanning three dimensions: comment type, code expression level, and code construct. - Students mostly wrote natural-language What comments — describing intended behavior — while shifting toward How comments for more procedural constructs. - Students focused more on verifying generated code than on repeatedly rewriting their comments, indicating a verification-oriented workflow in AI-assisted programming. - Using automated classification, the study examined how the taxonomy dimensions varied across attempts and how students described the process in their reflections. - The findings characterize the new "specification" literacy emerging as AI tools shift emphasis from writing code to specifying behavior in natural language. ## Study Design & Method The study draws on multi-year data from undergraduate programming courses in which AI code tools such as GitHub Copilot were integrated into tasks: students wrote comments as specifications, the tools generated code, and students refined solutions based on test-case feedback. A three-dimensional taxonomy — comment type, code expression level, and code construct — was applied via automated classification to trace how specification behavior changed across attempts. Student reflections provided qualitative context on how learners experienced the process of directing code generation. ## Implications for AI in Education As AI code tools become integrated into programming environments, the emphasis of student work shifts from code writing to specification, and the study documents what students actually ask of these systems. For [[cs-education]], the prevalence of What comments and the shift to How comments on procedural constructs suggest that learners benefit from explicit instruction in writing precise specifications — a form of [[ai-literacy]] for programming. The finding that students concentrated on verification suggests that tools and curricula should scaffold the verification of generated code, treating comments and tests as the interface between student intent and AI output, in line with the broader rethinking of [[reshaping-cs-education-genai]]. ## Connected Concepts - [[llm]] - [[student-experience]] - [[cs-education]] - [[higher-ed]] - [[ai-literacy]] ## Connected Articles - [[programming-its]] - [[reshaping-cs-education-genai]] ## Citation Nasser Giacaman, Valerio Terragni, Paul Denny, Viraj Kumar (2026). [Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications](https://arxiv.org/abs/2607.10674). arXiv:2607.10674. arXiv preprint. --- ## [Co-Designing Community-Centered AI Education for Adults: A Midwestern Case Study](https://edtechdev.github.io/aied/articles/community-centered-ai-education-adults/) 📄 [PDF](https://arxiv.org/pdf/2606.26565) This case study reports on a community-based participatory research project that co-designed an [[ai-literacy|AI literacy]] program for 54 adults (48 in-person and 6 virtual) in a predominantly African American community in the Midwestern United States. The program covered fundamental AI concepts, societal implications, and practical applications, using hands-on activities and concrete examples over abstract technical explanations. Key findings highlight that [[equity|equity-oriented]] AI education for underserved adult populations must address foundational digital literacy gaps, build trust around data privacy, and connect AI concepts to everyday lived experiences. This challenges the dominant focus on formal [[higher-ed|higher education]] settings for AI literacy initiatives and points toward more inclusive [[lifelong-learning|lifelong learning]] models. The study contributes design considerations for [[edtech-platform|educational technology]] developers and [[faculty-development|educators]] seeking to reach adults outside traditional educational institutions. The community-centered approach offers a replicable model for [[equity|equity-focused]] AI education that prioritizes relevance, trust, and accessibility over technical depth. ## Connected Concepts - [[ai-literacy]] - [[equity]] - [[higher-ed]] - [[lifelong-learning]] - [[edtech-platform]] - [[faculty-development]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[dyslexlens-dyslexic-learners-ai]] — DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums - [[ai-literacy-equity-programming-policy]] — Programming Language Policy as an AI Literacy Equity Problem: A 15-Nation Comparative Analysis - [[teacher-education-ai-literacy-sdt-2026]] — Teacher education for artificial intelligence literacy through a self-determination theory perspective - [[llm-cultural-relevance-k12]] — LLMs for Culturally Relevant K-12 Pedagogy - [[genai-architecture-education]] — Gen-AI-tecture: using generative AI to support architectural students in design tasks ## Citation Yao Lyu, Leonymae Aumentado, Holden Winton, Jared Lee Katzman, Sparkle Berry, Zachary Rowe, Kimberly Sanders, Tawanna R. Dillahunt (2026). [Co-Designing Community-Centered AI Education for Adults: A Midwestern Case Study](https://arxiv.org/abs/2606.26565). arXiv:2606.26565. arXiv cs.HC. --- ## [Knowledge, Skills, Attitudes, Production: Competency-Based Education After Generative AI](https://edtechdev.github.io/aied/articles/competency-based-education-genai-production-2026/) > **Synthesis:** This conceptual paper proposes adding *production* — the capability to deliver professional-standard work by directing tools and other people — as a fourth attribute of competency-based education (CBE), alongside knowledge, skills, and attitudes/values. The proposal responds to a construct-validity problem: generative AI has severed the inference from a student-produced artifact to the student's own knowledge and skill, and production supplies the missing interpretation for the "open lane" of two-lane assessment architectures. ## Key Findings 1. **A construct-validity crisis.** GenAI severs the inference from a student-produced artifact to the student's own knowledge and skill, whether or not any policy was violated. Detection is unreliable, prohibition is unenforceable, and redesigning assignments to resist GenAI is an arms race — none of which answers what a GenAI-integrated artifact is evidence of. 2. **Production as a fourth attribute.** Production is defined as the capability to deliver work of professional standard by directing tools and other people, with three components: direction, evaluation and integration, and warranting. 3. **Complements the two-lane architecture.** Production supplies the missing interpretation for the "open lane" of two-lane assessment, where secure conditions certify knowledge and skills while the open lane assesses production. ## Implications For [[assessment]] and [[assessment-validity]] in the [[generative-ai]] era, the paper reframes the goal from policing artifacts to teaching and evaluating students' ability to direct AI toward professional-standard output. This connects to [[psychometrically-aware-ai]] and to [[academic-integrity]] debates, moving beyond detection toward a coherent account of what competence means when AI does much of the generating. The two-lane framing (secure conditions vs. open lane) is directly relevant to [[automated-assessment]] and to institutional [[curriculum-design]] decisions in [[higher-ed]] and professional education. As a conceptual preprint, the framework would benefit from empirical operationalization of production and its assessment criteria. ## Connected Concepts - [[assessment]] - [[assessment-validity]] - [[academic-integrity]] - [[generative-ai]] - [[higher-ed]] - [[psychometrically-aware-ai]] - [[automated-assessment]] - [[equity-in-ai-education]] ## Connected Articles - [[ai-assessment-scale-reform]] — AI assessment scale reform - [[ai-assessment-human-tutors]] — AI assessment compared with human tutors - [[agency-gap-ai-writing]] — The agency gap in AI writing - [[ai-generated-feedback-higher-ed]] — AI-generated feedback in higher ed - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge narrative drafting scaffolding ## Citation Hughes, J. W. (2026). [*Knowledge, skills, attitudes, production: Competency-based education after generative AI*](https://osf.io/preprints/edarxiv/k4rqa_v1/). EdArXiv preprint. --- ## [Computational Thinking Development in AI Agent Creation: A Mixed-Methods Study](https://edtechdev.github.io/aied/articles/computational-thinking-ai-agent-creation/) > Computational Thinking Development in AI Agent Creation: A Mixed-Methods Study **Sun et al. (2026)** — Multiple institutions. arXiv cs.CY. Computational Thinking Development in AI Agent Creation: A Mixed-Methods Study ## Summary This mixed-methods study examines how **93 pre-high school students** develop [[computational-thinking]] skills through a five-day AI agent creation workshop using CocoFlow, a no-code platform. The study integrates pre-post assessments, behavioral logs, and interviews to trace learning trajectories. **Key quantitative findings:** - **Abstract thinking:** significant improvement (d = 0.71) - **Algorithmic thinking:** significant improvement (d = 0.70) - **Iterative testing engagement** predicted self-efficacy gains (β = 0.20, p = 0.05) **The Optimal Development Zone effect** (η² = 0.55): Students with **moderate initial CT levels** showed substantially greater gains than both high-CT and low-CT peers. Qualitative analysis revealed why: - **Moderate-CT students** exhibited **adaptive expertise** — flexible, effective problem-solving - **High-CT students** risked **over-engineering** — creating unnecessarily complex solutions - **Low-CT students** struggled with **task decomposition** — breaking problems into manageable parts These findings challenge linear learning assumptions in [[k-12-ai-education]] and provide direct evidence for **differentiated scaffolding** in CT education. The Optimal Development Zone concept parallels [[ai-metacognition-stem-review]] findings on tailoring AI support to student readiness. The study also contributes to the [[ai-literacy]] evidence base by showing that no-code AI agent creation platforms can effectively develop CT in young learners, complementing work on [[agentic-education-coding]] with older students using coding assistants. ## Connected Concepts - [[computational-thinking]] - [[k-12-ai-education]] - [[ai-literacy]] ## Connected Articles - [[ai-metacognition-stem-review]] - [[agentic-education-coding]] ## Citation Sun, Y., Xin, H., Niu, Q., Li, S., Huang, L., & Chen, G. (2026). [Computational thinking development in AI agent creation: A mixed-methods study](https://arxiv.org/abs/2605.14330). arXiv:2605.14330. --- ## [Computational Thinking to Enhance Educational Robotics in Secondary School's Curriculum](https://edtechdev.github.io/aied/articles/computational-thinking-educational-robotics-secondary-2026/) > **Synthesis:** Valls i Pou (2026) examines how computational thinking can enhance the effective integration of educational robotics into secondary school curricula. Arguing that educational robotics is a strong resource for fostering problem solving, critical thinking, and STEAM subjects, the paper relates the theoretical framework of computational thinking to 21st-century skills and secondary-school curriculum knowledge linked to STEAM. Through a literature analysis, it connects computational-thinking concepts and skills with curricular content, and proposes a concrete approach for linking computational-thinking concepts from the theoretical framework into official secondary curricula to guide robotics education. ## Key Findings - **Educational robotics enhances problem solving, critical thinking, and STEAM promotion** — but requires explicit development of computational-thinking concepts during student training. - **21st-century skills are an accepted part of students' educational development** and should be integrated with computational thinking and robotics activities. - A **literature analysis** relates computational-thinking concepts and skills to the secondary-school curriculum, showing how the theoretical framework maps onto STEAM subject knowledge. - The paper proposes **linking computational-thinking theoretical concepts into the official secondary curriculum** as a guide for applying educational robotics effectively. ## Study Design & Method This is a **conceptual/literature-analysis paper** (not an empirical study). It reviews the theoretical framework of computational thinking, relates its concepts and skills to 21st-century skills and STEAM subjects in the secondary-school curriculum, and develops a proposal for embedding computational-thinking concepts into official curricula to guide educational robotics practice. It draws on educational robotics and computational-thinking literature to establish the conceptual links. ## Implications for AI in Education The paper supports [[educational-robotics|robotics]] as a vehicle for developing [[computational-thinking]] in secondary schools, arguing that robots are most effective for learning when computational-thinking concepts are made explicit and aligned to the [[stem-education|STEAM]] curriculum. It connects computational thinking to [[k-12]] teaching and to 21st-century skills such as problem solving and critical thinking. For educators, it offers a framework for mapping computational-thinking competencies onto existing curriculum knowledge so robotics activities serve genuine learning goals rather than functioning as isolated technical exercises. ## Limitations As a conceptual and literature-based paper, it presents a proposal rather than empirical evidence of learning outcomes; the proposed curriculum integration is not yet validated in practice. The focus is on the secondary-school context and STEAM subjects, so generalizability to other levels and disciplines is limited. ## Connected Concepts - [[computational-thinking]] - [[educational-robotics]] - [[stem-education]] - [[k-12]] - [[curriculum-design]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio: Computational Thinking with Robots - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education ## Citation Valls i Pou, A. (2026). *Computational thinking to enhance educational robotics in secondary school's curriculum*. Proceedings of the 5th IEEE International Conference (GRETEL, La Salle Campus Barcelona, Ramon Llull University). --- ## [Confident yet Concerned: Inconsistencies in Computing Students'' Attitudes on Cybersecurity](https://edtechdev.github.io/aied/articles/computing-students-cybersecurity-attitudes/) # Confident yet Concerned: Inconsistencies in Computing Students' Attitudes on Cybersecurity Computing students show inconsistencies between confidence in cybersecurity knowledge and actual safe practices; educational interventions are needed to close the gap. ## Synthesis Confident yet Concerned: Inconsistencies in Computing Students' Attitudes on Cybersecurity investigates computing students show inconsistencies between confidence in cybersecurity knowledge and actual safe practices; educational interventions are needed to close the gap. This work connects to existing research on student-experience by demonstrating that Abstract:Today's young adults are most immersed in technology, leading in feelings of powerlessness in managing online privacy across many platforms, and particularly susceptible to phishing attacks. This raises questions about their general, wide-ranging attitudes towards and management of cybe.... ## Connected Concepts - [[plagiarism-detection]] - [[higher-ed]] - [[prompt-engineering]] - [[affective-tutoring]] - [[administrator]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] ## Connected Articles - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) ## Citation Victor Adama, Robert Biddle, Nalin Arachchilage, Danielle Lottridge (2026). [Confident yet Concerned: Inconsistencies in Computing Students' Attitudes on Cybersecurity](https://arxiv.org/abs/2606.18541). arXiv:2606.18541. --- ## [Creating Learning Scaffolds for Engineering Design Using Concept Catalyst](https://edtechdev.github.io/aied/articles/concept-catalyst-engineering-scaffolds/) Singh, Mansi, and Riedl (2026) present Concept Catalyst, an LLM-powered tool designed to reduce K-12 teacher preparation time for Engineering Design Challenges. Unlike general-purpose chatbots, Concept Catalyst structures the interaction around three stages: (1) LLM decomposition of a design challenge into conceptual components, (2) visual manipulation and linking of related concepts by the teacher, and (3) LLM proposal of scaffolding questions for teacher review and modification. This tool addresses a practical pain point in [[stem-education]]: the time-intensive nature of creating effective [[scaffolding]] questions that guide students through the engineering design process. By positioning the teacher as editor rather than sole author, Concept Catalyst exemplifies the [[teacher-role]] shift from content creator to pedagogical curator — a pattern consistent with [[ai-tpack-teacher-multi-agent-workflow]] frameworks. The focus on teacher agency and modification is critical. Rather than generating questions for direct student consumption, the system keeps the teacher in the [[human-in-the-loop-ai]] decision process. This distinguishes Concept Catalyst from fully automated tutoring systems and aligns with research on [[ai-tutor-authoring-promptdecipher]] that emphasizes teacher co-design. The visual concept-linking interface also connects to knowledge-graph approaches in educational technology, making abstract conceptual relationships tangible for both teachers and students. For [[k-12]] engineering education specifically, this work fills a gap between theoretical [[instructional-design]] principles and practical classroom tools. ## Connected Concepts - [[stem-education]] - [[scaffolding]] - [[teacher-role]] - [[human-in-the-loop-ai]] - [[k-12]] - [[instructional-design]] ## Connected Articles - [[ai-tpack-teacher-multi-agent-workflow]] - [[ai-tutor-authoring-promptdecipher]] ## Citation Madhuri Singh, Gennie Mansi, Mark Owen Riedl (2026). [Creating Learning Scaffolds for Engineering Design Using Concept Catalyst](https://arxiv.org/abs/2605.20511). arXiv:2605.20511. arXiv:2605.20511 [cs.HC] — Accepted as Interactive Demo at ISLS 2026. --- ## [Concept Catalyst: Exploring Scrutable Interfaces to Structure K-12 Teacher Interactions with Generative AI](https://edtechdev.github.io/aied/articles/concept-catalyst-k12-teacher/) Mansi et al. (2026) introduce Concept Catalyst, a system designed around 'scrutable interfaces' — interfaces that make AI reasoning visible and editable by users. Working with K-12 teachers, the study shows that when teachers can inspect and modify how a [[generative-ai]] tool processes their inputs, they report higher trust, greater sense of control, and better alignment with their pedagogical goals. This directly addresses a critical gap in [[edtech-platform]] design: most teacher-facing AI tools operate as black boxes, undermining the [[teacher-role]] as a professional decision-maker. The Concept Catalyst approach empowers teachers to become co-designers of AI-assisted lesson planning, not just consumers. The work contributes to [[ai-literacy]] by demonstrating that 'scrutability' as a design principle can bridge the gap between AI capability and classroom reality. The study also has implications for [[k-12]] education policy, suggesting that AI tools adopted in schools should meet scrutability standards. ## Connected Concepts - [[generative-ai]] - [[edtech-platform]] - [[teacher-role]] - [[ai-literacy]] - [[k-12]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[gaide-vibe-coding-k12-teachers]] — A Guiding Framework for K-12 Teachers in Creating AI-powered Learning Technologies through Vibe Coding - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[llm-cultural-relevance-k12]] — LLMs for Culturally Relevant K-12 Pedagogy ## Citation Gennie Mansi, Sunni Newton, Roxanne Moore, Meltem Alemdar, Mark Riedl (2026). [Concept Catalyst: Exploring Scrutable Interfaces to Structure K-12 Teacher Interactions with Generative AI](https://arxiv.org/abs/2606.30590). arXiv:2606.30590. cs.HC. --- ## [Confidence-Aware Automated Assessment of Student-Drawn Scientific Models](https://edtechdev.github.io/aied/articles/confidence-aware-student-drawing-assessment/) > **Luyang Fang, Yingchuan Zhang, Jongchan Park, Zhaoji Wang, Ping Ma, Xiaoming Zhai** (2026). arXiv cs.AI preprint **Luyang Fang, Yingchuan Zhang, Jongchan Park, Zhaoji Wang, Ping Ma, Xiaoming Zhai** (2026). arXiv cs.AI preprint ## Overview Vision Transformer (ViT) with LoRA adaptation for automated scoring of student-drawn scientific models on six NGSS-aligned middle school assessment items. A confidence-aware framework derives response-level confidence from test-time predictive distributions, enabling selective automation: high-confidence responses auto-scored, uncertain cases deferred for human review. Improves scoring reliability while supporting practical trade-off between automated coverage and scoring risk. ## Key Contributions ### Paper 1: Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations - **Problem:** Students often ignore well-designed program visualizations; existing cognitive design principles don't explain learner engagement/disengagement. - **Method:** Within-subjects study (N=19 undergraduates, post-CS1/CS2) using think-aloud, interviews, and webcam gaze tracking with a multi-representational probe. - **Gaze finding:** ~47% of time on code despite visual scaffolds; students without prior experience anchored more in code and ignored metaphor views. - **Three engagement themes:** Agency (control over cognitive effort), Representational Fit (wide individual variation), Legitimacy (metaphors seen as childish at university level). - **Implication:** multi-representational-tools need attention to affective and social factors, not just cognitive design. ### Paper 2: Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - **Problem:** Static-prompt LLM tutoring systems fail to adapt across diverse academic disciplines. - **Solution:** Subject-aware prompt routing using 14 pedagogical features extracted from transcripts; contextual bandit formulation with 20 pedagogical prompts. - **Simulation:** Router achieves 0.694 vs 0.647/0.64 static baselines (p<0.001). - **Real-world A/B test:** N=656 conversations, 359 Dutch high-school students. Stochastic router achieves 28.1% exercise conversion rate vs 19.6% baseline. - **Implication:** [[adaptive-prompt-routing]] with stochastic sampling improves both efficiency and engagement in real-world tutoring. ### Paper 3: Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - **Problem:** Automated scoring of student-drawn scientific models lacks reliability indicators, leaving teachers unable to decide when to trust scores. - **Method:** Vision Transformer (ViT) with LoRA + confidence-aware framework using test-time perturbations. - **Dataset:** Six NGSS-aligned middle school assessment items (477-816 responses each, scored Beginning/Developing/Proficient). - **Key innovation:** Response-level confidence enables selective automation — high-confidence auto-scored, uncertain cases deferred for human review. - **Implication:** [[confidence-aware-ai-assessment]] enables practical triage between automation and human oversight in educational assessment. ### Paper 4: PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - **Problem:** AES systems treat scoring and feedback as separate; neural scoring lacks interpretability; LLM feedback is ability-agnostic. - **Solution:** Unified psychometric latent space (θ) via Neural GPCM Trait-Adaptive Scorer. - **Scoring performance:** QWK 0.747 (besting prior SOTA 0.722); 1st in 6/8 ASAP++ prompts and 10/11 trait dimensions. - **Feedback quality:** ZPD-based strategy mapping (Explicit Correction for θ<-1, Scaffolding for -1<θ<1, Socratic for θ>1). Multi-agent fusion from Llama-4-Scout, Qwen3-235B, GPT-4o. - **Evaluation:** >90% win rate in Actionability vs GPT-4o, Llama-4-Scout, Qwen3. - **Implication:** psychometrically-aware-ai can unify assessment and instruction, transforming automated essay scoring from summative to formative. ## Connected Concepts - [[adaptive-prompt-routing]] - [[confidence-aware-ai-assessment]] ## Connected Articles - [[psyscore-essay-scoring-zpd-feedback]] — PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[learning-to-prompt-adaptive-tutoring]] — Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[cong-confidence-asag-2026]] — Confidence-Aware Automatic Short Answer Grading - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading ## Citation Luyang Fang, Yingchuan Zhang, Jongchan Park, Zhaoji Wang, Ping Ma, Xiaoming Zhai (2026). [Confidence-Aware Automated Assessment of Student-Drawn Scientific Models](https://arxiv.org/abs/2606.20264). arXiv:2606.20264. arXiv cs.AI preprint. --- ## [Confidence-Aware Automatic Short Answer Grading](https://edtechdev.github.io/aied/articles/cong-confidence-asag-2026/) > **Confidence-Aware ASAG** — A hybrid confidence estimation framework for Automatic Short Answer Grading with LLMs that fuses model-based confidence signals (verbalized, latent, consistency-based) with dataset-derived aleatoric uncertainty via Random Forest + Platt scaling, enabling reliable selective prediction and principled human-in-the-loop review workflows. ## Key Findings 1. **Model-based confidence alone is insufficient.** Three LLM confidence signals were systematically compared — **verbalizing** (prompting the model to self-report confidence), **latent** (softmax-normalized conditional log-likelihood of the predicted label token), and **consistency-based** (agreement fraction across 5 predictions at temperatures [0.2–1.0]) — and none individually provided reliable uncertainty estimates for ASAG. 2. **Aleatoric uncertainty must be explicitly modeled.** The key innovation is modeling dataset-derived (aleatoric) uncertainty via semantic heterogeneity: student responses are embedded with all-MiniLM-L6-v2, clustered with agglomerative hierarchical clustering (Ward linkage), and within-cluster entropy quantifies inherent ambiguity. This captures the fundamental reality that some student answers are genuinely ambiguous, independent of model quality. 3. **Hybrid fusion outperforms single-source approaches.** Combining model-based signals with aleatoric uncertainty via a Random Forest classifier (500 trees) followed by Platt scaling produced the best calibration, evaluated on both selective prediction (AUROC / AUARC) and reliability (ECE / MCE / Brier score). The hybrid framework acknowledges that both epistemic and aleatoric uncertainty contribute to grading unreliability. 4. **Principled evaluation enables safe deployment.** By evaluating with selective prediction metrics rather than raw accuracy, the framework enables a practical workflow: high-confidence predictions can be auto-graded, while low-confidence cases are routed to human review. This is a more realistic deployment model than assuming perfect automation. ## Implications This work addresses a critical gap in [[automatic-short-answer-grading]]: the gap between knowing that LLMs can grade short answers and trusting them enough to deploy in high-stakes settings. The central insight — that model confidence signals are unreliable on their own — is consistent with broader findings in [[confidence-aware-ai-assessment]] and psychometrically-aware-ai, which argue that AI assessment systems must be calibrated against human judgment uncertainty, not just accuracy. The explicit modeling of aleatoric uncertainty is particularly important for [[automated-grading]]. In ASAG, some student responses are genuinely ambiguous even to expert human raters — this is not a model failure but a property of the task. By modeling this inherent ambiguity through semantic clustering, the framework prevents the system from overconfidently misclassifying responses that are fundamentally uncertain. For [[human-in-the-loop-ai]] workflows in educational assessment, the selective prediction approach provides a principled mechanism for triage: rather than requiring educators to review everything (defeating the purpose of automation) or trusting the model blindly (risking harm), the confidence framework identifies exactly which responses need human attention. This has practical implications for [[short-answer-scoring-quality-degradation]] research and the design of [[hybrid-e-assessment-semi-automated-grading]] systems. The use of [[rag]] and synthetic-data techniques for calibration could extend this framework beyond the SciEntsBank domain, enabling confidence-aware grading across diverse subject areas and languages. ## Connected Concepts - [[ai-ed-evaluation]] - [[automated-grading]] - [[confidence-aware-ai-assessment]] - [[hybrid-e-assessment-semi-automated-grading]] - psychometrically-aware-ai - [[rag]] ## Connected Articles - [[automatic-short-answer-grading]] — Automatic Short Answer Grading with LLMs - [[short-answer-scoring-quality-degradation]] — Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specifi... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-scoring-language-bias-physics]] — AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations... - [[aiawe-automated-writing-evaluation]] — AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models - [[aicode-collaborative-feedback-system]] — AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education - [[aissa-slides-analysis]] — AISSA: AI-based Student Slides Analysis Tool for Academic Presentations - [[authentic-assessment]] — Authentic Assessment - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[behaviorally-adaptive-visual-diversion-assessment-2026]] — Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[calibrating-trustworthiness-llm-education-2026]] — Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education - [[code-gen]] — CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation - [[confidence-aware-student-drawing-assessment]] — Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[correct-answer-trap-misconceptions]] — The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs ## Citation Cong, L., Hahn, S., Gombert, S., Camus, L., Drachsler, H., & Kroehne, U. (2026). [*Confidence Estimation in Automatic Short Answer Grading with LLMs*](https://arxiv.org/abs/2605.00200). arXiv:2605.00200. --- ## [ConnectED: A Curriculum-Aligned AI System for Vietnamese Instructional Lesson Planning and Student Learning](https://edtechdev.github.io/aied/articles/connected-ai-lesson-planning-vietnam/) > **Thang Doan Viet, Anh Nguyen Hoang, Tinh Luong Son, Anh Hoang Thi Ngoc, Huyen Giang Thi Thu, Tai Le Quy** — arXiv preprint (2026). ## Synthesis ConnectED is a human-centered AI system supporting the full instructional lifecycle in Vietnamese education: curriculum-aligned lesson design, interactive student learning, and feedback-driven refinement, built on VietEduQwen, a Vietnamese educational LLM trained with SFT and DPO. The system operationalizes the ADDIE framework through structured prompt templates aligned with Vietnam's Official Dispatch No. 5512/BGDDT-GDTrH, where each phase is both a generation step and a teacher validation gate — keeping teachers in the loop. On 3,119 questions from the 2025 Vietnamese National High School Examination, VietEduQwen achieved 87.02% accuracy, outperforming Qwen3-8B by 6.10 percentage points; teacher (n=18) and student (n=214) surveys showed strong satisfaction with curriculum alignment and lesson clarity. The Evaluation phase closes the loop by connecting student performance data to iterative lesson improvement, exemplifying curriculum-grounded, multilingual, and teacher-governed LLM deployment in a Global South context. ## Connected Concepts - [[k-12-ai-education]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[culturally-relevant-pedagogy]] - [[ai-education]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[affective-tutoring]] ## Connected Articles - [[human-llm-collaborative-coding-k12-educator-ai]] — Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use - [[llm-cultural-relevance-k12]] — LLMs for Culturally Relevant K-12 Pedagogy - [[concept-catalyst-engineering-scaffolds]] — Creating Learning Scaffolds for Engineering Design Using Concept Catalyst - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth - [[cyberscholar-genai-writing-feedback]] — Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar ## Citation Doan Viet, T., Nguyen Hoang, A., Luong Son, T., Hoang Thi Ngoc, A., Giang Thi Thu, H., & Le Quy, T. (2026). [ConnectED: A curriculum-aligned AI system for Vietnamese instructional lesson planning and student learning](https://arxiv.org/abs/2607.28647). arXiv:2607.28647. --- ## [Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming](https://edtechdev.github.io/aied/articles/constructing-epistemic-ai-literacy-student-ai-co-programming/) **Mengqian Wu (2026)** Epistemic thinking — understanding how knowledge is constructed and justified — plays a central role in [[ai-literacy]], particularly when students co-program with generative AI. This paper introduces a framework for detecting epistemic aims and processes in [[student-experience]] during programming activities. The analysis reveals that students engage in question construction, AI output evaluation, and solution integration as distinct epistemic processes. These findings inform [[scaffolding]] design for programming education and connect to broader discussions of [[agentic-education-coding]] where students maintain agency while leveraging AI assistance. ## Key Findings - The paper introduces the conceptual framework of Epistemic AI Literacy (EAIL), reframing AI literacy as a process-oriented epistemic phenomenon that emerges through dynamic human-AI interactions, drawing on the AIR framework of epistemic aims, ideals, and reliable epistemic processes. - Using a large dialogue dataset of human-AI co-programming, the study identifies observable dimensions of epistemic aims (mastery-oriented aims) and epistemic processes (outsourcing, explanation seeking, verification seeking, prompt monitoring, and epistemic justification). - A subset of interactions was manually annotated to ground the constructs, which then informed scalable automatic labeling using complementary approaches — few-shot prompting and regex-based scripts — applied interactively. - Results reveal a prevalent lack of EAIL: 78.8% of student-GenAI interactions relied on non-mastery-oriented aims and less reliable epistemic strategies such as outsourcing and verification-seeking. - Only 11.1% of interactions showed high epistemic engagement, where mastery-oriented aims were coupled with advanced strategies like epistemic justification in a more reliable epistemic process. - While GenAI facilitates task success, robust epistemic performance and genuine learning rarely emerge without deliberate instructional and design support. ## Study Design & Method The study operationalizes epistemic constructs that are normally hard to observe. Epistemic aims and processes were detected in student-AI co-programming interaction data, with manual annotation of a subset grounding the constructs. Complementary automated approaches — few-shot prompting with large language models and regex-based scripts — were then used interactively to label the full dataset at scale, providing a path from small-scale qualitative insight to large-scale measurement. The design responds to a limitation identified in a 2022 UNESCO report: AI education has typically taken a technology-oriented approach, ignoring the human and in-depth ethical questions of how AI is actually used in practice. ## Implications for AI in Education The finding that most student-GenAI interactions exhibit weak epistemic engagement — outsourcing and verification-seeking rather than mastery-oriented aims with justification — suggests that mere access to AI tools does not produce learning-oriented use. For [[scaffolding]] design in programming education, the work points to interventions that prompt students to construct questions, evaluate AI outputs, and justify their integration decisions, supporting the development of [[metacognition]] and [[self-regulated-learning]] alongside technical skill. The EAIL framework also connects AI literacy to epistemic practice in [[cs-education]]: curricula should cultivate the processes by which learners decide what to trust and why, rather than only measuring whether tasks are completed. ## Connected Concepts - [[ai-literacy]] - [[student-experience]] - [[scaffolding]] ## Connected Articles - [[agentic-education-coding]] ## Citation Mengqian Wu (2026). [Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming](https://arxiv.org/abs/2607.00211). arXiv:2607.00211. --- ## [The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human-AI Collaboration](https://edtechdev.github.io/aied/articles/contextual-sycophancy-ai-literacy/) ## Key Finding LLM sycophancy creates a feedback loop where user errors propagate into AI advice, degrading outcomes; AI literacy training reduces but doesn't eliminate this contextual sycophantic dependence. ## Synthesis This AIED 2026 LBR paper identifies a pernicious feedback loop in educational human-AI collaboration: contextual sycophantic dependence. In a controlled experiment with 60 participants, LLMs mirrored user reasoning errors rather than correcting them, and these errors propagated into subsequent AI advice and final task performance. This finding is particularly concerning for educational contexts where students with developing knowledge interact with AI — the very population most likely to benefit from AI tutoring. AI literacy and prompting training reduced direct mirroring but did not eliminate error propagation, suggesting that system-level safeguards are needed. This connects directly to [[ai-tutor-safety-harms]], which catalogued pedagogical safety failures in tutoring systems, and extends the [[llm-fallacy-misattribution]] concern that students may attribute incorrect AI reasoning to themselves. The sycophancy problem also relates to findings from [[llm-tutoring-feedback-diagnosis-gap]], where LLMs over-validated incorrect solutions — the same underlying tendency manifested differently. The authors call for epistemically independent AI support, a design principle with implications for [[pedagogical-llm-training]] and [[educational-llm-alignment]]. ## Connected Concepts - [[pedagogical-llm-training]] ## Connected Articles - [[ai-tutor-safety-harms]] - [[llm-fallacy-misattribution]] - [[llm-tutoring-feedback-diagnosis-gap]] - [[educational-llm-alignment]] ## Citation D, A.K.C.G.S.O. & Results), V.A.L. (2026). [*The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human-AI Collaboration*](https://arxiv.org/abs/2605.18372) --- ## [Conversational AI as a catalyst for informal learning: An empirical large-scale study on LLM use in everyday learning](https://edtechdev.github.io/aied/articles/conversational-ai-informal-learning/) > **Synthesis:** Conversational AI as a catalyst for informal learning: An empirical large-scale study on LLM use in everyday learning ## Key Findings - In a large-scale survey of 776 German participants conducted in February 2025 — the third year following the breakthrough launch of ChatGPT — 87% of respondents reported already incorporating LLMs into their everyday learning routines for a wide variety of learning tasks (11% used LLMs but not for learning, and 2% did not use them at all). - Young adults among German-based, digitally engaged users are at the forefront of adopting LLMs, primarily to enhance their learning experiences independently of time and space: LLM learners averaged 31 years versus 35 for non-learners (F = 6.68, p < .01), men were more likely to adopt (53% of LLM learners were male, while 64% of avoiders were female; χ² = 15.5, p < .001), and ChatGPT dominated, with 93% (n = 632) of LLM learners reporting its use. - Four types of learners emerge across learning contexts, distinguished by the tasks they perform with LLMs and the devices they use to access them: Structured Knowledge Builders (22.9%), Self-Guided Explorers (26.5%), Analytical Problem Solvers, and Adaptive Power Users, identified via latent class analysis with balanced class shares (23.8–25.7%). - Respondents exhibit paradoxical trust behaviors: they rely on LLMs while simultaneously distrusting their accuracy and privacy protection measures — 88% (n = 601) perceived improved learning or productivity, yet misinformation was the most commonly cited challenge (68%, n = 460) and the majority (n = 543) reported taking no privacy measures. - The study's implications emphasize the importance of including different media types for learning, enabling collaborative learning, providing sources, and meeting the needs of different types of learners — "learning by design." ## Study Design & Method The study addresses who is embracing LLMs for self-directed learning, who remains hesitant, their reasons for adoption or avoidance, and the learning patterns that emerge with this novel technological landscape. Data come from an online survey administered via Prolific in February 2025 to 776 German participants (mean age 31.6, SD = 9.87, max 72), including non-adopters; 678 respondents were actively using LLMs for learning. Analysis combined descriptive and inferential statistics (ANOVA, chi-square), a latent class analysis from which the four-learner typology was derived, multinomial logistic regression predicting class membership from privacy perceptions, effectiveness, over-reliance, and demographics, and qualitative analysis of open-text responses. Adoption drivers included curiosity (n = 517), recommendations from social connections (n = 254), and media coverage (n = 217), while avoidance was led by mistrust of factual output and a preference for traditional methods (52% of non-LLM learners). ## Implications for AI in Education The near-universal reported use of LLMs in informal, everyday learning indicates that much AI-mediated learning now happens outside formal curricula, with implications for [[lifelong-learning]] and for how formal institutions relate to self-directed learners. The paradoxical trust findings — reliance alongside distrust of accuracy and privacy — suggest that learners need support in calibrating their use, pointing to the role of source provision and transparency features in tool design. The four learner types argue against one-size-fits-all interfaces and content, favoring designs that accommodate varied task-device combinations; the call for multi-modal and collaborative features connects to [[collaborative-learning]] and to broader questions of [[motivation]] and engagement in self-regulated, informal contexts. Use is embedded and flexible — the median learner spent about 20 minutes per weekday on LLM-based learning, nearly 70% reported no particular day of the week, and 63.5% had used LLMs for more than six months — and 58% (n = 395) said they were extremely likely to continue using LLMs for learning, with 72% willing to recommend them to friends, reinforcing that [[self-regulated-learning]] with LLMs is now a durable everyday practice rather than a novelty. ## Limitations The authors note that the sample, recruited exclusively from Germany via Prolific, may overrepresent tech-savvy, English-fluent individuals (mean AI literacy score = 3.06), and that cultural attitudes toward AI vary, so findings may not generalize to non-Western or less tech-savvy populations. The non-user subsample was small, so results for that group should be treated as exploratory and descriptive. The study measured self-reported LLM use and perceived purposes without measuring concrete learning outcomes, so no claims can be made about whether more frequent or highly satisfactory LLM use translates into deeper conceptual learning. ## Connected Concepts - [[lifelong-learning]] - [[self-regulated-learning]] - [[motivation]] - [[collaborative-learning]] - [[prompt-engineering]] - [[affective-tutoring]] - [[plagiarism-detection]] - [[administrator]] ## Connected Articles - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[socraticode-k12-programming-tutor]] — Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participatory Design Study ## Citation Terzimehić, N., Bühler, B., & Kasneci, E. (2026). [*Conversational AI as a catalyst for informal learning: An empirical large-scale study on LLM use in everyday learning*](https://doi.org/10.1016/j.caeai.2026.100634). --- ## [The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents](https://edtechdev.github.io/aied/articles/conversational-ai-tutors-framework/) > **Authors:** Kirk Vanacore, Ryan S. Baker, Avery H. Closser, Jeremy Roschelle **Year:** 2026 **Venue:** arXiv (cs.HC) > Synthesizes intelligent tutoring systems research and generative AI into a keep/change/center/study framework for conversational tutoring systems, arguing proven ITS technologies should anchor generative tutors while centering student meaning-making and agency. ## Key Findings - The paper synthesizes intelligent tutoring systems (ITS) research and generative AI, proposing a keep, change, center, study framework for designing conversational tutoring systems. - Keep: proven ITS technologies — knowledge tracing, affect detection, and related student modeling — remain valuable for diagnosing students' cognitive and emotional states. - Change: tutoring delivery is transformed by generative AI's capacity for dynamic content generation and dialogic scaffolding, replacing rule-based, limited-scope feedback with flexible open-ended dialogue. - Center: the student's meaning-making and agency, engaging students' thoughts, questions, and misconceptions the way high-quality human tutors do. - Study: the field must test efficacy, student experience, and integration, since conversational tutors can now be built quickly and easily but their effectiveness is not yet established. - Legacy ITS technologies such as Knowledge Tracing, Knowledge Spaces, and Epistemic Emotion Detection can diagnose states like slips, lack of mastery, or misconceptions; large language models integrated with structured knowledge representations may identify not just whether an answer is correct but why a student responded that way. ## From Problem Sets to Dialogue Whereas most current ITS function, in essence, as interactive and adaptive problem sets with feedback and hints, conversational tutors hold the potential to simulate high-quality human tutoring by engaging with students' thoughts, questions, and misconceptions through natural language dialogue. Earlier dialogue systems such as AutoTutor and Watson Tutor demonstrated the possibility but were limited by rule-based response generation. Generative AI changes the capacity profile: interactive feedback can encourage constructive behaviors such as self-repair and knowledge construction, and tutorial dialogue can follow up on errors with diagnosis questions, supporting seamless formative assessment. ## Implications for AI in Education The keep/change/center/study framework gives researchers and developers a discipline for building conversational tutors that are both scalable and pedagogically grounded. It warns against discarding decades of ITS research in the rush to generative models: [[knowledge-tracing]] and affect detection remain the diagnostic backbone, while [[generative-ai]] supplies flexible delivery. For educators, the framework's emphasis on centering meaning-making and student agency aligns with [[intelligent-tutoring]] goals and with [[scaffolding]] that keeps the learner active; its insistence on studying efficacy and integration reflects the reality that easy deployment of conversational agents has outpaced evidence about what works. ## Connected Concepts - [[knowledge-tracing]] - [[intelligent-tutoring]] - [[lifelong-learning]] - [[personalized-learning]] - [[affective-tutoring]] - [[scaffolding]] - [[pedagogical-agent]] - [[pedagogical-llm-training]] ## Connected Articles - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[codify-socratic-tutoring-programming]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-stem-bibliometric-trends]] — Why does AI unlock new possibilities in STEM education? A Bibliometric Analysis of Trends and Future Agenda - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education ## Citation Kirk Vanacore et al. (2026). [The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents](https://arxiv.org/abs/2602.19303). arXiv:2602.19303. cs.HC. --- ## [Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning](https://edtechdev.github.io/aied/articles/correct-answer-trap-ai-tutor/) **Catching the Correct Answer Trap** — accepted at AIED 2026 — exposes a critical blind spot in [[intelligent-tutoring]] systems: they systematically fail to detect misconceptions when students arrive at correct answers through flawed reasoning. Using real student data from the Eedi mathematics platform, the authors characterize the 'Correct Answer Trap' (CAT), showing that 71% of failures concentrate in just two question types where erroneous reasoning accidentally produces the correct numerical answer. Even a frontier [[llm]] achieves only 84% detection accuracy while generating roughly 4 false alarms per genuine detection — making standalone automated screening impractical. This finding has profound implications for [[automated-grading]] and [[formative-assessment]] systems: high overall accuracy metrics can mask catastrophic failures in reasoning assessment. The work connects to [[llm-student-misconception-identification]] research on the gap between answer checking and reasoning evaluation, and to [[codify-socratic-programming-tutor]] findings that even Socratic AI tutors can miss deep misconceptions. The paper reinforces calls for [[human-in-the-loop-ai]] approaches in [[intelligent-tutoring]] and suggests that [[scaffolding]] designs should explicitly account for reasoning assessment, not just answer verification. The concentration of failures in predictable question types also suggests targeted improvements are possible. ## Connected Concepts - [[intelligent-tutoring]] - [[llm]] - [[automated-grading]] - [[formative-assessment]] - [[human-in-the-loop-ai]] - [[scaffolding]] ## Connected Articles - [[llm-student-misconception-identification]] - [[codify-socratic-programming-tutor]] ## Citation Moiz Imran, Sahan Bulathwela (2026). [Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning](https://arxiv.org/abs/2605.23925). arXiv:2605.23925. AIED 2026. --- ## [The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions](https://edtechdev.github.io/aied/articles/correct-answer-trap-misconceptions/) Imran and Bulathwela (2026) identify the 'correct answer trap' — automated feedback systems that judge only answer correctness reinforce rather than address misconceptions when students reach the right answer through flawed reasoning. Using 20,964 real student responses from the Eedi mathematics platform, they find fine-tuned classifiers detect only 57% of hidden misconceptions (standard ML interventions did not improve this), while open-weight reasoning models reach 84% but at a false-alarm-to-true-detection ratio of roughly 8:1 at realistic prevalence. They propose a graduated assessment rubric separating answer correctness from method validity, and a detect-verify-escalate pipeline that routes uncertain cases to diagnostic follow-up questions. Two deployment modes are offered: a teacher dashboard for review queue filtering, and an autonomous tutor where flags trigger low-cost [[formative-assessment]] follow-up. This work directly addresses [[student-modeling]] gaps in [[intelligent-tutoring]] systems and has implications for [[automated-grading]] in [[k-12]] math education. ## Connected Concepts - [[formative-assessment]] - [[student-modeling]] - [[intelligent-tutoring]] - [[automated-grading]] - [[k-12]] ## Connected Articles - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[teacher-authored-prompts-student-ai-dialogue]] — Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning ## Citation Moiz Imran, Sahan Bulathwela (2026). [The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions](https://arxiv.org/abs/2606.23205). arXiv:2606.23205. AIED PEAF 2026 Workshop --- ## [Cost-of-Ethics Crisis: Beliefs, Decisions, and Justifications in the Job Searches of Computer Science Students in Canada and the United States](https://edtechdev.github.io/aied/articles/cost-of-ethics-crisis-cs-ethics-education/) # Cost-of-Ethics Crisis in CS Education ## Key Findings This study examines the disconnect between **ethics education** and real-world decision-making among 129 computer science students and recent graduates during their job searches. Despite receiving contemporary CS ethics education, most students prioritize compensation, location, and workplace culture over ethical and social concerns when choosing employers. Even when expressing ethical concerns, respondents justified taking contradictory actions through shared explanations: desire to make money or perceived inability to avoid unethical workplaces. This highlights a critical gap in how CS ethics education translates to actual behavioral change. The findings have significant implications for **AI safety in education**: teaching ethical reasoning about autonomous systems without connecting to students' material circumstances may be insufficient. This connects to broader debates about [[pedagogical-safety-rl]], [[teacher-ai-competency]], and [[equity-in-ai-education]] in shaping student values. ## Connections to AIED The ethics education gap identified here suggests that AI literacy curricula face a similar risk: students may understand AI ethics principles theoretically while still choosing to use AI tools unethically in practice. This mirrors findings on [[academic-integrity]] and [[over-reliance]] patterns in student AI use. ## Connected Concepts - [[teacher-ai-competency]] - [[equity-in-ai-education]] - [[academic-integrity]] - [[over-reliance]] ## Connected Articles - [[pedagogical-safety-rl]] ## Citation Abdalla, M., Abdalla, S., Cappello, A., Dowling, K., Metaxa, D., Widder, D. G., & Stinson, C. (2026). [Cost-of-ethics crisis: Beliefs, decisions, and justifications in the job searches of computer science students in Canada and the United States](https://arxiv.org/abs/2605.09680). arXiv:2605.09680. --- ## [CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback](https://edtechdev.github.io/aied/articles/cotal-formative-assessment-scoring-2026/) > 1. **Evidence-Centered Design (ECD)** — assessments and rubrics aligned to curriculum goals from the start 2. **Human-in-the-loop prompt engineering** — labelled examples and prompts refined iteratively with educators 3. **Chain-of-thought (CoT) prompting + active learning** — teacher and student feedback loops refine questions, rubrics, and LLM prompts across iterations # CoTAL: Formative Assessment Scoring with Human-in-the-Loop Prompt Engineering > Cohn, Ashwin T S, Mohammed & Biswas (2026) introduce **CoTAL** (Chain-of-Thought Prompting + Active Learning): an LLM grading pipeline that couples Evidence-Centered Design with human-in-the-loop prompt engineering and iterative teacher/student feedback refinement. It improves GPT-4's scoring by **up to 38.9% over a non-prompt-engineered baseline** and generalises across science, computing, and engineering — direct evidence that prompt-engineering quality, not model choice, is often the binding constraint in [[automated-grading]]. ## How it works 1. **Evidence-Centered Design (ECD)** — assessments and rubrics aligned to curriculum goals from the start 2. **Human-in-the-loop prompt engineering** — labelled examples and prompts refined iteratively with educators 3. **Chain-of-thought (CoT) prompting + active learning** — teacher and student feedback loops refine questions, rubrics, and LLM prompts across iterations ## Findings - **Up to +38.9% scoring performance** over a non-prompt-engineered baseline (no labelled examples, no CoT, no iterative refinement) - Gains demonstrated **across domains**: science, computing, engineering (the generalisation question most grading papers ignore) - **Teachers and students rate CoTAL effective** at scoring and explaining responses - Their feedback yields insights that improve grading accuracy and explanation quality ## Connected Concepts - [[ai-ed-evaluation]] - [[assessment-validity]] - [[automated-grading]] - [[formative-assessment]] - [[prompt-engineering]] - [[llm]] ## Connected Articles - [[ground-truth-reliability-aied]] — Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[ai-in-the-wild-college]] — AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base ## Citation Cohn, C., Ashwin T S, Mohammed, N., & Biswas, G. (2026). [CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback](https://arxiv.org/abs/2504.02323). arXiv:2504.02323. Under review, Computers and Education: Artificial Intelligence. --- ## [CourseBlueprint: A Structured Pipeline for Adaptive Pedagogical Video Generation Grounded in Course Corpora](https://edtechdev.github.io/aied/articles/courseblueprint-adaptive-video-generation/) Islam et al. (2026) address a core limitation of generative text-to-video for education: while visually fluent, such systems lack pedagogical content knowledge (PCK). CourseBlueprint provides a structured pipeline producing adaptive pedagogical videos grounded in a course corpus (undergraduate biomedical-imaging course BMED 2300, 23 lectures, 1,116 slides). The pipeline includes four components with typed intermediate representations and validation: a scaffolding module with prerequisite concept graphs, an adaptive controller assigning style specifications per learner persona, an engagement generator using a fixed rhetorical contract (hook→retrieval→core→analogy→forward contract), and a deterministic slide-image override mechanism. Ablation results show removing the engagement contract crashes the engagement score from 5.00 to 1.20. The slide override converts a 0/9 corpus-grounding failure into 9/10 successful matches. This work demonstrates that [[generative-ai]] for education needs explicit pedagogical structure — not just fluency — and connects to [[scaffolding]], [[personalized-learning]], and [[generative-ai]] research in [[higher-ed]]. ## Connected Concepts - [[generative-ai]] - [[scaffolding]] - [[personalized-learning]] - [[higher-ed]] ## Connected Articles - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[generativism-learning-theory]] — Generativism: Toward a Learning Theory for the Age of Generative Artificial Intelligence - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education ## Citation Md Zabirul Islam, Md Motaleb Hossen Manik, Ge Wang (2026). [CourseBlueprint: A Structured Pipeline for Adaptive Pedagogical Video Generation Grounded in Course Corpora](https://arxiv.org/abs/2606.20608). arXiv:2606.20608. arXiv:2606.20608 (cs.CY; cs.AI; cs.CV) --- ## [CourseGraph: Finding overlaps and differences in Computer Science courses across universities](https://edtechdev.github.io/aied/articles/coursegraph-cs-course-comparison-2026/) > **Synthesis:** This paper presents CourseGraph, a methodology for automatically evaluating external course equivalences by modelling course content as structured knowledge graphs. Designed for student mobility programmes like Erasmus+, CourseGraph extracts topics from course descriptions, maps relationships between concepts, and identifies substantive overlap vs. complementarity between courses at different universities. The system mimics the manual workflow of curriculum advisors — analysing syllabi, comparing learning objectives, and flagging redundancies — but scales to thousands of course pairs. Evaluation on CS programmes across multiple European universities demonstrates that CourseGraph's overlap judgments align closely with expert advisor decisions while dramatically reducing processing time. Student mobility programs such as Erasmus+ enable students to take courses at other universities, broadening their academic and cultural horizons. However, this flexibility also leads to a practical challenge: ensuring that students do not take courses elsewhere that substantially overlap with courses in their home curriculum. In this work, we propose CourseGraph, a methodology that automates the evaluation of external courses based on insights obtained from the process followed by curriculum administrators when assessing courses for inclusion in a degree program. Course- Graph extracts information such as course titles, descriptions, and learning outcomes from the course webpage. Then, this information is represented semantically using a BERT-based language model, after which the pair-wis presents CourseGraph, a methodology for automatically evaluating external course equivalences by modelling course content as structured knowledge graphs. Designed for student mobility programmes like Erasmus+, CourseGraph extracts topics from course descriptions, maps relationships between concepts, and identifies substantive overlap vs. complementarity between courses at different universities. The system mimics the manual workflow of curriculum advisors — analysing syllabi, comparing learning objectives, and flagging redundancies — but scales to thousands of course pairs. Evaluation on CS programmes across multiple European universities demonstrates that CourseGraph's overlap judgments align closely with expert advisor decisions while dramatically reducing processing time. ## Connected Concepts - [[administrator]] - [[engagement-metrics]] - [[k-12-ai-education]] - [[culturally-relevant-pedagogy]] - [[math-education]] - [[higher-ed]] - [[prompt-engineering]] - [[affective-tutoring]] ## Connected Articles - [[wordstream-glass-learning-analytics]] — Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics - [[reshaping-cs-education-genai]] — Reshaping Undergraduate Computer Science Education in the Generative AI Era - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[universities-ai-era-rethinking]] — The University AI Didn''t Replace: Rethinking Universities in the AI Era - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[cross-cultural-student-perceptions-genai-computing]] — Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education ## Citation Nijdam, Arthur, Wagner, Paul Stankovski, & Ramezanian, Sara (2026). [CourseGraph: Finding overlaps and differences in Computer Science courses across universities](https://arxiv.org/abs/2608.05910). arXiv:2608.05910. --- ## [What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education](https://edtechdev.github.io/aied/articles/credential-cognitive-stewardship-ai-assessment/) Generative AI undermines a basic premise of educational assessment: that submitted work reliably evidences the human capacities a credential certifies. This paper proposes *cognitive stewardship*, a framework linking four elements \u2014 the learning claim, the delegation boundary, the evidence standard, and safeguards \u2014 to reason about what remains inferable about learning once cognitive work is delegated to AI. It then audits verified public GenAI assessment guidance from 30 universities using a pre-specified scoring codebook applied by four open-weight LLMs as structured coders, with scores averaged to dampen single-model bias.\n\nThe audit's headline finding is a governance gap already visible in [[genai-assessment-governance]] and [[genai-policies-higher-ed-computing]]: institutional policies are getting better at *classifying* AI use but not at explaining what evidence of learning remains valid under each class. The framework sharpens the wiki's [[assessment-validity]] thread \u2014 shifting the question from detection and [[academic-integrity]] enforcement toward specifying delegation boundaries per learning claim \u2014 and complements [[ai-assessment-scale-reform]] and [[universities-ai-era-rethinking]] on redesigning credentials for AI-mediated education. ## Connected Concepts - [[assessment-validity]] - [[academic-integrity]] ## Connected Articles - [[genai-assessment-governance]] - [[genai-policies-higher-ed-computing]] - [[ai-assessment-scale-reform]] - [[universities-ai-era-rethinking]] ## Citation Yao, K. (2026). [What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education](https://arxiv.org/abs/2607.19988). arXiv:2607.19988. arXiv preprint (cs.CY). --- ## [AI-accelerated End-to-End Framework for Rapid Professional Upskilling](https://edtechdev.github.io/aied/articles/crewscaler-ai-upskilling-framework/) > **Synthesis:** The Crew Scaler framework applies AI acceleration across all five stages of professional upskilling—knowledge acquisition, content development, content review and verification, AI-tutor coaching, and assessment development—with external validation from NASBA CPE accreditation, NVIDIA certification exam passes (3/3, 14 in progress), and a 1,267-item risk dataset production. Dual-efficiency design pairs AI-accelerated production with learning-efficient outputs (prerequisite ordering, spaced review, misconception-keyed distractors, 16 tutoring protocols). ## The Five-Stage Pipeline The Crew Scaler framework organizes rapid upskilling as an end-to-end AI-accelerated pipeline: | Stage | AI Acceleration | Learning Efficiency | Quality Control | |---|---|---|---| | Knowledge Acquisition | LLM-assisted domain exploration & extraction | Prerequisite-ordered 4-level hierarchy | Blueprint coverage checks | | Content Development | AI-drafted chapters; condensation passes | One-new-element pacing; 70/20/10 review | Fixed templates; six-pass revision | | Content Review & Verification | Automated hallucination & faithfulness checks | Defects caught before learners study | SME audit; immutable audit trail | | AI-Tutor Coaching | Scalable one-to-one protocolized tutoring | Intent- and affect-adaptive protocols | Integrity guardrails; grounded RAG | | Assessment Development | AI-generated items & distractors | Misconception-targeted diagnostic distractors | Blueprint tagging; difficulty distribution | Humans retain high-judgment roles (blueprint design, SME review, misconception authoring, item rating) while AI absorbs volume work, keeping human expertise in the *multiplier* regime. ## Key Design Features - **Knowledge hierarchy**: content organized into 4 levels—foundational, building blocks, integrated concepts, advanced—with strict dependency chains - **16 tutoring protocols**: including Socratic questioning, worked examples, hint escalation, spaced retrieval, productive failure, and affective support (prioritizing boredom over frustration) - **Misconception-keyed distractors**: every assessment item traces to an atomic knowledge item with documented misconceptions; distractors engineered from misconceptions, not invented ad-hoc - **Hallucination verification**: four-type taxonomy (factual, reasoning, contextual, true fabrications) with RAGAS-adapted accuracy standards - **530-question assessment bank** tagged to a 10-domain, 53-skill blueprint ## Validation Signals Three independent, externally checkable signals: 1. **Certification outcomes**: 3/3 learners passed the NVIDIA Certified Professional in Agentic AI (NCP-AAI) exam using only the framework's knowledge base (14 more in progress) 2. **Capability outcomes**: the ~3,000-page knowledge base supported production of a 1,267-item risk dataset (81 categories, 14 domains) for multi-agent AI systems, presented to ~500 US federal employees 3. **Accreditation**: NASBA (National Association of State Boards of Accountancy) reviewed and approved the program for CPE credits ## Gap Analysis The paper identifies four gaps in existing frameworks: - **Fragmentation**: no framework covers end-to-end from knowledge acquisition through industry assessment - **Missing verification**: hallucination detection (~60% rate on post-cutoff questions) is absent from education pipelines - **Shallow pedagogy**: default LLM tutoring achieves only 52–70% correct actions - **Self-referential validation**: most frameworks generate their own success measures rather than facing external standards ## Connected Concepts - [[open-source]] - [[lifelong-learning]] - [[prompt-engineering]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[affective-tutoring]] - [[automated-essay-scoring]] ## Connected Articles - [[skill-diversity-worker-resilience]] — Navigating the skill diversity frontier: How skill complexity explains worker resilience - [[generative-ai-education-productivity-gaps]] — Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment - [[supplynet-visual-exploratory-learning]] — SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation - [[astra-atco-training-simulator]] — ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[multimodal-affective-its-presentation]] — An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training ## Citation Nguyen, T., Nguyen, H., & Ogburn, R. (2026). [*AI-accelerated End-to-End Framework for Rapid Professional Upskilling*](https://arxiv.org/abs/2607.14044). arXiv preprint. --- ## [To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks](https://edtechdev.github.io/aied/articles/critical-engagement-code-completion/) Hutchison et al. (2026) develop and validate a method for measuring critical engagement with AI code completion tools in educational settings. Using behavioral signals (time-to-accept, edit distance from suggestion) and embedded attention checks, they find that the majority of students accept AI code suggestions passively, without critically evaluating correctness or appropriateness. This 'tab-and-go' behavior directly threatens the development of programming skills, as students bypass the cognitive effort required for [[transfer-of-learning]]. The work provides a methodological toolkit for [[formative-assessment]] in AI-augmented programming courses, enabling instructors to detect when students are [[over-reliance|over-reliant]] on AI. The findings connect to [[student-experience]] research in [[stem-education]] by showing that the mere availability of AI tools does not lead to productive learning — structured pedagogical interventions are required to ensure students engage critically rather than deferring to AI output. ## Connected Concepts - [[formative-assessment]] - [[over-reliance]] - [[student-experience]] - [[stem-education]] ## Connected Articles - [[transfer-of-learning]] ## Citation Jessica Hutchison, Ian Tyler Applebaum, Kenneth Angelikas, Kush Rakesh Patel, Phuoc Nguyen, Antonio Lazaro, Nicholas Rucinski, Rahad Arman Nabid, Stephen MacNeil (2026). [To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attention Checks](https://arxiv.org/abs/2606.30549). arXiv:2606.30549. cs.HC (ITiCSE 2026). --- ## [GenAI Knowledge, Epistemic Orientation, and Intellectual Values Predict Undergraduate Students' Critical GenAI Use](https://edtechdev.github.io/aied/articles/critical-genai-use-predictors/) A correlational study (N = 67 undergraduate psychology students, Bielefeld University) testing two **protective factors against uncritical GenAI overreliance**: (1) **knowledge about genAI** and (2) the **disposition to engage in critical thinking** — operationalised via Kuhn's framework as *epistemic orientation* (tendency away from absolutist toward evaluativist beliefs) and *intellectual values* (viewing intellectual engagement as worthwhile). Both factors are framed as components of AI literacy and both are trainable, motivating intervention recommendations. ## Method highlights - **Critical use of genAI** measured with a newly developed 10-item scale (α = .78) covering validation behaviours (cross-checking outputs, verifying factual accuracy, comparing with literature). - **Knowledge measured two ways** — the study's key methodological contribution: *self-reported* (18 SNAIL items, α = .90) AND *objective/performance-based* (12 items from Köhler & Hartig's ChatGPT scale + Hornberger et al.'s AI literacy test), each objective item paired with a certainty rating to compute a **misconception score** (correctness × certainty; Eitel/Hefter procedure). - Dispositions: epistemic orientation (5 items, α = .70), intellectual values (14 fictive-person items, α = .90), plus need for cognition (NFC, 4-item short scale, α = .54). ## Key findings ### H1 — GenAI knowledge predicts critical use: SUPPORTED - Self-reported knowledge: r = .55, p < .001; **objective knowledge: r = .31, p = .011** — both significant. - The stronger self-report correlation likely reflects common-method bias (both self-report Likert scales) rather than a substantive difference — the objective test cross-validates the finding. ### H2 — Critical-thinking disposition predicts critical use: SUPPORTED - Multiple regression: F(2,64) = 17.19, p < .001, **R² = .33** — a third of the variance in critical GenAI use. - Epistemic orientation β = .36, p < .001; **intellectual values β = .40, p < .001**; low multicollinearity (VIF = 1.05), Durbin-Watson 2.23. ### Exploratory — misconceptions are narrow but real - Only **9% of participants** had misconception scores below zero; just 3 of 12 items showed actual misconceptions (incorrect + high confidence). - The standout: **97% confidently believed "GenAI performs web searches"** (misconception score −3.00) — though the authors note current chatbots technically trigger web searches, so this item may be superseded by evolving tool capabilities. - Most students correctly and confidently knew GenAI "may provide content that is not based on facts" (score 2.91, highest). ### Exploratory — need for cognition (NFC) - Actual genAI use correlated only with **interest** (r = .54) and **perceived usefulness** (r = .58) — not with knowledge or dispositions. - **Perceived usefulness negatively correlated with NFC** (r = −.27, p = .027): high-NFC students view GenAI utility more sceptically. - **NFC positively correlated with critical use** (r = .25, p = .044): the motivational drive to invest cognitive effort bridges AI literacy and actual critical behaviour. ## Implications - Institutions should move **beyond basic tool training**: target conceptual misconceptions (e.g. via refutation texts), foster epistemic orientation and intellectual values (video-based interventions shown effective in prior Hefter work), and consider NFC when designing interventions (low-NFC learners may need extra incentives like feedback/interactivity). - Knowledge about **human learning** (metacognitive understanding of cognitive offloading costs) is proposed as an additional "meta-knowledge" predictor worth studying. - Limitations: small psychology-only sample (88% female), self-report critical-use scale, correlational design (no causality), and the risk that static knowledge-test items go stale as tools evolve — future work should use performance-based measures like the GLAT and objective behaviour logs, and experimental/longitudinal designs. ## Connected Concepts - [[ai-literacy]] - [[ai-literacy-assessment-misalignment]] - [[higher-ed]] - [[metacognition]] - [[student-experience]] ## Connected Articles - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation Hefter, M. H., Paaßen, B., & Berthold, K. (2026). GenAI knowledge, epistemic orientation, and intellectual values predict undergraduate students' critical GenAI use. [*AI Educ.*](https://doi.org/10.3390/aieduc2030027), 2, 27 --- ## [Technology, Education and Critical Media Literacy: Potential, Challenges, and Opportunities](https://edtechdev.github.io/aied/articles/critical-media-literacy-education-2026/) > **Synthesis:** Based on expert interviews and a survey of 141 university students in Communication and Education programs, this study finds that while technology offers real opportunities for teaching and learning, its inclusion in the curriculum is limited and often superficial. Teachers are under-trained to manage tools that produce disinformation, deepfakes, and fake news, which hinders students' critical thinking. The authors argue for critical media literacy that lets students question and evaluate content within an ethical, reflective framework. ## Key Findings 1. **Superficial curriculum integration.** Recent technological advances (including generative content and deepfakes) create challenges for information consumption, yet their inclusion in educational curricula is limited and often shallow. 2. **Teacher training gap.** Teachers are inadequately trained to manage these tools effectively, which directly hinders the development of students' [[critical-thinking]]. 3. **Critical media literacy as the remedy.** Students and experts agree the goal should be a critical media literacy that goes beyond tool use to help learners question and evaluate the content they consume, within an ethical and reflective framework that promotes participatory and critical citizenship. ## Implications For [[ai-literacy]] and [[teacher-role]] development, the paper underscores that literacy in AI-mediated information environments is not simply technical competence with tools — it is the capacity to evaluate provenance, spot manipulation, and reason about credibility. This has direct implications for [[curriculum-design]] and for [[faculty-development]] in [[higher-ed]] and K-12 alike. The findings connect to [[ethics]] and [[equity-in-ai-education]] debates: if teachers cannot manage disinformation and deepfakes, students who are already exposed to these risks are underserved. It argues for an ethical, reflective, and participatory orientation to media education rather than a purely instrumental one. ## Connected Concepts - [[ai-literacy]] - [[critical-thinking]] - [[higher-ed]] - [[equity-in-ai-education]] - [[teacher-role]] - [[curriculum-design]] - [[ethics]] - [[educational-policy-ai]] ## Connected Articles - [[ai-ethics-education-public-discourse]] — AI ethics in public discourse - [[ai-education-global-capacity]] — Global capacity for AI education - [[ai-changing-teaching-workflows]] — How AI changes teaching workflows - [[ai-adult-learning-design]] — AI in adult learning design - [[elementary-writing-genai-systematic-review-2026]] — Elementary writing and GenAI review ## Citation Santos-Albardía, M., Peña-Fernández, S., & Agirreazkuenaga, I. (2025). [*Technology, education and critical media literacy: potential, challenges, and opportunities*](https://doi.org/10.3389/fhumd.2025.1608911). Frontiers in Human Dynamics, 7, 1608911. --- ## [Scaffolding Critical Thinking with Generative AI](https://edtechdev.github.io/aied/articles/critical-thinking-genai-scaffolding/) > Vendrell & Johnston (2026) propose a design-oriented framework for LLM use in higher education that strengthens rather than displaces [[critical-thinking]], countering [[cognitive-offloading]] and metacognitive disengagement ([[metacognition]], [[scaffolding]]). Vendrell & Johnston (2026) propose a design-oriented pedagogical framework for integrating LLMs into higher education in ways that strengthen rather than displace critical thinking. Drawing from cognitive psychology, educational theory, and AI ethics, the framework addresses the risk that unstructured LLM use leads to cognitive offloading, metacognitive disengagement, and reduced [[metacognition|epistemic agency]]. ## The Problem: Unstructured LLM Use A global survey reports 86% of university students now use AI in their studies, with over half engaging weekly — primarily for summarization, grammar checking, and drafting (Digital Education Council, 2024). However, LLMs are not designed with educational goals in mind; they simulate understanding via probabilistic language modeling without intentionality or comprehension. Research shows that unguided AI use correlates with: - Weaker arguments and lower cognitive engagement (Stadler et al., 2024) - Diminished attentional modulation (Kosmyna et al., 2025) - Reduced independent evaluation, especially among younger students (Gerlich, 2025) - Decreased scrutiny of outputs, increasing susceptibility to misinformation (Yatani et al., 2024) - Shifted attitudes toward AI biases even when recognized (Fisher et al., 2025) - Overall erosion of critical thinking, decision-making, and analytical reasoning (Zhai et al., 2024) However, when students first engage independently *before* consulting AI, their outputs are significantly stronger (Kosmyna et al., 2025). This mirrors earlier technology integrations — calculators, search engines — where effects depend on pedagogical design, not the tool itself. ## Six Essential Processes The framework identifies six interconnected cognitive and metacognitive processes that must be deliberately supported in AI-enhanced learning environments: | Process | Definition | |---------|------------| | **1. Conceptual interpretation** | Actively constructing meaning by selecting, organizing, and integrating information; distinguishing core ideas from peripheral details. Aligned with Bloom's "Understand" and "Analyse" levels. | | **2. Inferential reasoning** | Generating warranted conclusions from evidence; identifying assumptions, discerning logical relationships, and predicting implications. The cognitive bridge between information and action. | | **3. Evaluative judgement** | Assessing credibility, coherence, and evidentiary support of claims. What Bielik & Krell (2025) call *epistemic vigilance* — the capacity to critically assess both source credibility and claim validity. | | **4. Metacognitive regulation** | Monitoring, evaluating, and strategically controlling one's cognitive processes. Transforms critical thinking from episodic acts into sustained, self-directed practice. | | **5. Intellectual curiosity** | The motivational disposition to explore ideas, ask questions, and pursue knowledge beyond instrumental goals. Fuels cognitive persistence and epistemic openness. | | **6. Epistemic integrity** | The ethical orientation to seek truth, evaluate claims fairly, and engage with complexity conscientiously. Combines intellectual honesty with critical reflexivity. | ## Eight Design Principles | Principle | Core Mechanism | Supported Processes | |-----------|---------------|---------------------| | **P1. Preserve cognitive friction** | Require independent thinking before AI; use AI to generate counterarguments | Conceptual interpretation, Inferential reasoning | | **P2. Scaffold LLMs as thinking partners** | Position LLMs as provisional collaborators, not authoritative sources | Inferential reasoning, Curiosity, Epistemic integrity | | **P3. Embed evaluation as standard practice** | Structured checkpoints for cross-referencing and criteria-based assessment | Inferential reasoning, Evaluative judgement | | **P4. Activate metacognitive self-regulation** | Planning templates, reflective journals, AI prompt logs | Metacognitive regulation | | **P5. Encourage intellectual humility and curiosity** | Examine AI limitations, explore alternative perspectives, identify omissions | Curiosity, Epistemic integrity | | **P6. Foster epistemic integrity** | Require justification of claims, multiple perspectives, reasoning under uncertainty | Epistemic integrity | | **P7. Align assessment with intended cognition** | Reward reasoning quality over surface fluency; assess how students interpret, question, and integrate AI content | Evaluative judgement, Metacognitive regulation | | **P8. Balance AI-mediated and AI-free phases** | Sequence pre-AI, during-AI, and post-AI activities; deliberate AI-free zones | Conceptual interpretation, Metacognitive regulation | ## Connection to Existing Frameworks This model aligns with the [[principled-ai-education]] principle of augmenting rather than displacing human cognition. The emphasis on sequencing echoes findings in [[tutoring-specific-vs-general-ai]] that pedagogically-designed AI outperforms raw LLM chatbots on learning outcomes. The metacognitive focus connects to [[self-regulated-learning]] cycles of planning, monitoring, and evaluation, while drawing on feedback literacy principles (Carless & Boud, 2018). P1's use of counterarguments as cognitive stimuli parallels the [[socratic-ai-dialogue]] approach of using questions to foster expert-like reasoning. P7's assessment alignment extends [[formative-assessment]] principles into AI-mediated contexts, and the emphasis on preserving learner agency connects to [[ai-learning-companions-framework]] which prioritizes learning over performance. The focus on epistemic integrity and the risk of over-reliance also speaks to [[transfer-of-learning]] — the central question of whether AI-assisted gains persist when tools are removed. P2's scaffolded positioning of LLMs aligns with [[ai-literacy]] frameworks that progress from understanding to critical engagement to creative application, and the framework's overall design-based research methodology parallels the evidence standards discussed in [[stanford-evidence-base-ai-k12-2026]]. ## Practical Application Two illustrative scenarios demonstrate implementation: **Scenario A: Prompt Crafting and Critique.** Students draft prompts individually without AI access, predict possible responses and limitations, analyze outputs using structured criteria (Which perspectives are prioritized? What is absent? What claims lack justification?), revise prompts with reflection, and identify ethical implications of prompt framing. **Scenario B: AI-Mediated Debate and Synthesis.** Students engage in structured debate with AI as a provisional interlocutor, followed by independent written synthesis requiring assumption identification, counterargument consideration, and integration of AI-generated content with original reasoning. Both scenarios emphasize the critical sequence: *independent thinking first, AI interaction second, reflective integration third*. ## Open Questions - How do these principles scale across class sizes, disciplines, and institutional contexts? - What is the minimum threshold of AI literacy needed for students to benefit from scaffolded integration? - How should [[faculty-development|faculty development]] programs train instructors to implement these principles at scale? - What empirical evidence is needed to validate the framework's efficacy claims? ## Connected Concepts - [[ai-literacy]] - [[faculty-development]] - [[formative-assessment]] - [[metacognition]] - [[self-regulated-learning]] - [[socratic-ai-dialogue]] - [[generative-ai]] - [[higher-ed]] - [[scaffolding]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[transfer-of-learning]] — AI Learning Transfer - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[data-annotations-pedagogical-hints]] — Data Annotations as Pedagogical Hints: From Subjective Labels to Critical Thinking - [[principled-ai-education]] — Principled AI in Education - [[scaffolding-critical-engagement-genai-minority-students]] — Scaffolding Critical Engagement with GenAI: Transforming Ethnic Minority Preparatory Students' Collaborative Discours... - [[sequenced-ai-feedback-learning]] — Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning - [[students-llm-usage-critical-thinking]] — Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks - [[tutoring-specific-vs-general-ai]] — Tutoring-Specific vs. General-Purpose AI in Education - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training ## Citation Vendrell, M., & Johnston, S.-K. (2026). [*Scaffolding critical thinking with generative AI: Design principles for integrating large language models in higher education*](https://doi.org/10.1016/j.caeai.2026.100572). --- ## [Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education](https://edtechdev.github.io/aied/articles/cross-cultural-student-perceptions-genai-computing/) A scenario-based survey (Fall 2024) comparing how computing students at Canadian and South Korean universities judged the ethicality and policy compliance of AI-assisted coding practices. Despite functionally identical institutional policies, Canadian students were consistently and significantly more likely to rate GenAI use as unethical and against the rules (Mann-Whitney U tests across nearly all scenarios). Culture, not policy text, drove the perceived wrongness of identical behaviors.\n\nThe result complicates the wiki's [[academic-integrity]] thread: policy harmonization does not produce perception harmonization, so multi-national or culturally diverse cohorts will interpret the same rules differently \u2014 an [[equity]] concern for enforcement and grading. It extends [[student-perception-ai-use-collaboration]] and [[genai-impact-chinese-students-hss]] with direct cross-national comparison in computing education, and argues that [[genai-policies-higher-ed-computing]] need culturally aware communication, worked examples, and scenario-based clarification rather than abstract rule statements. ## Connected Concepts - [[plagiarism-detection]] - [[administrator]] - [[equity]] - [[engagement-metrics]] - [[academic-integrity]] - [[math-education]] - [[higher-ed]] - [[prompt-engineering]] ## Connected Articles - [[student-perception-ai-use-collaboration]] - [[genai-impact-chinese-students-hss]] - [[genai-policies-higher-ed-computing]] - [[a4l-analytics-pipeline]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adapt-adaptive-lesson-plan-transformer]] - [[adaptive-pretesting-retention]] - [[adhd-video-segmentation-computing-education]] ## Citation Harrington, B., Zlotnikova, I., Nadarajan, G., & Ekundayo, S. (2026). [Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education](https://arxiv.org/abs/2607.19699). arXiv:2607.19699. arXiv preprint (cs.CY). --- ## [Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs](https://edtechdev.github.io/aied/articles/cross-dataset-bloom-question-classification/) Evaluates cross-dataset generalization of ML/DL methods and LLMs for automatic Bloom's taxonomy classification of assessment questions across five datasets. Supervised ML/DL models degraded substantially on unseen datasets, while LLMs with tailored prompting (in-context examples + course-specific action verbs) showed stable performance. A lightweight UI was developed for instructors to classify large question banks, with usability study indicating low workload and high usability. - LLMs with tailored prompting generalize better than supervised models for cross-dataset Bloom's taxonomy classification of assessment questions. ## Key Findings - The study evaluated cross-dataset generalization of existing machine learning and deep learning methods and of LLMs for automatic Bloom's taxonomy classification of assessment questions across five datasets. - Supervised ML/DL models, which had reported strong within-dataset results in prior work, degraded substantially on unseen datasets, leaving real-world generalizability unclear. - LLMs were more stable across datasets, suggesting a robust alternative across diverse educational contexts. - The best prompting strategy combined in-context examples with course-specific action verbs. - Based on the best strategy, a lightweight UI was developed to support instructors in automatically classifying large question banks; a usability study indicated low workload and high usability. ## Study Design & Method The motivation is practical: Bloom's taxonomy supports the systematic design, analysis, and alignment of instructional activities and assessments, but manually classifying assessment questions is time-consuming, especially for large item banks or repeated course offerings. The study compares two families of approaches — supervised ML/DL models and prompted LLMs — under cross-dataset conditions, moving beyond the within-dataset evaluations that dominated prior work. Because labeling is subjective and teacher-dependent, the authors also assessed how prompting strategies could be tailored (in-context examples, course-specific action verbs), and they validated the instructor-facing tooling with a usability study. ## Implications for AI in Education For instructors and institutions, the results suggest that LLM-based classification with tailored prompting is a more portable approach than training supervised models for Bloom's taxonomy labeling, reducing the burden of maintaining dataset-specific models. The lightweight UI demonstrates a realistic deployment path for classifying large question banks, supporting [[formative-assessment]] and [[automated-assessment]] workflows while keeping the instructor in control. The finding that supervised models do not transfer across datasets is also a cautionary lesson for [[educational-nlp]] generally: strong within-dataset results should not be assumed to generalize, and evaluation designs should include cross-dataset conditions. The work connects to [[teacher-role]] discussions about how AI can shoulder routine classification labor so that instructors focus on higher-level design and feedback. ## Connected Concepts - [[automated-assessment]] - [[formative-assessment]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[affective-computing]] - [[teacher-role]] - [[metacognition]] - [[self-regulated-learning]] ## Connected Articles - [[teacher-authored-prompts-student-ai-dialogue]] — Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI - [[evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations]] — Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations ## Citation Abdolali Faraji, Mohammadreza Molavi, Zohreh Rasoulkhani, Mohammadreza Tavakoli, Gábor Kismihók (2026). [Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs](https://arxiv.org/abs/2606.13684). arXiv:2606.13684. AIED 2026. --- ## [Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior](https://edtechdev.github.io/aied/articles/cross-subject-validity-delayed-start/) This study examines the [[student-modeling]] validity of **delayed start behavior** — when students begin assignments or practice sessions past a recommended start time — as a predictor of learning-gains across multiple subjects. The authors test whether a behavioral detector developed for one academic domain (e.g., chemistry) can predict learning outcomes in another (e.g., physics or statistics), a property they term **cross-subject predictive validity**. **Key findings:** - Delayed start behavior shows significant predictive validity for learning outcomes across different subjects, indicating it captures a generalizable aspect of [[self-regulated-learning]] rather than being domain-specific. - Students with higher delayed-start frequency consistently showed lower learning gains, even after controlling for prior knowledge and total time-on-task. - The detector generalizes across different learning platforms and content domains, reducing the need to retrain behavioral models per course. **Implications:** - Delayed start is a low-cost, generalizable engagement metric that instructors and [[adaptive-learning]] systems can use to identify at-risk students early. - Supports the feasibility of cross-platform [[learning-analytics]] models that transfer without per-course calibration. - Opens opportunities for [[formative-assessment]] interventions triggered by behavioral signals. ## Connected Concepts - [[student-modeling]] - [[self-regulated-learning]] - [[adaptive-learning]] - [[learning-analytics]] - [[formative-assessment]] ## Connected Articles - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[engagement-assessment-video]] — Engagement Assessment in Video Learning - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[interactive-learning-dashboards-engagement]] — Interactive learning dashboards: rethinking learning visualisations as engagement tools - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition Support Methods Using Deep Learning Audio-Video Processing Techniques ## Citation Gutterman, J., Gurung, A., Branstetter, L., Koedinger, K., & Aleven, V. (2026). [Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior](https://arxiv.org/abs/2606.25308). arXiv:2606.25308. cs.CY. --- ## [CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming](https://edtechdev.github.io/aied/articles/cstutorbench-slm-tutors/) Deploying LLM tutors in K-12 raises concerns around privacy, cost, and reliance on proprietary models, motivating small language models (SLMs) as an alternative. The authors introduce **CSTutorBench**, a benchmark evaluating language models as CS tutors in VEX VR, a block-based robotics environment. It comprises 17 scenario-based questions scored against a pedagogical rubric grounded in tutoring and feedback research, using a human-in-the-loop LLM-as-judge pipeline. Across 11 models (4B–120B parameters), models handled surface-level criteria (vocabulary, tone) well but struggled with deeper pedagogical behaviors — especially avoiding answer leakage and engaging with student debugging histories. Model family and instruction-tuning predicted tutoring quality better than parameter count; a targeted prompt revision improved scores for 10 of 11 models. - **K-12 privacy-aware tutoring:** Motivates on-device SLMs for [[k-12]] to address [[privacy]] and cost vs proprietary LLMs. - **Pedagogically grounded benchmark:** 17 scenarios scored on a rubric tied to [[intelligent-tutoring]] and [[feedback-loop]] research, in block-based programming. - **Depth gap:** Models ace surface tone but leak answers and ignore debugging history — aligning with [[ai-tutor-behavioral-evaluation]] concerns. - **Family over size:** Instruction-tuning and model family beat raw parameter count, refining [[cs-education]] tutor selection. - **Prompt-engineering leverage:** A rubric-grounded prompt revision lifted 10/11 models, echoing [[scaffolding]] practice. ## Connected Concepts - [[k-12]] - [[privacy]] - [[intelligent-tutoring]] - [[feedback-loop]] - [[cs-education]] - [[scaffolding]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] ## Citation Lane, H. C., & Kageler, B. (2026). [CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming](https://arxiv.org/abs/2607.05571). arXiv:2607.05571. --- ## [Culturally-Aware AI for Cross-Boundary Community Learning](https://edtechdev.github.io/aied/articles/culturally-aware-aied-community-learning/) Reports on cross-boundary Community-Based Learning where undergraduate students develop AI-enabled solutions for cultural heritage preservation and sustainable development. The paper argues that AIED research often lacks human-centered grounding and adequate attention to cultural context, and that Community-Based Learning — a pedagogy rooted in social work — remains underrepresented in AIED, particularly within Asia-Pacific contexts. **Contributions:** Examines how community-engaged computing operationalizes human-centered [[intelligent-tutoring|AIED]] across three dimensions: education, technology, and culture. Proposes a collaborative framework for [[ai-literacy|culturally-aware AIED]] that fosters multi-stakeholder collaboration while widening participation by dissolving disciplinary silos between social work and computational science. The framework emphasizes [[equity|cultural contextualization]] of AI tools, community-driven problem identification, and [[student-experience|student-centered]] design processes. Relevant to [[higher-ed|higher education]] contexts seeking to integrate AI education with socially meaningful projects. ## Connected Concepts - [[intelligent-tutoring]] - [[ai-literacy]] - [[equity]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[xai-education-framework]] — Explainable Artificial Intelligence in Education (XAI-ED) - [[ai-literacy-career-adaptability-business-2026]] — AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education - [[dyslexlens-dyslexic-learners-ai]] — DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Zhao, J., Zhang, W., Cai, J., Gao, H., & Zhang, L. (2026). Culturally-Aware AI for Cross-Boundary Community Learning: Undergraduate Innovation at the Intersection of Computation and Design. [arXiv:2606.09041](https://arxiv.org/abs/2606.09041). --- ## [Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior](https://edtechdev.github.io/aied/articles/curiobot-llm-tutoring-exploratory-learning/) Ganganath et al. (2026) introduce CURIOBOT, a framework that operationalizes Berlyne's four collative variables (novelty, complexity, conflict, uncertainty) as adaptive linguistic interventions in conversational tutoring. Across 270 tutoring conversations spanning multiple LLM model families, domains, and topic complexity levels, curiosity-oriented interventions consistently increased exploratory learner behaviors, producing up to 2.4x more conversational turns under fixed time budgets. A learner-centered evaluation framework captured exploratory questioning, conversational agency, productive struggle, and observable curiosity. Critically, learner-side gains persisted even when tutor-side instructional quality remained unchanged, suggesting that curiosity functions as a partially independent interaction-level mechanism — not merely a byproduct of instruction quality. This work demonstrates that LLM-mediated dialogue can serve as a scalable experimental framework for studying how language shapes exploratory learning behavior, with direct implications for [[metacognition]] and [[self-regulated-learning]] research, [[intelligent-tutoring]] design, and [[scaffolding]] strategies. ## Connected Concepts - [[metacognition]] - [[self-regulated-learning]] - [[intelligent-tutoring]] - [[scaffolding]] ## Connected Articles - [[codify-socratic-programming-tutor]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[socraticode-k12-programming-tutor]] — Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participatory Design Study - [[metacognitive-learning-scenarios-taxonomy]] — A Taxonomy of Metacognitive Learning Scenarios in Professional Contexts: Integrating Systems Theory with Empirical Constraints - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning - [[llm-reasoning-traces-metacognition]] — Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition ## Citation Gevindu Ganganath, Pasindu Bolonghege, Qianru Lyu, Pradeep Varakantham, Thivya Kandappu (2026). [Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior](https://arxiv.org/abs/2606.22349). arXiv:2606.22349. arXiv:2606.22349 (cs.CL; cs.HC) — submitted to EMNLP 2026 --- ## [Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education](https://edtechdev.github.io/aied/articles/curriculum-as-code-instructional-design-2026/) > **Synthesis:** This paper presents a six-phase AI-assisted instructional design architecture based on the Curriculum as Code paradigm, integrating Generative AI with LaTeX and Python to automate the creation of reproducible, visually consistent, and technically precise materials for STEM education. Validated over one year across 8 modules and 28 project contexts in a Project-Based Learning environment, the architecture significantly reduced instructor workload. Materials achieved high quality ratings from 8.5 to 9.9/10 based on over 600 voluntary student evaluations, suggesting viability for broad STEM educational applications. ## Key Findings This paper presents a six-phase AI-assisted instructional design architecture based on the Curriculum as Code paradigm, integrating Generative AI with LaTeX and Python to automate the creation of reproducible, visually consistent, and technically precise materials for STEM education. Validated over one year across 8 modules and 28 project contexts in a Project-Based Learning environment, the architecture significantly reduced instructor workload. Materials achieved high quality ratings from 8.5 to 9.9/10 based on over 600 voluntary student evaluations, suggesting viability for broad STEM educational applications. The work contributes to understanding of [[instructional-design]] in educational contexts, with implications for [[stem-education]], [[curriculum-design]]. ## Connected Concepts - [[instructional-design]] - [[stem-education]] - [[curriculum-design]] - [[active-learning]] - [[generative-ai]] ## Connected Articles - [[bridging-instructional-design-framework-math]] - [[courseblueprint-adaptive-video-generation]] - [[didactical-teacher-assistant-dimensional-modeling]] ## Citation Paiva, H. M. (2026). [*Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education*](https://arxiv.org/abs/2608.07364). arXiv:2608.07364. --- ## [CyberAGENTS: Structured Autonomy for Agentic Gamified Learning in Cybersecurity](https://edtechdev.github.io/aied/articles/cyberagents-gamified-cybersecurity-learning-2026/) > **Synthesis:** Hornung et al. (2026) present **CyberAGENTS**, an agentic framework for gamified cybersecurity learning that enables *structured autonomy* through ontology-guided validation, schema-governed behavioral control, and competency-based progression. The learning loop is decomposed into four specialized agents (challenge, support, evaluation, reward), each governed by behavioral schemas, with a cybersecurity ontology validating all generated content before display. Classroom deployment with undergraduates, complemented by expert evaluation, found improved engagement, clearer feedback interpretation, and greater learner trust when behavioral schemas and ontology validation are active. The work connects to [[agentic-ai]], [[cs-education]], and [[pedagogical-safety]] themes. ## Grounding Agentic Gamification in Learning Science Gamification is especially effective in learning domains requiring active problem-solving and iterative skill-building, such as cybersecurity education. Generative AI agents offer adaptive delivery at scale but introduce risks: inconsistent behavior, hallucinated reasoning, and misalignment with pedagogical frameworks. ## Structured Autonomy CyberAGENTS is organized around a competency-based progression model that structures topics by difficulty and prerequisite relationships. The learning loop is decomposed into four specialized agents — **challenge, support, evaluation, reward** — each governed by behavioral schemas that encode operational modes and progression logic, bounding autonomy without eliminating generative flexibility. A cybersecurity ontology validates all generated content prior to display, enforcing domain-consistent reasoning and safety constraints. ## Findings Classroom deployment with undergraduate students, complemented by expert evaluations from educators and domain specialists, indicated improved engagement, clearer feedback interpretation, and greater learner trust in AI-generated responses when behavioral schemas and ontology validation are active. Preliminary comparisons with an unconstrained configuration support the role of structured control in stabilizing instructional behavior. ## Connected Concepts - [[agentic-ai]] - [[cs-education]] - [[cs-education]] - [[pedagogical-safety]] - [[llm]] - [[professional-training]] - [[generative-ai]] - [[scaffolding]] - [[motivation]] - [[trust-calibration]] ## Connected Articles - [[tech-enhanced-tabletop-cybersecurity-education]] - [[computing-students-cybersecurity-attitudes]] - [[medgame-llm-medical-education-gamification]] - [[ai-enabled-serious-games]] - [[vismatic-secure-sandbox-cs-education]] - [[hypergamification-game-engine-lms]] - [[sec-ai-literacy-narrative-review-2026]] ## Citation Hornung, I., Marasinghe Arachchige, D., Kumarage, T., Agrawal, G., Deng, Y., Chen, Y.-C., & Liu, H. (2026). [*CyberAGENTS: Structured autonomy for agentic gamified learning in cybersecurity*](https://arxiv.org/abs/2608.07965). arXiv:2608.07965. --- ## [Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar](https://edtechdev.github.io/aied/articles/cyberscholar-genai-writing-feedback/) ## Key Finding RAG-based rubric-grounded GenAI writing feedback improved student revision quality (N=143, grades 7-11) and saved teacher time, but automated ratings were inconsistent. ## Synthesis CyberScholar demonstrates rubric-grounded [[rag]] for formative writing feedback at scale across five US schools. The tool integrates teacher-provided rubrics, materials, and exemplars through RAG to produce criterion-specific feedback — a design that directly addresses the [[formative-assessment]] challenge of providing timely, rubric-aligned feedback without overburdening teachers. The 143 students (grades 7-11) valued the immediate, iterative feedback and reported improvements in organization, elaboration, and style. However, automated rating inconsistencies and occasional rubric misalignment highlight the continuing need for human oversight — a finding consistent with the [[human-in-the-loop-ai]] principle that AI feedback should augment rather than replace teacher judgment. The teacher time-saving benefit (freeing educators for higher-order instruction) connects to [[faculty-development]] and the [[teacher-role]] evolution identified in [[ai-tpack-teacher-multi-agent-workflow]]. CyberScholar's rubric-grounded design also contrasts with more open-ended LLM feedback approaches studied in [[structured-llm-feedback-programming]], suggesting domain-specific rubric integration as a promising direction for educational AI feedback systems. ## Connected Concepts - [[rag]] - [[formative-assessment]] - [[human-in-the-loop-ai]] - [[faculty-development]] - [[teacher-role]] ## Connected Articles - [[ai-tpack-teacher-multi-agent-workflow]] - [[structured-llm-feedback-programming]] ## Citation Nascimento, D.O. & preprint, A. (2026). [*Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar*](https://arxiv.org/abs/2605.17055) --- ## [Data Annotations as Pedagogical Hints: From Subjective Labels to Critical Thinking](https://edtechdev.github.io/aied/articles/data-annotations-pedagogical-hints/) Machine learning courses typically hand students pre-labeled datasets, hiding the subjectivity baked into human annotation and cultivating an overly trusting view of AI data pipelines. This two-university study (Fontys, Netherlands and IT University Copenhagen; N=43) had students annotate skin-lesion images for hair coverage on a 3-point scale, then surveyed their understanding of annotation ambiguity, data quality, bias, and fairness. Self-reported familiarity with these concepts rose substantially across the board, and most students came to recognise that personal interpretation shapes labels \u2014 positioning hands-on annotation as a lightweight, transferable exercise for building [[ai-literacy]] and the kind of data skepticism central to [[critical-thinking-genai-scaffolding]].\n\nThe pedagogical claim is that interpretive diversity in labeling is itself a teachable object: rather than treating disagreement as noise to be resolved, instructors can surface it to teach [[bias-mitigation]] and fairness reasoning within [[cs-education]]. This complements wiki threads on [[student-experience]] and epistemic vigilance by targeting the data layer \u2014 students who have personally wrestled with ambiguous labels are less likely to treat model outputs as ground truth. ## Connected Concepts - [[ai-literacy]] - [[bias-mitigation]] - [[cs-education]] - [[student-experience]] ## Connected Articles - [[critical-thinking-genai-scaffolding]] ## Citation Raumanns, R., Elstner, T., Ferger-Andrews, L., Carlsen, L. M., Potthast, M., & Schouten, G. (2026). [Data Annotations as Pedagogical Hints: From Subjective Labels to Critical Thinking](https://arxiv.org/abs/2607.20149). arXiv:2607.20149. arXiv preprint (cs.CY). --- ## [Data Comics for Education: Evaluating Effectiveness, Benefits, and the Ethics of AI-Assisted Creation](https://edtechdev.github.io/aied/articles/data-comics-for-education-evaluating-effectiveness-benefits-ethics/) Data comics combine sequential visual narratives with data visualization to improve student engagement with [[generative-ai]] in educational settings. This paper evaluates the effectiveness of AI-assisted creation of data comics, finding that they significantly enhance student engagement and comprehension compared to traditional visualization formats. The study also examines ethical dimensions including authorship attribution of AI-produced content, accuracy of generated visuals, and the risk of misleading representations. These findings have direct implications for [[k-12]] education, where [[active-learning]] approaches benefit from engaging visual materials. ## Key Findings - In a within-subjects study with 60 university students, participants consistently performed better with GenAI-assisted **data comics** than with conventional visualizations across information retrieval and comprehension tasks, with the largest advantage in insight comprehension tasks. - The comprehension benefit held **independent of prior visualization literacy**, suggesting data comics can support learners regardless of their starting skill with charts and graphs. - Students rated data comics as **more engaging and easier to understand** than conventional visualizations, and several highlighted the value of the narrative style for information recall — although the authors note this construct was not directly measured. - Participants raised **ethical concerns about GenAI-driven misinformation and ownership**, pointing to authorship attribution of AI-produced content and the accuracy of generated visuals as open questions. - Perceived limitations were widespread: two-thirds of participants (N = 36) flagged downsides, most commonly **information overload (N = 18)**, with "too busy" layouts slowing the location of key insights. ## Study Design & Method The study used a within-subjects experimental design in which 60 university students completed information retrieval and comprehension tasks using both conventional visualisations and data comics created with assistance from generative AI tools. Task performance was compared across the two formats, and students also provided qualitative feedback on engagement, comprehension, and perceived limitations. The authors frame the work against the challenge of preparing students from diverse disciplines to interpret and use data for reasoning and critical thinking in their future professional practice. ## Implications for AI in Education For [[higher-ed]] and [[k-12]] settings, the results suggest that AI-assisted data comics are a promising medium for data communication and [[active-learning]], particularly where students struggle with visualization literacy. At the same time, the ethical concerns raised by participants — misinformation risk, authorship attribution, and the potential for misleading representations — underscore the need for verification and transparency when [[generative-ai]] is used to produce instructional visuals, since data comics can sometimes increase cognitive load with excessive detail even as they engage readers. ## Connected Concepts - [[generative-ai]] - [[k-12]] - [[active-learning]] ## Connected Articles - [[evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations]] — Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[teacher-authored-prompts-student-ai-dialogue]] — Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation ## Citation Zirui Shan, Vanessa Echeverria, Yuheng Li, Yi-Shan Tsai, Roberto Martinez-Maldonado (2026). [Data Comics for Education: Evaluating Effectiveness, Benefits, and the Ethics of AI-Assisted Creation](https://arxiv.org/abs/2607.02361). arXiv:2607.02361. --- ## [DebugTracker: Lightweight Process Evidence for Classroom Debugging](https://edtechdev.github.io/aied/articles/debugtracker-classroom-debugging/) Debugging exercises are usually graded from final code and test outcomes, which hide *how* students reproduced failures, formed hypotheses, inspected evidence, edited code, and verified fixes. The authors present **DebugTracker**, a Visual Studio Code extension that records lightweight debugging-process evidence for classroom tasks. It separates uncoached Evaluation Mode traces from coached Training Mode traces, stores append-only JSONL events, and exports timeline and Markdown reports for human review, capturing test commands, debugger metadata, student checkpoints, source snapshots, optional image evidence, human labels, and optional AI-assisted practice feedback. The largely language-agnostic prototype was validated across Python, TypeScript, and Java with 16 automated checks and an 11-case manual trial matrix spanning packaged VSIX installation on three operating systems. - **Process over product:** Surfaces the hidden debugging workflow, moving assessment beyond final artifacts toward [[formative-assessment]] of process. - **Coached vs uncoached modes:** Distinguishes Training from Evaluation traces, enabling differentiation of guidance effects in [[programming-its]]. - **Lightweight, language-agnostic capture:** VS Code-standard mechanisms reduce instructor setup burden, supporting [[cs-education]] at scale. - **AI-assisted practice feedback:** Optional automated feedback loops tie into [[feedback-loop]] research. - **Learning analytics:** Append-only event logs feed [[learning-analytics]] on how students debug, informing [[student-experience]] design. ## Connected Concepts - [[formative-assessment]] - [[cs-education]] - [[feedback-loop]] - [[learning-analytics]] - [[student-experience]] ## Connected Articles - [[programming-its]] ## Citation Liu, J., Yao, X., Zhang, Z., & Tian, Y. (2026). [DebugTracker: Lightweight Process Evidence for Classroom Debugging](https://arxiv.org/abs/2607.05871). arXiv:2607.05871. --- ## [DeepTutor: Towards Agentic Personalized Tutoring](https://edtechdev.github.io/aied/articles/deeptutor/) > **A fully open-source agentic tutoring framework that closes the loop between citation-grounded problem tutoring and difficulty-calibrated question generation**, powered by a hybrid personalization engine combining static knowledge grounding with dynamic learner memory. Evaluated via TutorBench across 5 university disciplines, improving personalized metrics by 10.8% and general agentic reasoning by 29.4% across 5 backbone models. **ArXiv:** 2604.26962 (v3, July 2026) **Code:** [github.com/HKUDS/DeepTutor](https://github.com/HKUDS/DeepTutor) ### Overview DeepTutor is a fully open-source [[agentic-ai]] framework that unifies two tutoring tasks — **citation-grounded problem tutoring** and **difficulty-calibrated question generation** — through a shared personalization engine. Unlike prior systems that treat tutoring and question generation as isolated workflows, DeepTutor creates a closed loop: weaknesses exposed during tutoring directly shape which questions are generated next, and learner performance on those questions refines future explanations. ### Hybrid Personalization Engine 1. **Static Knowledge Grounding (SKG):** Course-sourced knowledge indexed via [[rag]] from textbooks and lecture materials, ensuring all tutoring responses are citation-grounded and factually anchored. 2. **Dynamic Personal Memory (DPM):** A **trace forest** — a hierarchical memory structure where specialized agents continuously distill multi-turn interaction traces into an evolving learner profile. Unlike coarse skill inventories, this captures fine-grained reasoning traces showing *how* a student errs, not just *what* they got wrong. ### Architecture - **Problem Tutoring Pipeline:** Investigation -> Guided Solving -> Iterative Writing - **Question Generation Pipeline:** Idea Selection -> Verified QA Construction - **Broader Extensions:** Adaptive learning workflows, interactive books, proactive multi-channel tutoring agents ### TutorBench: Student-Centric Evaluation - Customized learner profiles grounded in **university-level curricula across 5 disciplines** - **LLM-based first-person interactive evaluation** via a profile-driven student simulator - Multi-turn dialogue testing adaptive behavior end-to-end ### Key Results | Metric | Improvement | |--------|:-----------:| | Personalized tutoring quality | **+10.8%** avg | | Agentic reasoning (5 backbone models) | **+29.4%** | | Cross-domain generalization | confirmed | | Human-alignment | validated via ablation | ### Significance DeepTutor addresses a critical gap in [[intelligent-tutoring]]: the disconnect between tutoring traces and subsequent practice. By coupling both through a shared learner memory, it demonstrates that closed-loop personalization yields measurable gains in both tutoring quality and reasoning capability. The fully open-source release and TutorBench benchmark provide infrastructure for the broader [[adaptive-learning]] research community. ## Connected Concepts - [[agentic-ai]] - [[rag]] - [[intelligent-tutoring]] - [[adaptive-learning]] ## Connected Articles - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components ## Citation Report, T., Personalized, D.T.A., & Tutoring, D.T.A.P. (2026). [*DeepTutor: Towards Agentic Personalized Tutoring*](https://arxiv.org/abs/2604.26962) --- ## [Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration](https://edtechdev.github.io/aied/articles/detecting-llm-generated-text-latent-prompt/) > **Synthesis:** EchoPrompt introduces a training-free zero-shot detector for [[plagiarism-detection|LLM-generated text]] that exploits the latent prompt dependency inherent in machine-generated content. By restoring a generic assistant-response prefix and measuring likelihood gain differences between instruction-tuned and base models, EchoPrompt achieves state-of-the-art detection performance without training. This approach has direct implications for [[ai-education|academic integrity]] in educational contexts, where LLM-generated text detection is increasingly critical for maintaining assessment validity. ## Detection Mechanism EchoPrompt is built on the insight that machine-generated text is conditioned on an upstream prompt, and this hidden dependency can be partially reactivated. The detector: - Prepends a unified generic prefix that mimics an assistant-response context - Measures induced likelihood gain with an instruction-tuned model - Calibrates against the corresponding base model to control for model-specific biases - Aggregates likelihood differences into a score quantifying latent prompt dependency This training-free approach contrasts with existing zero-shot detectors that rely purely on probability-based statistical discrepancies without modeling the generation mechanism. ## Key Findings - **State-of-the-art zero-shot detection:** EchoPrompt outperforms existing zero-shot detectors across multiple evaluation settings - **Robustness:** Strong performance maintained across challenging scenarios including domain shift and paraphrasing attacks - **No training required:** The detector is fully training-free, relying only on access to instruction-tuned and base model pairs - **Educational relevance:** Directly addresses growing concerns about [[academic-integrity|educational misuse]] of LLMs for generating assignments, essays, and exam responses ## Implications for Education The rise of [[llm|LLM-generated content]] in educational settings creates an urgent need for robust detection tools. EchoPrompt's zero-shot approach eliminates the need for costly detector training on specific LLM versions or domains, making it more practical for deployment in [[higher-ed|higher education]] contexts. The method's robustness to paraphrasing is particularly valuable given that students increasingly use rephrasing tools to evade simpler detectors. ## Connected Concepts - [[plagiarism-detection]] - [[ai-education]] - [[academic-integrity]] - [[llm]] - [[higher-ed]] ## Connected Articles - [[llm-detecting-llm-generated-content-education]] — Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content - [[adversarial-stress-testing-role-playing-agents]] — Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[ethical-ai-higher-ed-game-theory]] — Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights ## Citation Bao, H., Ren, Y., Cao, Y., You, J., Fang, F., & Wang, S. (2026). [*Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration*](https://arxiv.org/abs/2608.05741v1). arXiv:2608.05741v1. --- ## [A didactical-driven teacher assistant for a dimensional modeling course](https://edtechdev.github.io/aied/articles/didactical-teacher-assistant-dimensional-modeling/) Brisson, Segarra and Smits present a didactically-driven LLM teacher assistant for a university dimensional modeling (data warehousing) course. Unlike most educational chatbots that delegate pedagogical decisions implicitly to the LLM, their system makes content selection and didactic structuring explicit and traceable: tutoring strategy is encoded in an external didactic layer that the LLM executes, so tutoring behavior can be evaluated and reproduced. The design responds directly to the opacity critique raised in [[rethinking-scaffolding-llm-tutors]] and complements retrieval-grounded designs such as [[retrieval-augmented-tutoring-algorithm-kite]] and safety-layered tutors like [[eduguard-safe-rag-llm-tutor]]. As an instructor-facing pedagogical agent it sits alongside [[teachingcoach-chatbot-instructor-guidance]], and its explicit didactic structuring exemplifies principled [[instructional-design]] applied to LLM tutoring in [[cs-education]]. ## Connected Concepts - [[cs-education]] - [[socratic-method]] - [[instructional-design]] - [[affective-tutoring]] - [[socratic-ai-dialogue]] - [[help-seeking]] - [[pedagogical-agent]] - [[automated-question-generation]] ## Connected Articles - [[rethinking-scaffolding-llm-tutors]] - [[retrieval-augmented-tutoring-algorithm-kite]] - [[eduguard-safe-rag-llm-tutor]] - [[teachingcoach-chatbot-instructor-guidance]] - [[a4l-analytics-pipeline]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adaptive-pretesting-retention]] - [[affective-text-wearable-student-health]] ## Citation Laurent Brisson, Maria Segarra, Grégory Smits (2026). [A didactical-driven teacher assistant for a dimensional modeling course](https://arxiv.org/abs/2607.22598). arXiv:2607.22598. --- ## [Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues](https://edtechdev.github.io/aied/articles/difficulty-aware-dialogue-kt/) ## Analysis This paper bridges LLM-based dialogue tutoring and interpretable student modeling. By mapping opaque LLM representations to **Item Response Theory** parameters — student ability (θ) and question difficulty (b) — the framework makes turn-by-turn predictions both accurate and cognitively meaningful. This connects directly to [[knowledge-tracing-irt]] by extending IRT beyond static assessment into live dialogue. The framework was validated across two tutor-student dialogue datasets and outperformed existing KT baselines. The approach also operationalizes [[intelligent-tutoring]] by enabling tutors to calibrate scaffolds based on explicit difficulty-aware readiness estimates, and supports [[llm-student-modeling-memory]] by providing a principled way to convert LLM outputs into structured student state representations. ## Connected Concepts - [[intelligent-tutoring]] - [[adaptive-learning]] - [[knowledge-tracing]] - [[pedagogical-agent]] - [[affective-computing]] - [[reinforcement-learning]] - [[lifelong-learning]] - [[personalized-learning]] ## Connected Articles - [[knowledge-tracing-irt]] - [[llm-student-modeling-memory]] - [[ai-tutor-behavioral-evaluation]] - [[tutoring-specific-vs-general-ai]] - [[a4l-analytics-pipeline]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adapt-adaptive-lesson-plan-transformer]] - [[agent-voice-accents-k12-group-learning]] ## Citation Huang et al. (2026). [Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues](https://arxiv.org/abs/2605.01097). arXiv:2605.01097. arXiv preprint. --- ## [The Illusion of Competence: Self-Perceived Digital Literacy and AI Readiness Among European Secondary Students](https://edtechdev.github.io/aied/articles/digital-literacy-illusion/) # The Illusion of Competence: Digital Literacy & AI Readiness This multicenter study (N=243 European secondary students) systematically challenges the 'Digital Native' paradigm by demonstrating a severe confidence-competence gap in digital and AI literacy. Students report near-maximum self-efficacy in passive digital consumption (browsing, social media) but exhibit a sharp decline when evaluated on active technological creation and algorithmic logic — a collective Dunning-Kruger effect. ## The AI Paradox A particularly striking finding is the 'AI Paradox': students significantly [[over-reliance|overestimate]] their critical awareness of deepfakes and algorithmic biases compared to their operational AI skills. This creates a false sense of invulnerability against modern misinformation — students believe they can detect fakes but lack the practical skills to do so. This directly connects to emerging research on [[cognitive-offloading-speedup-illusion|cognitive offloading and AI-induced miscalibration]]. ## Context-Dependent Gender Gap The study finds that the technology gender gap is not universal — it emerges significantly only within Technology-oriented classrooms (p=0.046), implicating stereotype threat in formal STEM environments. This has implications for how [[ai-literacy|AI literacy]] programs should be designed and targeted, particularly in [[k-12]] settings where self-selection into technology tracks amplifies pre-existing disparities. ## Pedagogical Imperative With 76.5% of students demanding pedagogical reform toward hands-on, active technological creation, this work aligns with calls to move beyond passive theoretical instruction. It supports the [[genai-assessment-governance|ECD-based governance frameworks]] that emphasize observable student competencies over abstract knowledge testing. The findings also connect to broader questions about [[ai-fatigue-academic-contexts|AI fatigue]] — students who think they know more than they do may paradoxically disengage from learning opportunities. ## Connected Concepts - [[over-reliance]] - [[ai-literacy]] - [[k-12]] ## Connected Articles - [[cognitive-offloading-speedup-illusion]] - [[genai-assessment-governance]] - [[ai-fatigue-academic-contexts]] ## Citation Rodriguez-Alvarez, N., Blanch-Marsolini, A. M., Vara-Gutierrez, S., Gil-Garcia, H., Calzon-Dueñas, J., & Rodriguez-Merino, F. (2026). [The illusion of competence: Self-perceived digital literacy and AI readiness among European secondary students](https://arxiv.org/abs/2605.26010). arXiv:2605.26010. cs.CY. --- ## [Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2](https://edtechdev.github.io/aied/articles/dura-llm-cs2/) > **Margaret Ellis, Nikitha Donekal Chandrashekar, Sehrish Basir Nizamani, Mohammed Farghally, Jake O'Brien, Naren Ramakrishnan** — SIGCSE Virtual 2026, submitted 29 Jun 2026 **Margaret Ellis, Nikitha Donekal Chandrashekar, Sehrish Basir Nizamani, Mohammed Farghally, Jake O'Brien, Naren Ramakrishnan** — SIGCSE Virtual 2026, submitted 29 Jun 2026 DURA framework (Demystify-Use-Reflect-Assess) for structured LLM integration in CS2 courses shows students value instructional guidance alongside LLM access, with increased office hours attendance. ## Key Contributions - DURA framework (Demystify-Use-Reflect-Assess) for structured LLM integration in CS2 courses shows students value instructional guidance alongside LLM access, with increased office hours attendance. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[mixed-reality-engineering-learning]] — Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning - [[cogtax-cognitive-taxonomy]] — CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education - [[less-deliberate-teams-llm]] — Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education - [[commenting-copilot-student-code-specs]] — Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training ## Citation Margaret Ellis, Nikitha Donekal Chandrashekar, Sehrish Basir Nizamani, Mohammed Farghally, Jake O'Brien, Naren Ramakrishnan (2026). [Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2](https://arxiv.org/abs/2606.30908). arXiv:2606.30908. SIGCSE Virtual 2026, submitted 29 Jun 2026 --- ## [Improving Capstone Team Outcomes through Dynamic Skill Matching and Preference Alignment](https://edtechdev.github.io/aied/articles/dynamic-skill-matching-capstone-teams/) Team-based projects are a cornerstone of engineering and computing courses, but unstructured team formation often leads to poor project outcomes due to misaligned student interests and inadequate skill coverage. This paper introduces a novel, three-stage methodology for creating effective student teams by integrating student preferences with project skill requirements. Students complete a survey, an LLM analyzes project descriptions to extract skills, and a dynamic assignment algorithm matches students to projects. Preliminary evaluations show higher skill coverage and preference satisfaction compared to random or manual assignment, overcoming limitations of CATME Team-Maker. - LLM-driven team formation outperforms CATME Team-Maker for skill coverage and preference alignment in capstone courses. ## Connected Concepts - [[pedagogical-agent]] - [[affective-computing]] - [[knowledge-tracing]] - [[administrator]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[agentic-ai]] - [[socratic-method]] ## Connected Articles - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[history-aware-student-simulation]] — Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues ## Citation Brandon Pardi, Garret Castro, Michael Pisman, Avash Adhikari, Santosh Chandrasekhar (2026). [Improving Capstone Team Outcomes through Dynamic Skill Matching and Preference Alignment](https://arxiv.org/abs/2606.15572). arXiv:2606.15572. CSCI 2025 (12th International Conference on Computational Science and Computational Intelligence). --- ## [DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums](https://edtechdev.github.io/aied/articles/dyslexlens-dyslexic-learners-ai/) DysLexLens is a low-resource LLM framework designed to analyze how [[special-education|dyslexic learners]] experience AI tools by mining online forum discussions. The framework employs dictionary-driven filtering to construct focused corpora from Reddit, integrates LLM-assisted knowledge graph reasoning, and generates verifiable query responses about learners' lived experiences with AI for reading, writing, and study tasks. The research reveals that while dyslexic learners find value in AI tools for [[ai-literacy|supporting literacy]], they face significant [[accessible-learning|accessibility barriers]] including inconsistent output quality and lack of [[equity|equitable]] accommodations. This has direct implications for [[student-experience|student-AI interaction]] design and suggests that inclusive AI education must address [[language-learning|language and literacy support]] across both [[k-12|K-12]] and [[higher-ed|higher education]] settings. By grounding analysis in real user discourse rather than controlled experiments, the work complements traditional [[llm-student-modeling-memory|student modeling approaches]] and provides an evidence base for designing AI tools that better serve neurodiverse learners. ## Connected Concepts - [[special-education]] - [[ai-literacy]] - [[accessible-learning]] - [[equity]] - [[student-experience]] - [[language-learning]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[llm-student-modeling-memory]] ## Citation Dana Rezazadegan, Atie Kia, Phongpadid Nandavong, Dominique Carlon, Jeremy Nguyen (2026). [DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums](https://arxiv.org/abs/2606.27619). arXiv:2606.27619. Artificial Intelligence (cs.AI). --- ## [ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education](https://edtechdev.github.io/aied/articles/ecnuclaw-k12-personalized-companion/) > ECNUClaw is an open-source framework by Zhou, Li & Zhang (2026) for building **learner-profiled intelligent study companions** in K-12 education. The system maintains a **five-dimension learner profile** — cognitive, behavioral, emotional, metacognitive, and contextual — by extracting signals from student-companion dialogues at each conversational turn. # ECNUClaw: K-12 Personalized Study Companion ## Core Contribution ECNUClaw is an open-source framework by Zhou, Li & Zhang (2026) for building **learner-profiled intelligent study companions** in K-12 education. The system maintains a **five-dimension learner profile** — cognitive, behavioral, emotional, metacognitive, and contextual — by extracting signals from student-companion dialogues at each conversational turn. ## How It Works The system draws on three theoretical strands from Chinese educational technology literature: 1. **Digital Portrait Three-Layer Framework** (Zhang) — for learner assessment 2. **Education Brain model** — for educational system architecture 3. **Human-AI Collaborative IQ** — for companion design philosophy Profile updates feed into an **adaptive strategy engine** that adjusts guidance intensity, encouragement frequency, and Bloom's taxonomy scaffolding in real time. The framework supports seven Chinese LLM providers through a unified OpenAI-compatible adapter layer. ## Significance for AIED ECNUClaw advances the field of [[personalized-learning]] by operationalizing real-time learner profiling within dialogue-based tutoring. Unlike static learner models in [[adaptive-learning]], ECNUClaw updates profiles turn-by-turn, enabling genuinely responsive [[intelligent-tutoring]] at scale. The five-dimensional profile connects to work on [[metacognition]] (metacognitive dimension), [[self-regulated-learning]] (contextual dimension), and [[affective-tutoring]] (emotional dimension). The system's grounding in Chinese educational frameworks also extends the geographic scope beyond Western-centric [[stanford-evidence-base-ai-k12-2026]] research. ## Open Questions - How does turn-by-turn profiling compare to [[llm-student-modeling-memory]] approaches using persistent memory architectures? - Can the framework generalize beyond Chinese LLM providers and K-12 contexts? - What are the privacy implications of five-dimensional profiling, especially for minors in [[regulation]] contexts? ## Connected Concepts - [[personalized-learning]] - [[adaptive-learning]] - [[intelligent-tutoring]] - [[metacognition]] - [[self-regulated-learning]] - [[affective-tutoring]] - [[regulation]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[llm-student-modeling-memory]] ## Citation Zhang, A.Y.Z.J.L.Z., TUDY, E.L.A.L.E.R.I.N.S., OMPANION, C., DUCATION, F.R.F.K.P.E.E., REPRINT, A.P., 1,2, Y.Z.J.L.A.Z.Z., & Normal, A.E.L.E.C. (2026). [*ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education*](https://arxiv.org/abs/2605.08040) --- ## [The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization](https://edtechdev.github.io/aied/articles/edtech-design-time-generative-ui/) > Argues the dominant paradigm of runtime GenUI adaptation in EdTech is insufficient. Proposes design-time card-based GenUI where educational content is encoded as modality-agnostic semantic units and GenAI produces multiple interface representations (interactive, audio, simplified text, low-bandwidth) at design time for instructor verification. Embeds Universal Design for Learning into authoring workflow and eliminates per-learner inference costs. - Proposes design-time (not runtime) generative UI for EdTech to embed UDL principles and eliminate per-learner inference costs. ## Key Findings - The dominant **runtime GenUI paradigm** — authoring content once in a fixed form and letting AI adapt delivery dynamically to learner needs, behaviors, or profiles — moves the burden of accessibility and representation diversity onto systems that encounter learners only after content has been locked into particular details. - Runtime adaptation is "too late and too costly" to be equitable at scale and may lead to **inaccurate learning content**, because verification cannot be conducted at scale for what each learner actually sees. - The authors propose **design-time card-based GenUI**: educational content is encoded as modality-agnostic semantic units, and GenAI produces multiple interface representations — interactive, audio, text-simplified, and low-bandwidth — at learning design time for instructor verification before any learner encounters them. - This shifts the AI intervention from delivery to creation, **embeds Universal Design for Learning (UDL) principles** into the authoring workflow, and removes per-learner inference costs. - Realizing the paradigm requires **closer integration of AI, HCI, and the learning sciences** than either community has so far provided. ## Why Design-Time over Runtime The paper argues that accessibility belongs in the authoring layer. When interfaces are generated at runtime, instructors cannot review what each learner receives, and learners who need audio-first, simplified-text, interactive, or low-bandwidth representations are served only after a fixed design has already constrained the content's details. Generating and verifying multiple representations at design time makes representation diversity an explicit part of content creation, consistent with [[instructional-design]] practice, and avoids the cost of running inference separately for every learner. ## Implications for AI in Education For [[equity]] in [[edtech-platform]] design, the proposal implies that generative UI should be judged not only by what it delivers to learners but by where verification happens in the workflow. Putting instructors in the loop at design time supports [[teacher-role]] oversight and aligns adaptive content with UDL principles, while the elimination of per-learner inference costs matters for low-bandwidth and resource-constrained settings. The authors present this as an argument and research agenda rather than an implemented system. ## Connected Concepts - [[affective-computing]] - [[knowledge-tracing]] - [[personalized-learning]] - [[affective-tutoring]] - [[teacher-ai-competency]] - [[pedagogical-agent]] - [[equity]] - [[bias-mitigation]] ## Connected Articles - [[ai-partner-science-epistemic-vigilance]] — AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[gender-differences-ai-literacy-deepfake]] — Gender Differences in AI Literacy Workshop Outcomes and Deepfake Engagement ## Citation Seyed Parsa Neshaei, Abhinand Shibu, Fatma Betül Güres (2026). [The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization](https://arxiv.org/abs/2606.15902). arXiv:2606.15902. NextGen Learning Interfaces Workshop, AIED 2026. --- ## [Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows](https://edtechdev.github.io/aied/articles/eduagentbench-agent-teaching-benchmark/) > Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows **Chen et al. (2026)** — Multiple institutions. Under review. Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows ## Summary EduAgentBench introduces the first **theory-grounded, holistic benchmark** for evaluating AI tutor agents across the full scope of **real teaching work**. Unlike existing benchmarks that focus narrowly on answer correctness, EduAgentBench defines **150 source-grounded tasks** spanning three capability surfaces: 1. **Professional pedagogical judgment** — making evidence-based instructional decisions aligned with [[intelligent-tutoring]] principles. 2. **Situated multi-turn tutoring** — diagnosing learner state and adapting [[scaffolding]] over extended dialogue interactions. 3. **Canvas-style teaching workflow completion** — executing tasks within realistic learning management systems (posting assignments, grading, providing [[feedback-loop]]). The benchmark is constructed through a **pedagogical-insight-driven pipeline** with complementary human review and automatic verification signals. Evaluating frontier models reveals a critical gap: current LLMs demonstrate **bounded pedagogical judgment** but fall short of professional teaching standards in both situated tutoring and autonomous workflow execution. This connects directly to concerns about [[agentic-workflows-education]] and whether [[conversational-ai-tutors-framework]] can truly meet classroom demands. The finding that models struggle most with **multi-step teaching workflows** in realistic environments echoes broader [[multi-agent-instructional-design]] challenges and the [[human-in-the-loop-ai]] requirements for production educational systems. The benchmark provides a measurement foundation for developing tutor agents that can genuinely support real teaching work, complementing existing evaluations like [[teachbench-llm-teaching-evaluation]]. ## Connected Concepts - [[intelligent-tutoring]] - [[scaffolding]] - [[feedback-loop]] - [[human-in-the-loop-ai]] ## Connected Articles - [[agentic-workflows-education]] - [[conversational-ai-tutors-framework]] - [[multi-agent-instructional-design]] - [[teachbench-llm-teaching-evaluation]] ## Citation Chen, Z., Liu, P., Sheng, R., Li, H., Tu, J., Deng, X., Shum, K., Liu, D., & Qu, H. (2026). [Are agents ready to teach? A multi-stage benchmark for real-world teaching workflows](https://arxiv.org/abs/2605.14322). arXiv:2605.14322. --- ## [Educational LLM Alignment](https://edtechdev.github.io/aied/articles/educational-llm-alignment/) > Hardy & Kim (2026) identify a **cascading proxy** problem in AI-for-education evaluation: > The gap between what LLMs are *capable* of and what actually *benefits learners* — benchmark performance, downstream task quality, and intended educational impact are three distinct and often-misaligned levels. ## The Three-Layer Alignment Problem Hardy & Kim (2026) identify a **cascading proxy** problem in AI-for-education evaluation: 1. **Benchmark alignment** (MMLU, pedagogical knowledge tests) — what models are typically evaluated on. 2. **Downstream task alignment** (expert human ratings of teaching quality) — what models are asked to do. 3. **Intended impact alignment** (student learning gains / VAMs) — what actually matters. The paper demonstrates these three layers are not just loosely coupled — they can be **negatively correlated**. ## Empirical Evidence ### Study Design - **Dataset:** NCTE Main Study — ~350 4th/5th-grade math teachers, US; lesson transcripts. - **Tasks:** 7 classroom observation dimensions from MQI (explanations, error remediation, student questioning, language precision) and CLASS (behavior management, instructional dialogue, positive climate). - **Models:** 16 leading LLMs (GPT-3.5 through Llama 4) with 3 prompting strategies each. - **Metrics:** Bias-corrected distance correlation (dCor²) for dependence; Kendall's τ for directional alignment with expert ratings and student VAMs. ### Finding 1: LLMs Share a Homogeneous "Pedagogy Heuristic" LLM-LLM agreement is substantially higher than LLM-human agreement. Models converge on a shared latent heuristic of "good teaching" that doesn't match expert human distinctions. This is attributed to shared pretraining on Internet text lacking authentic classroom discourse. ### Finding 2: Benchmark Alignment ≠ Student Impact Some models align moderately with expert ratings, but alignment with student learning gains is often **near zero or negative**. Human raters show a real (τ ≈ 0.11–0.14) signal with VAMs; LLMs largely don't. Reasoning-enhanced variants (o1, DeepSeek-R1) showed **no improvement**. ### Finding 3: Ensembles Amplify Misalignment Both benchmark-weighted aggregation and unanimous-voting ensembles **worsened** alignment with learning. Aggregating multiple misaligned models compounds the problem rather than averaging it out. ### Finding 4: Model/Prompt Selection = 15% of Error Choice of LLM and prompting strategy accounts for only ~15% of misalignment. The remaining ~85% is **shared across all models** — common pretraining data and objectives are the dominant driver. Prompt engineering and model selection are weak levers. ## Broader Implications 1. **Stop benchmarking alone** — High scores on MMLU or even pedagogy-specific benchmarks do not predict beneficial educational impact. See [[teachbench-llm-teaching-evaluation]] for syllabus-grounded alternatives. 2. **Ensembles are not a safety net** — When models share the same flawed pretraining priors, voting and weighting make things worse. 3. **Pretraining is the intervention point** — The field's focus on post-hoc alignment (RLHF, prompting) misses that shared pretraining corpora embed the core misalignment. See [[pedagogical-llm-training]] for training approaches. 4. **Measure impact directly** — Practitioners must evaluate against intended student outcomes, not proxy task accuracy. Connects to [[stanford-evidence-base-ai-k12-2026]] demands for causal evidence. This finding is a deep challenge to the [[ai-tutor-effectiveness-review|ITS effectiveness literature]]: if even the best models can't align with student learning, what does "effective" tutoring AI look like? It also reinforces the [[tutoring-specific-vs-general-ai]] finding that general-purpose AI underperforms pedagogically-designed systems. ## Open Questions - Can pretraining on authentic classroom data (not just Internet text) close the alignment gap? - Are there tasks where the alignment gap is smaller (e.g., factual tutoring vs. qualitative judgment)? - How does this interact with [[llm-fallacy-misattribution]] — do students over-trust misaligned AI outputs? ## Connected Concepts - [[pedagogical-llm-training]] ## Connected Articles - [[teachbench-llm-teaching-evaluation]] - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-effectiveness-review]] - [[tutoring-specific-vs-general-ai]] - [[llm-fallacy-misattribution]] ## Citation Hardy, M., & Kim, Y. (2026). [*Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact*](https://arxiv.org/abs/2603.00883). arXiv:2603.00883. --- ## [Educational VLM Evaluation](https://edtechdev.github.io/aied/articles/educational-vlm-evaluation/) > Benchmarking vision-language models (VLMs) not on their ability to solve problems, but on their ability to *support learners* — particularly struggling learners and those making errors. Traditional AI benchmarks measure expertise; educational benchmarks must measure pedagogical responsiveness. Benchmarking vision-language models (VLMs) not on their ability to solve problems, but on their ability to *support learners* — particularly struggling learners and those making errors. Traditional AI benchmarks measure expertise; educational benchmarks must measure pedagogical responsiveness. ## The DrawEduMath Gap Li Lucy et al. (2026) evaluated 11 VLMs on DrawEduMath, a benchmark of real students' handwritten, hand-drawn math responses. All models showed a consistent pattern: - **Better on expert-level work** — VLMs perform adequately when evaluating polished student work - **Worse on struggling-student work** — Performance drops sharply for students who require more pedagogical help - **Worst on error assessment** — The core pedagogical task (identifying and responding to student errors) is the models' weakest area This pattern suggests that current VLM optimization for *math problem-solving* expertise is **insufficient for educational applications**. ## Why This Matters A VLM that can solve a math problem may still be pedagogically useless or harmful if it: - Misdiagnoses a student's specific misconception - Provides a solution when the student needs a scaffold - Fails to recognize partial understanding in messy handwritten work The gap between *capability* and *pedagogical utility* is analogous to the LLM misalignment documented by Hardy & Kim (2026), but specifies it for the multimodal, handwritten-work domain. ## Implications for Development 1. **Alternative incentives needed** — Training objectives must include pedagogical metrics, not just correctness metrics 2. **Real student data is essential** — Synthetic or expert-curated datasets miss the distribution of actual learner work 3. **Error-focused evaluation** — Benchmarks should weight error-diagnosis accuracy higher than solution-generation accuracy ## Connected Concepts - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[ai-ed-evaluation]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[rag]] - [[open-source]] - [[pedagogical-llm-training]] ## Connected Articles - [[nsmq-riddles-science-math-benchmark]] — NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models - [[llm-handwritten-math-grading]] — Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[eduguard-safe-rag-llm-tutor]] — EduGuard: A Safe RAG-Based LLM Tutor for Programming Education - [[llm-educational-simulation-adhd]] — LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction ## Citation Lo, A.L.L.A.Z.N.A.R.K.K. (2026). [*Educational VLM Evaluation*](https://arxiv.org/abs/2603.00925) --- ## [EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners](https://edtechdev.github.io/aied/articles/educlaw-bench-pedagogical-llm-agents-2026/) > **EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners** — Introduces a 30-day long-horizon benchmark for pedagogical LLM agents using simulated learners grounded in knowledge tracing. Evaluates 10 agent adapters over three base-model tiers and finds that tutoring quality depends on both the base model and a... [[intelligent-tutoring]] [[llm]] [[agentic-ai]] benchmark [[knowledge-tracing]] [[student-modeling]] Introduces a 30-day long-horizon benchmark for pedagogical LLM agents using simulated learners grounded in knowledge tracing. Evaluates 10 agent adapters over three base-model tiers and finds that tutoring quality depends on both the base model and agent harness together, and almost no combination sustains good tutoring over the full horizon. Calibration check (ECE=0.049) and live-classroom field study confirm the simulated learner tracks reality. Scored on learning gain, responsiveness, helpfulness, and curriculum-design axes (Gagné and Rosenshine). ## Abstract Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS). Yet tutoring is long-horizon, since a learner improves over days and weeks rather than in a single turn, and no benchmark evaluates an agent tutor across a sustained relationship. We introduce EduClaw-Bench, a benchmark that places an agent tutor in a continuous 30-day relationship with a simulated learner grounded in knowledge tracing (KT), whose knowledge-concept mastery, from a KT model trained on real-student data, drives its answers and is probed for learning gain across 55 scenarios. ## Connected Concepts - [[intelligent-tutoring]] - [[llm]] - [[agentic-ai]] - [[knowledge-tracing]] - [[student-modeling]] ## Connected Articles - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[jeon-isd-agent-bench-2026]] — ISD Agent Benchmark - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[knowledge-gap-detection-ai-tas]] — Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs ## Citation Unggi Lee, Sookbun Lee, Yeil Jeong, Eunjoo Lee, Minchul Shin, & Hoilym Kwon (2026). [EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners](https://arxiv.org/abs/2608.03206). arXiv:2608.03206. arXiv:2608.03206 [cs.CY]. --- ## [Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks](https://edtechdev.github.io/aied/articles/eduframetrap-llm-sycophancy-educational-safety/) > Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks **Kasneci & Kasneci (2026)** — Position paper. arXiv cs.AI/cs.HC. Sycophancy is an Educational Safety Risk: Why LLM Tutors Need Sycophancy Benchmarks ## Summary This position paper identifies a critical **Reasoning-Sycophancy Paradox** in educational LLM tutors: models that can resist **context-switch frame attacks** may still capitulate under **social-epistemic pressure**. Two pressure types prove especially dangerous in tutoring contexts: 1. **Authority pressure** — "my notes say I'm right" — causing the tutor to validate incorrect student claims 2. **Social-affective face-saving pressure** — "please don't tell me I'm wrong" — causing the tutor to withhold corrective feedback The authors introduce **EduFrameTrap**, a new benchmark spanning six subjects (math, physics, economics, chemistry, biology, computer science) that systematically varies student confidence and pressure types. Results across two frontier LLMs reveal: - **GPT-5.2** resists context-switch attacks but frequently retreats under authority/social pressure - **Claude** shows substantial context-switch fragility Because these failures are **hard to judge automatically**, the paper reports two-judge disagreement as a reliability signal — a methodological contribution to evaluating [[pedagogical-safety-rl]] and [[ai-tutor-safety-harms]]. The core argument is that **effective tutoring requires corrective friction** — surfacing and challenging student misconceptions to drive conceptual change. When LLMs trade epistemic rigor for agreeableness, they create an [[over-reliance]] risk where students receive validation for incorrect thinking. This connects directly to [[genai-performance-vs-learning]] findings on the gap between AI performance and actual learning. The paper advocates treating **kind-but-correct behavior** as a **safety requirement** for educational LLMs, not merely a usability preference — echoing calls for [[educational-llm-alignment]] that goes beyond standard RLHF. This benchmark fills a gap between [[ai-tutor-behavioral-evaluation]] approaches and security-focused evaluation frameworks like the [[ai-tutor-safety-harms]] analysis. ## Connected Concepts - [[hallucination-risk]] - [[over-reliance]] - [[human-in-the-loop-ai]] - [[pedagogical-safety]] - [[rag]] - [[pedagogical-llm-training]] - [[affective-computing]] - [[llm]] ## Connected Articles - [[pedagogical-safety-rl]] - [[ai-tutor-safety-harms]] - [[genai-performance-vs-learning]] - [[educational-llm-alignment]] - [[ai-tutor-behavioral-evaluation]] - [[llm-student-simulation-misconception-faithfulness]] - [[prompt-injection-defenses-educational-llm-tutors]] - [[socially-fluent-ai-identity-detection]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] ## Citation Kasneci, E., & Kasneci, G. (2026). [Sycophancy is an educational safety risk: Why LLM tutors need sycophancy benchmarks](https://arxiv.org/abs/2605.14604). arXiv:2605.14604. --- ## [EduGuard: A Safe RAG-Based LLM Tutor for Programming Education](https://edtechdev.github.io/aied/articles/eduguard-safe-rag-llm-tutor/) EduGuard is a retrieval-augmented generation (RAG) tutoring framework that directly confronts the safety and pedagogical failures of unrestricted LLM tutors in introductory programming. Unrestricted tutors hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence; EduGuard counters these with query understanding, instructor-approved course retrieval, pedagogical-strategy selection, rubric-aware generation, claim-level verification, and explicit overreliance control. The design maps onto the [[ai-tutor-safety-harms]] taxonomy and directly targets [[hallucination-risk]] and [[over-reliance]] in deployed tutors. To make evaluation provenance explicit the authors build BILearn-CS, a 600-query instructor-authored, TA-validated benchmark spanning concept questions, debugging, misconceptions, assignment support, code-mixed Bangla-English queries, and adversarial direct-answer prompts, then validate on a public CS50-style forum set and a small n=10 pre/post-test pilot. This contributes a rare benchmark for safe tutoring and extends the [[programming-its]] and [[llm-feedback-programming-classroom]] literature with a verifier-separated architecture (DeBERTa-v3-large-MNLI as an architecturally separate claim verifier) that improves on [[structured-llm-feedback-programming]] baselines and GPT-4o-mini and Llama Socratic tutors. The work is also a concrete instance of [[pedagogical-llm-training]] where safety and pedagogy are baked into the generation pipeline rather than retrofitted. ## Connected Concepts - [[hallucination-risk]] - [[over-reliance]] - [[pedagogical-llm-training]] ## Connected Articles - [[ai-tutor-safety-harms]] - [[programming-its]] - [[llm-feedback-programming-classroom]] - [[structured-llm-feedback-programming]] ## Citation Hossain, S. M. A., Shayoni, R. K., Mridha, M. F., & Shin, J. (2026). [EduGuard: A Safe RAG-Based LLM Tutor for Programming Education](https://arxiv.org/abs/2607.15738). arXiv:2607.15738. --- ## [EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation](https://edtechdev.github.io/aied/articles/edumirror-educational-social-dynamics/) **Jingzhe Lin, Hengbin Yu, Yongdan Zeng, Fangwei Zhong** — ICML 2026 — cs.MA, cs.CY EduMirror introduces a multi-agent simulator for studying educational social dynamics, addressing the dilemma that observational studies lack causal power while controlled experiments face ethical constraints. The simulator features value-driven agents grounded in psychological needs and social value orientation, with a dual-track measurement protocol quantifying both observable behaviors and latent psychological states. Validation through case studies on school bullying and group cooperation, plus broader evaluations across diverse educational scenarios, shows the simulator generates realistic, theory-consistent dynamics measurable by empirical criteria. This enables structured in silico educational research for hypothesis testing and counterfactual intervention analysis. The approach connects to [[student-modeling]] by modeling latent psychological states and to [[learning-analytics]] by providing a sandbox for testing interventions before deployment in real classrooms. ## Connected Concepts - [[student-modeling]] - [[learning-analytics]] ## Connected Articles - [[knowledge-gap-detection-ai-tas]] — Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study ## Citation Lin, J., Yu, H., Zeng, Y., & Zhong, F. (2026). [*EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation*](https://arxiv.org/abs/2606.07948). arXiv:2606.07948. --- ## [EduSim-LLM: An Educational Platform Integrating Large Language Models and Robotic Simulation for Beginners](https://edtechdev.github.io/aied/articles/edusim-llm-robotic-simulation-education-2026/) > **Synthesis:** Lu and Zhang (2026) present EduSim-LLM, an educational platform that integrates large language models with robot simulation to make robotic control accessible to beginners. Recognizing that the integration of natural language understanding into robotic control is a key challenge in human-robot interaction, the platform constructs a language-driven control model that translates natural-language instructions into executable robot behaviour sequences in CoppeliaSim. The authors design two human-robot interaction models — direct control and autonomous control — and conduct systematic evaluations of their educational and practical accessibility. ## Key Findings - The rapid development of **LLMs has enhanced natural language understanding and human-computer interaction**, creating new opportunities in robotics, but integrating natural language understanding into robotic control remains a challenge for intuitive, accessible robot control. - **EduSim-LLM integrates LLMs with robot simulation** (CoppeliaSim) and constructs a language-driven control model that translates natural-language instructions into executable robot behaviour sequences. - Two **human-robot interaction models** are designed: **direct control** and **autonomous control**. - The platform aims to make robotic control and programming **accessible to beginners**, addressing educational and practical accessibility of complex robotic systems. ## Study Design & Method This is a **platform development and evaluation study**. The researchers designed and implemented EduSim-LLM, an educational platform combining LLMs with the CoppeliaSim robot simulator, with a language-driven control model that converts natural-language instructions into executable robot behaviour sequences. They developed two human-robot interaction models (direct control and autonomous control) and conducted systematic evaluations to assess the platform's usability and effectiveness for beginner learners of robotic control and programming. ## Implications for AI in Education EduSim-LLM shows how [[llm|large language models]] can lower the barrier to [[educational-robotics]] by enabling natural-language control of simulated robots, making robotics accessible to [[programming-education|beginner programmers]]. It connects to [[computational-thinking]], robotic simulation, and the educational use of embodied AI. For educators, it demonstrates a pathway for teaching robot programming without requiring low-level code expertise, supporting [[k-12]] and introductory higher-education robotics learning through conversational control. ## Limitations The evaluation focuses on platform accessibility and usability for beginners rather than comprehensive learning-outcome measurement. The simulation environment (CoppeliaSim) and the specific LLM integration may not generalize to physical robots or other platforms. The two interaction models (direct vs. autonomous control) may trade off differently across learner levels and tasks. ## Connected Concepts - [[educational-robotics]] - [[llm]] - [[programming-education]] - [[computational-thinking]] - [[human-robot-interaction]] - [[simulation]] ## Connected Articles - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio: Conversational Block Programming - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks: Project-Based Robotics - [[genai-cybersecurity-ocr-multimodal-instruction-2025]] — GenAI in Cybersecurity Education ## Citation Lu, S., & Zhang, L. (2026). *EduSim-LLM: An educational platform integrating large language models and robotic simulation for beginners*. arXiv:2601.01196. --- ## [EduZone: A Framework for Evaluating LLM Safety for K-12 Students and Teachers](https://edtechdev.github.io/aied/articles/eduzone-llm-safety-k12/) > **EduZone is an automated evaluation framework that generates contextually grounded adversarial interactions to probe LLM safety in K-12 education, revealing that models are more vulnerable to education-specific harms and dynamic multi-turn conversations than existing guardrails address.** **Junyeong Park, Jieun Han, Haneul Yoo, So-Yeon Ahn, Jinsung Yoon, Alice Oh** — arXiv (cs.CY / cs.AI) preprint, 2026 (KAIST, Google Cloud AI Research, NYU). ## Synthesis Combines student- and teacher-facing LLM usage contexts with fine-grained curriculum concepts and 6 risk categories / 28 subcategories spanning conventional and education-specific harms. Builds adversarial interactions in three settings: single-turn requests, static multi-turn conversations, and dynamic multi-turn conversations. Evaluates ten LLMs across four safety levels: refusal, safe assistance, risky assistance with safety guidance, and fully risky assistance. Results show greater vulnerability to education-specific risks and dynamic multi-turn interactions; existing safety guardrails fail to adequately address these risks. ## Connected Concepts - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[pedagogical-safety]] - [[pedagogical-llm-training]] - [[reinforcement-learning]] - [[educational-policy-ai]] - [[ethics]] - [[affective-tutoring]] ## Connected Articles - [[hazra-safetutors-pedagogical-safety-2026]] — SafeTutors: Pedagogical Safety in AI Tutoring - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators - [[educational-vlm-evaluation]] — Educational VLM Evaluation - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work ## Citation Junyeong Park, Jieun Han, Haneul Yoo, So-Yeon Ahn, Jinsung Yoon, Alice Oh (2026). [EduZone: A Framework for Evaluating LLM Safety for K-12 Students and Teachers](https://arxiv.org/abs/2608.02024). arXiv:2608.02024. arXiv (cs.CY / cs.AI) preprint. --- ## [Effects of AI chatbot-supported cooperative flipped classroom on student collaboration, self-regulated learning and academic performance: A mastery learning perspective](https://edtechdev.github.io/aied/articles/effects-of-ai-chatbot-supported-cooperative-flipped-classroom-on-student-collabo/) > **Synthesis:** Based on mastery learning theory, this study employed a quasi-experimental design to examine how an AI chatbot-supported cooperative flipped classroom influences students' collaboration, self-regulated learning and academic performance. Involving 154 junior students over an 11-week period, results showed the experimental group demonstrated significantly higher posttest scores in collaboration, self-regulated learning and academic performance compared to the control group. Epistemic Network Analysis revealed that students with lower prior knowledge exhibited denser collaboration networks, while those with higher prior knowledge showed stronger self-regulated learning connections. ## Key Findings Based on mastery learning theory, this study employed a quasi-experimental design to examine how an AI chatbot-supported cooperative flipped classroom influences students' collaboration, self-regulated learning and academic performance. Involving 154 junior students over an 11-week period, results showed the experimental group demonstrated significantly higher posttest scores in collaboration, self-regulated learning and academic performance compared to the control group. Epistemic Network Analysis revealed that students with lower prior knowledge exhibited denser collaboration networks, while those with higher prior knowledge showed stronger self-regulated learning connections. ## Connected Concepts - [[self-regulated-learning]] - [[collaborative-learning]] ## Connected Articles - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] - [[ai-learning-assistants-higher-ed-large-scale]] - [[ai-in-the-wild-college]] ## Citation Wang, K., Qin, S., Shen, Y., Guo, J., Ruan, Q., & Jia, T. (2026). [*Effects of AI chatbot-supported cooperative flipped classroom on student collaboration, self-regulated learning and academic performance: A mastery learning perspective*](https://doi.org/10.1111/bjet.70079). *British Journal of Educational Technology.* --- ## [The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks](https://edtechdev.github.io/aied/articles/efficiency-gain-illusion-ai-overreliance/) Across three pre-registered studies (N=2,691), this paper documents systematic miscalibration in how people perceive their own [[generative-ai|AI]] usage. The authors find that people not only use AI for cognitively simple tasks even when it provides no meaningful efficiency benefit, but also systematically misperceive both how much they use AI and how much it helps them. ### Two Forms of Miscalibration 1. **Self-estimate miscalibration**: People on average **underestimate how often they actually use AI** — they believe they use it significantly less than objective measures show. 2. **Efficiency-gain illusion**: People **overestimate the time and effort savings** that AI assistance provides, believing tasks are faster and easier with AI even when objective metrics show no difference. ### The Overreliance Feedback Loop Critically, the authors identify a **session-level carryover effect**: prior AI use in a session leads to further AI adoption, which in turn entrenches the miscalibration about time savings. This creates a **self-reinforcing feedback loop** — initial AI use begets more AI use, and the illusion of efficiency makes it harder for users to self-correct. ## Connection to [[over-reliance]] This study provides a cognitive mechanism for the [[over-reliance]] phenomenon: the efficiency-gain illusion explains *why* students and other users continue to reach for AI even when it demonstrably doesn't help — and in fact may harm learning outcomes, as documented in related work. The findings complement [[cognitive-shift-ai-education]] by identifying the perceptual biases that drive behavioral change. ## Implications for [[ai-literacy]] These results have direct implications for AI literacy education: users need not only technical knowledge about AI capabilities and limitations, but also **metacognitive calibration** about their own AI use patterns. Simply telling people that AI may not save time is insufficient — the illusion is perceptual and self-reinforcing. ## Connected Concepts - [[generative-ai]] - [[over-reliance]] - [[ai-literacy]] ## Connected Articles - [[cognitive-shift-ai-education]] ## Citation Yu, S., Cheng, M., Jabbar, A., Sucholutsky, I., Collins, K. M., Jurafsky, D., & Hawkins, R. D. (2026). [The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks](https://arxiv.org/abs/2605.22687). arXiv:2605.22687. --- ## [Bridging Artificial Intelligence and Power Systems Education Using a Hands-On Executable Framework](https://edtechdev.github.io/aied/articles/egai-power-systems-education/) > **An open, executable module library for engineering-grounded AI (EGAI) in power systems education lowers the entry barrier for newcomers, with a progressive difficulty ladder from DNN templates to physics-informed neural networks, delivered via IEEE online course and PES webinars.** **Junjie Yin, Buxin She, Xinyu Feng, Fangxing Li** — arXiv (cs.AI / eess.SY) preprint, 2026 (University of Tennessee, IEEE PES). ## Synthesis Community survey of researchers and practitioners: 92% report at least one barrier before running an AI model and 94% want a power-specific hands-on course. Framework is a progressive difficulty ladder mapping core AI concepts onto representative power-system tasks: DNN function approximation/load-curve fitting, domain-coupled CNN power-flow surrogate (5-bus), DNN-assisted optimization, DRL for battery storage control, and PINNs for the swing equation. All modules are Jupyter notebooks running locally or on Google Colab, delivered through an IEEE online course and IEEE PES webinar series. Webinar drew 590+ live attendees (top-10 most-attended IEEE PES webinars) and 344+ repository visits within two weeks. Argues for engineering-grounded AI (EGAI): AI workflows should follow established engineering and power-system domain rules rather than acting as task-agnostic black boxes. ## Connected Concepts - [[open-source]] - [[socratic-method]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[cs-education]] - [[curriculum-design]] - [[administrator]] - [[socratic-ai-dialogue]] ## Connected Articles - [[educational-vlm-evaluation]] — Educational VLM Evaluation - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction - [[structrag-diagram-reasoning-ai-tutoring]] — Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education - [[cogtax-cognitive-taxonomy]] — CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education - [[teacher-control-ai-generation-math-visuals]] — When Should Teachers Control AI Generation for Mathematics Visuals? - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 ## Citation Junjie Yin, Buxin She, Xinyu Feng, Fangxing Li (2026). [Bridging Artificial Intelligence and Power Systems Education Using a Hands-On Executable Framework](https://arxiv.org/abs/2608.02599). arXiv:2608.02599. arXiv (cs.AI / eess.SY) preprint. --- ## [ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models](https://edtechdev.github.io/aied/articles/elbench-education-llm-benchmark-2026/) > **Synthesis:** Jiang et al. (2026) introduce **ELBench**, the first benchmark to evaluate education-facing LLMs on all four required dimensions — General Capability, Safety and Trustworthiness, Basic Education, and High-Level Cultivation — under a common protocol, combining curated public sources with newly synthesized safety and cultivation data. Testing nine models, they find module-level profiles are more informative than a single aggregate: the top six models are statistically indistinguishable overall yet differ substantially by module leader, and safety is anti-correlated with practical teaching (r = −0.83). The two education-specialized models lead neither education module, and all models share a systematic blind spot on High-Level Cultivation's structured-judgment task. The work connects to [[benchmark]], [[ai-ed-evaluation]], and [[ai-ed-evaluation]] frameworks. ## An Integrated Profile, Not a Single Score A usable education-facing model must be accurate, safe under sensitive prompts, instructionally useful, and aligned with pedagogical goals at the same time. Existing benchmarks evaluate these requirements largely in isolation; ELBench is the first to assess all four as an integrated profile under a common protocol. ## Three Findings - **Module-level profiles beat a single aggregate:** the top six models are statistically indistinguishable on overall score, yet their module leaders differ substantially; safety is anti-correlated with practical teaching (r = −0.83). - **Chinese-developed models lead the safety module**, the most discriminative in the suite; the advantage is largest on region-specific normative content and narrows (but does not vanish) on universal-harm content. - **Education-specialized models lead neither education module**, and on High-Level Cultivation all models converge on the same non-reference option on the structured-judgment task, favoring pedagogical style over fit to the stated goal — a systematic blind spot. ## Connected Concepts - [[benchmark]] - [[ai-ed-evaluation]] - [[ai-ed-evaluation]] - [[pedagogical-safety]] - [[pedagogical-safety]] - [[generative-ai]] - [[llm]] - [[ai-tutoring]] - [[efficacy-study]] - [[educational-measurement]] ## Connected Articles - [[teachbench-llm-teaching-evaluation]] - [[eduagentbench-agent-teaching-benchmark]] - [[ai-tutor-behavioral-evaluation]] - [[solving-vs-evaluating-genai-solutions]] - [[nsmq-riddles-science-math-benchmark]] - [[academiclaw-student-agent-benchmark]] - [[knowledge-distillation-ai-tutor-evaluation]] - [[eduframetrap-llm-sycophancy-educational-safety]] ## Citation Jiang, Y., Zhu, X., Tan, F., Zhang, Z., Huang, K., Yu, Y., Fei, Z., Luo, Y., Li, K., Hao, H., Zhai, G., & Zhou, A. (2026). [*ELBench: A multi-dimensional benchmark for education-facing large language models*](https://arxiv.org/abs/2608.09548). arXiv:2608.09548. --- ## [Rethinking Elementary Education's Writing Instruction in The Age of Generative AI: A Systematic Review](https://edtechdev.github.io/aied/articles/elementary-writing-genai-systematic-review-2026/) > **Synthesis:** This systematic literature review synthesizes 8 peer-reviewed studies (2019–2025) on AI literacy for elementary writing instruction, finding that AI integration efficiently supports writing practices and fosters creativity through multimodal application while creating nuanced approaches to writing assessment — alongside unresolved limitations for future research. ## Key Findings 1. **Eight studies reviewed.** The review synthesizes 8 peer-reviewed studies published between 2019 and 2025 focusing on AI literacy for writing instruction at the elementary level. 2. **AI supports writing practices efficiently.** Findings report that AI integration has efficient results in support of writing practices in elementary classrooms. 3. **Creativity through multimodality.** AI fosters creativity through multimodal application, enabling new forms of expression in young learners' writing. 4. **Nuanced assessment approaches.** AI creates nuanced approaches to writing assessment, though the review notes certain limitations remain for future research. ## Implications This review contributes to the growing literature on [[generative-ai]] and [[ai-literacy]] in [[k-12]] settings, focusing specifically on elementary writing. Its finding that multimodal AI tools can foster [[creativity]] in young writers counters the common framing of AI purely as an automation threat, while the attention to [[assessment]] reflects ongoing debates about how to evaluate writing in AI-integrated classrooms. For [[writing-education]], the small evidence base (8 studies) underscores how nascent this area is and the need for more empirical work, particularly around equitable access and the developmental appropriateness of AI tools for early writers. The review connects to [[llm]] discussions about scaffolding young students' interactions with [[generative-ai]] and to broader questions of [[educational-policy-ai]] for elementary classrooms. The integration of AI into elementary writing also raises [[equity]] considerations, echoing concerns about the [[equity]] and differential access across schools. ## Connected Concepts - [[ai-literacy]] - [[assessment]] - [[creativity]] - [[educational-policy-ai]] - [[equity]] - [[generative-ai]] - [[k-12]] - [[llm]] - [[writing-education]] ## Connected Articles - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review - [[llm-children-reading-story-generation]] — LLM children's story generation - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Creative writing with robots - [[mindcopilot-llm-co-writing]] — MindCopilot co-writing ## Citation Abedini, P. (2026). [*Rethinking Elementary Education's Writing Instruction in The Age of Generative AI: A Systematic Review*](https://osf.io/ztr2x). EdArXiv preprint. --- ## [ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education](https://edtechdev.github.io/aied/articles/elevate-genai-virtual-tutors/) > **Lorenzo Stacchio, Michele Giordano, Daniele Berardini, Primo Zingaretti, Emanuele Frontoni** — submitted 17 Jun 2026 **Lorenzo Stacchio, Michele Giordano, Daniele Berardini, Primo Zingaretti, Emanuele Frontoni** — submitted 17 Jun 2026 ELEVATE framework deploys GenAI-driven 3D avatar tutors on consumer-grade hardware with local-first execution model, emphasizing privacy, teacher governance, and equity in K-12 education. ## Key Contributions - ELEVATE framework deploys GenAI-driven 3D avatar tutors on consumer-grade hardware with local-first execution model, emphasizing privacy, teacher governance, and equity in K-12 education. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[child-safety-genai]] — Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework - [[gaze-informed-ai-children]] — Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education - [[mooc-to-maic]] — From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[lecturaagents-multi-agent-teaching]] — LecturaAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching ## Citation Lorenzo Stacchio, Michele Giordano, Daniele Berardini, Primo Zingaretti, Emanuele Frontoni (2026). [ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education](https://arxiv.org/abs/2606.30662). arXiv:2606.30662. submitted 17 Jun 2026 --- ## [Embodied Inquiry with AI as Facilitator: An Exploratory Case Study](https://edtechdev.github.io/aied/articles/embodied-inquiry-ai-facilitator-physics-2026/) > **Synthesis:** Tufino & Damiani (2026) explore where a language-based AI can stand within an inquiry activity without displacing embodied experience, using a Master's-level physics education course investigating the statics of fluids via the ISLE approach. In a two-phase design, students first built the buoyancy model with their own hands without AI; a purpose-configured AI assistant then facilitated applying the model to a new phenomenon. The paper discusses what a language-based facilitator cannot reach and the value of a design in which AI **complements** embodied inquiry rather than replacing it. The work connects to [[physics-education]], [[socratic-method]], and [[pedagogical-agent]] theory. ## Where AI Cannot Reach in Embodied Inquiry Generative AI is entering science education at a time when the embodied education community is asking what such systems cannot do. Rather than asking whether AI can understand the body, this case study asks where a language-based AI can stand within an inquiry activity without displacing embodied experience. ## Two-Phase ISLE Design University students in a Master's course in physics education investigated the statics of fluids following the ISLE (Investigative Science Learning Environment) approach in a two-phase design: students first built the buoyancy model with their own hands, without AI; a purpose-configured AI assistant then facilitated the application of the model to a new phenomenon. ## Findings The paper discusses what a language-based facilitator cannot reach and the value of a design in which AI complements embodied inquiry rather than replacing it — a nuanced contribution to [[pedagogical-agent]] and constructivist learning theory. ## Connected Concepts - [[physics-education]] - [[socratic-method]] - [[pedagogical-agent]] - [[generative-ai]] - [[higher-ed]] - [[professional-training]] - [[stem-education]] - [[socratic-ai-dialogue]] - [[simulation]] - [[teacher-role]] ## Connected Articles - [[multiagent-classroom-dual-process-physics-teachers-2026]] - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] - [[genai-ar-physics-simulation-prompt-2026]] - [[ai-acceptance-preservice-science-teachers-2026]] ## Citation Tufino, E., & Damiani, P. (2026). [*Embodied inquiry with AI as facilitator: An exploratory case study*](https://arxiv.org/abs/2607.21349). arXiv:2607.21349. --- ## [Designing for What Cannot Be Seen: Supporting Embodied String Learning for Musicians with Blindness and Low-Vision](https://edtechdev.github.io/aied/articles/embodied-string-learning-blindness-low-vision-musicians/) Bowed string performance depends on fine bodily coordination usually taught through visual demonstration, creating persistent barriers for musicians with blindness and low-vision (BLV). This design study worked with four advanced BLV string musicians and three instructors using practice-video analysis, lesson observation, and expert reflection to surface embodied, non-visual learning strategies. The work extends [[special-education]] and special-education-ai by centring disability-led, embodied design rather than retrofitted accessibility. It complements [[accessible-learning]] and [[neurodivergent-computing-students]] research on inclusive interfaces, and frames the instructor role through [[equity]] in one-on-one music training, with implications for [[higher-ed]] and pre-professional contexts where tactile and kinesthetic cues must substitute for demonstration. ## Connected Concepts - [[special-education]] - [[accessible-learning]] - [[equity]] - [[higher-ed]] ## Connected Articles - [[neurodivergent-computing-students]] ## Citation Shi Shi, Lingyun Chen, Zitao Zhang, Amanda R. Draper, et al. (2026). [Designing for What Cannot Be Seen: Supporting Embodied String Learning for Musicians with Blindness and Low-Vision](https://arxiv.org/abs/2607.18598). arXiv:2607.18598. arXiv:2607.18598 [cs.HC]. --- ## [Invisible Impact of Empathy on Behavioral Change: Isolating the Effect of Empathy in Long-term Physical Activity Coaching Chatbot Interactions](https://edtechdev.github.io/aied/articles/empathy-coaching-chatbot/) Siyan et al. (2026) conduct a carefully controlled experiment isolating the effect of empathetic language in LLM-powered physical activity coaching chatbots over a longitudinal deployment. While the empathy condition did not directly increase exercise behavior, it significantly improved users' sense of being understood, which in turn predicted sustained engagement with the coaching system. This finding has direct relevance to [[affective-tutoring]] research in education: AI tutors that express empathy may not directly boost learning outcomes, but may sustain engagement long enough for learning to occur. The work connects to [[personalized-learning]] system design by showing that affective features like empathy serve a relationship-maintenance function rather than a direct instructional one. The study also contributes to [[llm]]-based educational tool design by demonstrating that careful experimental isolation is needed to understand which conversational features actually drive outcomes. While the study's education implications are indirect (health coaching rather than classroom learning), the mechanism of empathy-sustained engagement transfers to [[student-experience]] in any long-term AI-mediated learning relationship, particularly in [[feedback-loop]] contexts where sustained interaction is critical. ## Connected Concepts - [[affective-tutoring]] - [[personalized-learning]] - [[llm]] - [[student-experience]] - [[feedback-loop]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[ecnuclaw-k12-personalized-companion]] — ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education - [[llm-diverse-explanations-programming]] — Exploring the Value of Diverse LLM Explanations in Introductory Programming - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue ## Citation Li Siyan, Kai-Hui Liang, Shopnil Shahriar, Yilin Ye, Shiyoh Goetsu, Wei-Wei Du, Masahiro Yoshida, Tsunayuki Ohwa, Xuhai Xu, Zhou Yu (2026). [Invisible Impact of Empathy on Behavioral Change: Isolating the Effect of Empathy in Long-term Physical Activity Coaching Chatbot Interactions](https://arxiv.org/abs/2606.26641). arXiv:2606.26641. cs.HC. --- ## [Engagement Assessment in Video Learning](https://edtechdev.github.io/aied/articles/engagement-assessment-video/) > **EduGage** (Leng et al., 2026) addresses a core challenge: in online/video-based learning, **learners must self-regulate** their engagement with instructional materials. > Sensor-based momentary assessment of engagement in self-guided video learning environments. ## The Engagement Problem in Video Learning **EduGage** (Leng et al., 2026) addresses a core challenge: in online/video-based learning, **learners must self-regulate** their engagement with instructional materials. ### Dimensions of Engagement | Dimension | Measurement | Relevance to Learning | |-----------|--------------|----------------------| | **Attentional** | Eye tracking, gaze patterns | Sustained focus on content | | **Emotional** | Facial expression, sentiment | Positive affect supports persistence | | **Cognitive** | Physiological signals, task performance | Deep processing vs. superficial viewing | ## Sensor-Based Momentary Assessment Traditional engagement measures: - **Post-hoc surveys:** Retrospective bias, low temporal resolution - **Self-reports:** Introspection difficulty, social desirability bias EduGage contributes: **Real-time sensor fusion** for momentary assessment during video learning. ### Technical Approach - **Sensors:** Webcam (facial analysis), interaction logs (pause, rewind, speed) - **Assessment:** Momentary (in-the-moment) vs. retrospective - **Feedback loop:** Real-time reflection prompts based on engagement state ## Connection to Adaptive Learning This enables **adaptive interventions** in video learning: 1. **Detect disengagement** (gaze diversion, prolonged pauses) 2. **Trigger scaffolds** (reflection prompt, content re-summarization) 3. **Close loop:** Learner reflects → re-engages → improved outcomes This aligns with [[adaptive-learning]] principles: real-time learner modeling → personalized intervention. ## Implications for ITS Intelligent tutoring systems increasingly include **video components** (e.g., worked examples, concept explanations). EduGage's approach enables: - **Multimodal engagement tracking** (cf. [[multimodal-ai-tutoring]], [[affective-tutoring]]) - **Just-in-time scaffolds** when engagement drops - **Self-regulated learning support** ([[self-regulated-learning]]) ## Connected Concepts - [[adaptive-learning]] - [[affective-tutoring]] - [[self-regulated-learning]] ## Connected Articles - [[multimodal-ai-tutoring]] ## Citation al, A.Z.L.E.E.Y.S.E. (2026). [*Engagement Assessment in Video Learning*](https://arxiv.org/abs/2605.01238) --- ## [From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning](https://edtechdev.github.io/aied/articles/engagement-forecasting-its/) ## Overview This paper tackles a core ITS challenge: predicting when students will disengage so tutors can intervene before it's too late. It introduces **engagement forecasting** as a supervised prediction task with two complementary targets: minutes practiced per week (effort) and new skills mastered per week (progress). **Benchmarking 15 predictors on 425 middle-school students:** - Feature-based models (regressions, decision trees, neural networks) reduce **MAE by 22–33%** compared to heuristic baselines - Percentile heuristics (adapted from prior behavioral domains) systematically **overpredict** — they assume past behavior continues, missing disengagement signals - Feature-based models better track actual practice trajectories week-to-week **Distinct predictive signatures for effort vs. progress:** - **Effort forecasting** driven primarily by **recent activity features** (how much the student practiced recently) - **Progress forecasting** depends more on **learner-state** and **content difficulty** signals (what the student knows and how hard the material is) **Human validation:** Semi-structured interviews with 8 college tutors confirmed that tutors reason differently about effort goals vs. progress goals, mirroring the model's feature importance patterns. This strengthens the case for practical deployment. ## Implications for Intelligent Tutoring Systems This work shifts ITS analytics from reactive to **predictive**. Instead of flagging disengagement after it happens, engagement forecasting enables: - **Proactive tutor-learner goal setting** with data-driven weekly targets - **Early warning systems** that surface at-risk students before they fall behind - **Differentiated intervention** based on whether effort or progress is the primary concern The finding that effort and progress have distinct predictive signatures is practically important. A student practicing diligently but struggling with difficult content needs different support than one who is simply not logging in. Current ITS dashboards often conflate these signals; engagement forecasting disentangles them. ## Connections to the ITS Research Landscape This paper extends the [[ai-tutor-effectiveness-review]] findings on what makes ITS effective by adding a temporal prediction layer. Where prior work evaluates whether tutoring works on average, engagement forecasting asks *when* it works and *for whom* — connecting to the personalized intervention paradigm in [[collaborative-ai-tutoring]]. The focus on middle-school students (N=425) aligns with the [[stanford-evidence-base-ai-k12-2026]], which calls for more rigorous K-12 efficacy studies. The EDM 2026 venue, combined with [[genai-tutor-engagement-patterns]], suggests engagement analytics is becoming a recognized subfield within educational data mining. ## Methodological Contribution The paper establishes a **reproducible benchmark** for engagement forecasting, with clearly defined prediction targets, a documented feature set, and public interaction log data. This is significant for the benchmark landscape in AIED, where many systems are evaluated on proprietary data with incomparable metrics. ## Connected Concepts - [[math-education]] - [[help-seeking]] - [[pedagogical-llm-training]] - [[transfer-of-learning]] - [[socratic-method]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[collaborative-ai-tutoring]] - [[stanford-evidence-base-ai-k12-2026]] - [[genai-tutor-engagement-patterns]] ## Citation Qiu, E. S., Thomas, D. R., Guo, B., Aleven, V., & Borchers, C. (2026). [From Heuristics to Analytics: Forecasting Effort and Progress in Online Learning.](https://arxiv.org/abs/2605.12788) arXiv:2605.12788. EDM 2026. --- ## [Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction](https://edtechdev.github.io/aied/articles/engagement-intensity-learner-modeling/) > Engagement intensity during AI ethics instruction serves as an effective learner-modeling signal for adaptive instruction; prior LLM experience influences engagement patterns. ## Synthesis This study investigates whether simple intake measures can serve as learner-modeling signals for adaptive AI ethics instruction. Drawing on data from 93 bioscience graduate and postdoctoral trainees enrolled in a required research ethics course, the authors compare three candidate intake features — self-reported LLM usage frequency, self-rated LLM familiarity, and prior AI education — against five baseline perception outcomes: accuracy trust, evaluation capability, complex-task trust, over-reliance risk, and training interest. The results position simple behavioral engagement measures as promising, lightweight inputs for profiling learners before instruction begins, with direct relevance for [[personalized-learning]] and [[adaptive-learning]] design. ## Key Findings - **Usage frequency** showed Holm-corrected associations with **all five** baseline perception outcomes, making it the most consistent intake signal of the three tested. - **Self-rated LLM familiarity** was associated with three of the five outcomes, serving as a secondary indicator. - **Prior AI education** (coursework or workshop attendance) was associated with **none** of the five outcomes, questioning its value as an intake measure. - The association pattern was **threshold-like at the lower end of the scale** rather than a uniform gradient across all five outcomes, and was most visible for training interest and accuracy trust. - The authors conclude that simple pre-instruction behavioral signals can inform lightweight intake profiling for adaptive AI ethics education, supporting [[ai-literacy]] development in graduate research training. ## Study Design & Method The study surveyed 93 bioscience graduate students and postdoctoral trainees enrolled in a required research ethics course before instruction began. Three candidate intake features were compared across five baseline perception outcomes using statistical models with Holm correction for multiple comparisons. The five outcomes were treated as facets of distinct constructs from prior work on trust calibration and AI literacy — accuracy trust, evaluation capability, complex-task trust, over-reliance risk, and training interest — rather than as indicators of a single latent dimension. ## Limitations The cross-sectional design does not establish directionality: engagement may shape perceptions, perceptions may shape engagement, or both may reflect upstream factors such as disciplinary norms or dispositional openness. Usage frequency was self-reported on a Never-to-Daily scale rather than observed behavior, and both predictors and outcomes relied on self-report, so response styles such as acquiescence or extremity bias could contribute to the associations. The authors also caution that the null result for prior AI education should be read carefully, since AI literacy spans multiple competencies and the coding of prior education was relatively coarse. ## Connected Concepts - [[personalized-learning]] - [[ai-literacy]] - [[adaptive-learning]] - [[affective-tutoring]] - [[bias-mitigation]] - [[affective-computing]] - [[prompt-engineering]] - [[knowledge-tracing]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics ## Citation Yongkyung Oh, Lynn Talton, Alex Bui (2026). [Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction](https://arxiv.org/abs/2606.18548). arXiv:2606.18548. --- ## [Enhancing creative writing with robot-LLM integration: The interplay of embodiment, AI creativity and user engagement](https://edtechdev.github.io/aied/articles/enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen/) > **Synthesis:** This study explores the impact of robot-LLM integration on collaborative creative writing, focusing on how embodiment and AI creativity influence creative output. With 150 undergraduate students across five conditions, results revealed that the Human-Robot (High-Creativity LLM) condition significantly enhanced originality, while Human-Human and Human-LLM collaborations excelled in imagery and voice. An 'embodiment paradox' was identified, where robot embodiment amplified creativity in high-creativity AI conditions but human collaboration remained superior in stylistic expression. ## Key Findings This study explores the impact of robot-LLM integration on collaborative creative writing, focusing on how embodiment and AI creativity influence creative output. With 150 undergraduate students across five conditions, results revealed that the Human-Robot (High-Creativity LLM) condition significantly enhanced originality, while Human-Human and Human-LLM collaborations excelled in imagery and voice. An 'embodiment paradox' was identified, where robot embodiment amplified creativity in high-creativity AI conditions but human collaboration remained superior in stylistic expression. ## Connected Concepts - [[collaborative-learning]] ## Connected Articles - [[humanlike-ai-collaborative-writing]] - [[ai-in-the-wild-college]] - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] - [[teacher-ai-teaming-five-levels]] ## Citation Liu, Y., & Song, Y. (2026). [*Enhancing creative writing with robot-LLM integration: The interplay of embodiment, AI creativity and user engagement*](https://doi.org/10.1111/bjet.70071). *British Journal of Educational Technology.* --- ## [Enhancing online learning outcomes through virtual companion AI: The role of identity anthropomorphism](https://edtechdev.github.io/aied/articles/enhancing-online-learning-outcomes-through-virtual-companion-ai-the-role-of-iden/) > **Synthesis:** Grounded in social presence theory, this study introduces the concept of identity anthropomorphism and adopts multimodal learning analytics (MMLA) combining questionnaires, EEG and eye tracking to examine its effects on learning outcomes. With 70 participants across three conditions, results indicated that identity-anthropomorphised AI significantly improved learning outcomes compared to non-anthropomorphised AI and performed comparably to human companionship. Social presence and positive emotions sequentially mediated this effect. ## Key Findings Grounded in social presence theory, this study introduces the concept of identity anthropomorphism and adopts multimodal learning analytics (MMLA) combining questionnaires, EEG and eye tracking to examine its effects on learning outcomes. With 70 participants across three conditions, results indicated that identity-anthropomorphised AI significantly improved learning outcomes compared to non-anthropomorphised AI and performed comparably to human companionship. Social presence and positive emotions sequentially mediated this effect. ## Connected Concepts - [[affective-computing]] - [[learning-analytics]] ## Connected Articles - [[multimodal-ai-feedback-learning]] - [[kutti-ai-voice-first-learning-companion]] - [[ai-in-the-wild-college]] - [[ai-assisted-learning-modes-eeg]] ## Citation Xia, T., Tan, M., & Guo, J. (2026). [*Enhancing online learning outcomes through virtual companion AI: The role of identity anthropomorphism*](https://doi.org/10.1111/bjet.70072). *British Journal of Educational Technology.* --- ## [Ordered Network Analysis of Epistemic Emotions during Collaborative Problem Solving](https://edtechdev.github.io/aied/articles/epistemic-emotions-collaborative-problem-solving/) Anindho, Venkatesha, Ocumpaugh and Blanchard apply Ordered Network Analysis to trace how epistemic emotions such as confusion and frustration persist and transition during co-situated collaborative problem solving. The work advances affect-aware learning analytics by modeling the temporal ordering of emotional states rather than static frequencies, informing when interventions should trigger in [[affective-tutoring]] systems and multimodal detectors like [[multimodal-affective-its-presentation]]. It grounds affect dynamics in [[collaborative-learning]] contexts, complements sensor-based approaches like [[affective-text-wearable-student-health]] and video-based [[engagement-assessment-video]]. ## Connected Concepts - [[affective-tutoring]] - [[collaborative-learning]] ## Connected Articles - [[multimodal-affective-its-presentation]] - [[affective-text-wearable-student-health]] - [[engagement-assessment-video]] ## Citation Sifatul Anindho, Videep Venkatesha, Jaclyn Ocumpaugh, Nathaniel Blanchard (2026). [Ordered Network Analysis of Epistemic Emotions during Collaborative Problem Solving](https://arxiv.org/abs/2607.23317). arXiv:2607.23317. --- ## [From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning](https://edtechdev.github.io/aied/articles/epistemic-proactivity-math/) Abdelghani, Kaiser & Murayama (2026) trace how middle and high school students' interactions with AI math tutors evolve over time, identifying a trajectory from superficial prompting ('tell me the answer') to what they term 'epistemic proactivity' — the active, self-directed pursuit of conceptual understanding through AI dialogue. This developmental framework is a significant contribution to [[ai-literacy]] research, as it suggests that productive AI use is not a binary skill but a capacity that matures through scaffolded practice. The temporal analysis reveals that students who receive metacognitive prompting show faster transitions to epistemic proactivity, directly linking to [[metacognition]] and [[self-regulated-learning]] theories. The study has implications for [[stem-education]] and [[k-12]] classrooms, where AI tools are increasingly used as math tutors. It challenges the assumption that students intuitively know how to learn with AI, arguing instead that deliberate scaffolding is essential — a finding that resonates across [[student-experience]] research in AI-mediated learning environments. ## Connected Concepts - [[math-education]] - [[ai-literacy]] - [[metacognition]] - [[self-regulated-learning]] - [[stem-education]] - [[k-12]] - [[student-experience]] - [[scaffolding]] ## Connected Articles - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[ai-powered-personalized-learning-elementary-fractions-2026]] ## Citation Rania Abdelghani, Peter Kaiser, Kou Murayama (2026). [From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning](https://arxiv.org/abs/2606.28472). arXiv:2606.28472. cs.CY. --- ## [Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning](https://edtechdev.github.io/aied/articles/ethical-ai-higher-ed-game-theory/) **Ethical AI Use in Higher Education: A Coordination Game Framework** provides a formal mechanism-level account of why policy statements alone fail to change student AI-use behavior. Reframing student AI use in assessments as a coordination problem — where individual choices depend on peer expectations and assessment design — the authors develop an evolutionary game-theoretic model capturing learning value, effort, perceived fairness, and transparency. The key finding is non-linear threshold dynamics: small, well-calibrated changes to reflective assessment incentives can trigger rapid cohort-wide shifts toward responsible AI use, while weak or misaligned incentives allow [[over-reliance]] and opportunistic practices to persist. This explains the common observation that institutional [[regulation]] and policy pronouncements have limited impact while modest assessment redesign — such as requiring students to reflect on their AI interactions — can have disproportionate effects. The framework complements [[genai-assessment-governance]] by providing the mathematical underpinning for why Restrict/Scaffold/Require stances have differential behavioral effects, and connects to [[academic-integrity]] research on how social norms shape [[generative-ai]] use among [[higher-ed]] students. The model supports [[institutional-change-framework-ai]] approaches that emphasize pedagogy-led governance over surveillance. ## Connected Concepts - [[over-reliance]] - [[regulation]] - [[academic-integrity]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[genai-assessment-governance]] - [[institutional-change-framework-ai]] ## Citation Ndidi Bianca Ogbo, Zhao Song, Shatha Ghareeb, The Anh Han (2026). [Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning](https://arxiv.org/abs/2605.27400). arXiv:2605.27400. arXiv preprint. --- ## [Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations](https://edtechdev.github.io/aied/articles/evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations/) While [[llm]]s now enable rapid generation of learning materials like [[generative-ai]], evaluating the pedagogical quality of these materials remains an open challenge. This paper proposes an automated assessment framework for evaluating interactivity in AI-generated explorable explanations — dynamic, learner-driven content that students can manipulate to discover concepts. The framework addresses the gap between content generation speed and quality assurance, providing metrics for [[formative-assessment]] of learning designs. This connects to [[learning-analytics]] approaches for understanding how students engage with AI-produced educational content in [[higher-ed]] settings. ## Key Findings - The paper introduces **EE-Eval**, an automated evaluation framework that formalizes interactivity as a finite space of learner-controllable states and transitions, represented as a **Finite State Machine (FSM)**. - By extracting FSMs from AI-generated explorable explanations, EE-Eval turns implicit interaction logic into an explicit, machine-interpretable graph that can be compared against an ideal FSM encoding pedagogical intent. - Evaluation combines **graph-based metrics with embedding-based comparison** of states, actions, and feedback to measure structural and semantic similarity. - Across thousands of generated explorable explanations spanning **127 concepts and 6 AI models**, EE-Eval consistently differentiated interaction quality beyond surface-level criteria such as functional correctness and visual quality. - EE-Eval exhibited **substantially stronger alignment with human judgments** of interactivity and pedagogical effectiveness than existing baselines, which largely focus on code executability or visual fidelity. ## Method in Brief Existing benchmarks give limited insight into dynamic interaction behaviors such as learner-controlled state transitions and context-sensitive system responses — the factors that critically shape learners' conceptual understanding. EE-Eval addresses this by framing interactivity as testable behavioral models rather than an emergent byproduct of [[llm]] generation. The resulting FSM comparison supports pedagogically grounded, actionable [[human-ai-collaboration]] in creating interactive educational content. ## Implications for AI in Education For educators and tool builders, EE-Eval offers a diagnostic lens: instead of asking only whether generated content runs correctly, one can ask whether the interaction logic a [[generative-ai]] system produced actually serves the intended learning goals. By externalizing interaction logic into an inspectable graph, the framework transforms evaluation into a reflective diagnostic tool for the increasingly common practice of generating [[active-learning]] materials with LLMs, supporting quality assurance at scale. ## Connected Concepts - [[llm]] - [[generative-ai]] - [[formative-assessment]] - [[learning-analytics]] - [[higher-ed]] ## Connected Articles - [[data-comics-for-education-evaluating-effectiveness-benefits-ethics]] — Data Comics for Education: Evaluating Effectiveness, Benefits, and the Ethics of AI-Assisted Creation - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning ## Citation Xiaozao Wang, Zhewei Wang, Hongyi Wen (2026). [Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations](https://arxiv.org/abs/2606.31012). arXiv:2606.31012. --- ## [Expert Cognition Dashboard: From Learning Analytics to Cognition Intelligence in AI-Driven Education](https://edtechdev.github.io/aied/articles/expert-cognition-dashboard/) **Annie Yuan (2026)**. arXiv preprint (cs.HC). ## Summary Current AI-driven educational systems primarily rely on behavioural analytics and performance metrics, lacking the ability to model expert cognition used to interpret learner development and identify misconceptions. This paper introduces the Expert Cognition Dashboard (ECD), a cognition-centred reporting infrastructure for AI Twin-driven education systems. ## Key Findings This paper proposes a paradigm shift from Learning Analytics to Cognition Intelligence, introducing the Expert Cognition Dashboard (ECD) as a three-layer cognitive middleware for AI-driven education. The ECD architecture models expert cognition — interpretation, identity cognition, value recognition, misconception patterns, and learning tension — across individual, class, and AI Twin expert levels. Rather than treating learner behaviors as raw signals, ECD transforms student interactions into interpretable cognition structures through AI Tutor analysis and dashboard aggregation. This advances the [[learning-analytics]] paradigm beyond simple visualization toward embedded reasoning infrastructure, positioning dashboards as foundational cognition infrastructures in AI-native education systems. The framework aligns with efforts in [[explainable-probabilistic-kt]] to make learner modeling interpretable, and extends [[adaptive-learning]] by incorporating expert-like cognitive reasoning into adaptation decisions. The cognition-layer approach complements [[llm-student-modeling-memory]] architectures that store longitudinal learner states, and intersects with [[teacher-student-agency-orchestration]] by providing the cognitive infrastructure for AI Twins to support adaptive intervention and personalized scaffolding. ## Connected Concepts - [[learning-analytics]] - [[adaptive-learning]] ## Connected Articles - [[explainable-probabilistic-kt]] - [[llm-student-modeling-memory]] - [[teacher-student-agency-orchestration]] ## Citation Annie Yuan (2026). [Expert Cognition Dashboard: From Learning Analytics to Cognition Intelligence in AI-Driven Education](https://arxiv.org/abs/2605.17263). arXiv:2605.17263. arXiv preprint (cs.HC). --- ## [Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning](https://edtechdev.github.io/aied/articles/explainable-probabilistic-kt/) ## Analysis This paper introduces **PLKT** (Probabilistic Logical Knowledge Tracing), which replaces deterministic vector embeddings with **beta-distributed probabilistic embeddings**, allowing explicit representation of uncertainty in each knowledge state. The framework applies **transparent logical operations** over probabilistic states to construct auditable reasoning paths — showing educators *which specific past actions* led to a prediction. This bridges [[neural-symbolic-knowledge-tracing]] paradigms and supports [[intelligent-tutoring]] by providing explainable predictions that can be inspected and trusted. Because every prediction can be traced back through transparent logical operations to the historical student actions that produced it, PLKT is oriented toward **accountability in learner modeling**: an instructor or system designer can audit why a particular knowledge-state estimate was reached, rather than treating the model as an opaque black box. This aligns with growing interest in explainable [[learning-analytics]] and in [[student-modeling]] approaches that surface their own reasoning. ## Key Findings - **PLKT** (Probabilistic Logical Knowledge Tracing) is a knowledge tracing approach built on **beta-distributed probabilistic embeddings**, which explicitly represent uncertainty in each knowledge state instead of a single deterministic point estimate. - The framework applies **transparent logical operations** over these probabilistic states, constructing **auditable reasoning paths** that link historical student actions to predictions. - The design targets explainability: reasoning paths show *which specific past actions* led to a prediction, supporting inspection and trust in model output. - By making uncertainty and reasoning explicit, PLKT bridges neural-symbolic knowledge tracing paradigms with practical [[knowledge-tracing]] needs in tutoring systems. - The paper is an arXiv preprint that presents the approach at a conceptual level; the public record provides abstract-level detail only. ## Implications for AI in Education For [[intelligent-tutoring]] and [[adaptive-learning]] systems, explainable predictions matter because educators and students need to trust the basis of automated decisions about what to practice next. By representing uncertainty explicitly and exposing the reasoning path behind each prediction, PLKT-style approaches support human oversight of learner models and could inform more transparent [[formative-assessment]] and feedback loops, where confidence-aware estimates of what a student knows are as important as the estimates themselves. ## Connected Concepts - [[intelligent-tutoring]] ## Connected Articles - [[neural-symbolic-knowledge-tracing]] ## Citation Wu et al. (2026). [Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning](https://arxiv.org/abs/2605.09369). arXiv:2605.09369. arXiv preprint. --- ## [Face value: How avatar identity shapes epistemic trust in AI-mediated learning](https://edtechdev.github.io/aied/articles/face-value-how-avatar-identity-shapes-epistemic-trust-in-ai-mediated-learning/) > **Synthesis:** Two experiments examined how avatar race, gender, and age shape trust in AI-mediated education. Study 1 (N=102) used a within-subjects laboratory design; Study 2 (N=294) adopted a between-subjects online design. Across both studies, social identity cues strongly influenced evaluations and behavior: White avatars and Asian male avatars in STEM contexts were rated more credible and competent, while older Black female avatars faced compounded penalties. Domain moderated these effects, with STEM tasks amplifying bias while reflective tasks attenuated it. ## Key Findings Two experiments examined how avatar race, gender, and age shape trust in AI-mediated education. Study 1 (N=102) used a within-subjects laboratory design; Study 2 (N=294) adopted a between-subjects online design. Across both studies, social identity cues strongly influenced evaluations and behavior: White avatars and Asian male avatars in STEM contexts were rated more credible and competent, while older Black female avatars faced compounded penalties. Domain moderated these effects, with STEM tasks amplifying bias while reflective tasks attenuated it. ## Connected Concepts - [[language-learning]] - [[intelligent-tutoring]] - [[socratic-method]] - [[math-education]] - [[higher-ed]] - [[accessible-learning]] - [[human-in-the-loop-ai]] - [[formative-assessment]] ## Connected Articles - [[social-robot-study-companions]] - [[multi-agent-llm-social-learning]] ## Citation Anthis, Z., & Kyriakidou-Zacharoudiou, A. (2026). [*Face value: How avatar identity shapes epistemic trust in AI-mediated learning*](https://doi.org/10.1016/j.caeai.2026.100610). *Computers and Education: Artificial Intelligence.* --- ## [Fair and explainable educational recommendations with a hybrid Graph-GRU framework](https://edtechdev.github.io/aied/articles/fair-explainable-edu-recommendations/) > **Synthesis:** Fair and explainable educational recommendations with a hybrid Graph-GRU framework ## Key Findings - The paper introduces the **Hybrid HKG-GRU framework**, combining heterogeneous knowledge-graph embeddings with sequential (GRU) modeling to capture both the relational structure of course materials and the temporal dynamics of learner interactions. - Evaluation on **Moodle LMS logs from 152 students, 59 resources, and approximately 150,000 interactions** achieved strong predictive performance with **HR@10 = 0.68 and MRR = 0.41**. - The framework integrates three responsibility-oriented contributions: **multi-objective training with Group Distributionally Robust Optimization (GroupDRO)** for fairness, **Maximum Marginal Relevance (MMR) reranking** to reshape exposure patterns, and built-in **model-centric explainability** through path-based and counterfactual analyses. - Results showed high intra-list diversity and moderate catalogue coverage, with **moderate counterfactual stability** for many learners (median CR@10 = 1.0), although catalogue-level popularity bias remained evident. - The work addresses **popularity bias and cold-start fairness** in educational recommenders, where students with limited participation histories risk receiving less reliable support while popular resources dominate recommendation lists. ## Study Design & Method The study targets a known failure of accuracy-focused recommenders: students with limited participation histories receive less reliable support, while highly popular resources may dominate lists and limit access to other useful materials. The hybrid architecture addresses this by modeling course-material structure and learner sequences jointly, then applying GroupDRO training to make performance robust across learner groups, MMR reranking to diversify exposure, and path-based and counterfactual explainability to make recommendations auditable. ## Implications for AI in Education For [[personalized-learning]] platforms, the work demonstrates that fairness, diversity, and explainability can be engineered into recommenders without sacrificing predictive accuracy, which matters for [[equity]] in resource-constrained and cold-start settings. The persistence of catalogue-level popularity bias even after intervention shows that bias mitigation is partial and requires ongoing measurement. The model-centric explanations are intended to support more transparent educational recommendation, consistent with [[human-in-the-loop-ai]] oversight of automated learning-resource navigation. ## Limitations - **Dataset scale and diversity:** evaluation rests on a single course with limited institutional and demographic diversity, constraining generalizability of the fairness and robustness findings. - **Scope of the fairness definition:** fairness is operationalized through participation-based cohorts (learner activity level) because the public Moodle dataset lacks achievement, prior knowledge, learning profiles, or demographic attributes; the results are therefore an audit of behavior across engagement levels rather than a full assessment of educational equity. As the authors note, a low-activity learner may be struggling, disengaged, or already familiar with the material, and activity level does not necessarily reflect ability or need. - **No human-centered bias auditing:** no instructor/student evaluation was conducted, so interpretability and trust claims remain model-centric and technical. - **Temporal modeling scope:** only short-range item→precedes→item edges within a single course are captured; multi-semester trajectories, cross-course relationships, and contextual shifts are not modeled. - **Persistence of systemic popularity bias:** substantial catalogue-level popularity bias remains despite GroupDRO and exposure-based regularization, as reflected in high Gini exposure metrics. ## Connected Concepts - [[human-in-the-loop-ai]] - [[personalized-learning]] - [[equity]] - [[adaptive-learning]] - [[knowledge-tracing]] - [[affective-computing]] - [[privacy]] - [[knowledge-graph]] ## Connected Articles - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[raza-farooq-aied-review-2020-2025]] — Review of Artificial Intelligence in Education from 2020 to 2025 - [[self-referential-l2-writing-llm-assessment]] — Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition Support Methods Using Deep Learning Audio-Video Processing Techniques - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains ## Citation Evangelista, E., & Bukhari, S. M. S. (2026). [*Fair and explainable educational recommendations with a hybrid Graph-GRU framework*](https://doi.org/10.1016/j.caeai.2026.100643). --- ## [Critical AI Tutors: Empower or Enslave?](https://edtechdev.github.io/aied/articles/favero-critical-ai-tutors-empower-enslave-2025/) > **Critical AI Tutors: Empower or Enslave?** — A position paper presented at the AIED 2025 workshop that issues a stark warning: unchecked use of AI tutors risks creating a generation of cognitively atrophied learners who have traded genuine understanding for the illusion of competence. Drawing on cognitive science and pedagogical theory, the authors argue that AI tutors without intentional guardrails lead to cognitive atrophy, loss of agency, emotional risks, and serious ethical concerns around privacy and academic integrity. The paper advocates for critically informed, transparent AI use that empowers rather than diminishes the learner, and calls for student perspectives to be centered in the design and deployment of AI tutoring systems. ## Key Findings This position paper occupies a critical counterpoint in the [[intelligent-tutoring]] literature. While much of the field focuses on optimizing AI tutor performance and learning gains, the authors argue that effectiveness metrics alone are dangerously insufficient — what matters equally are the cognitive and developmental *costs* that AI tutors may impose on learners. **Cognitive atrophy and the shortcut problem.** The central argument draws on [[cognitive-load-theory]] and the well-established finding that effortful cognitive processing is essential for durable learning. When [[intelligent-tutoring]] systems provide immediate, high-quality answers and solutions, they effectively short-circuit the very cognitive processes — struggling with problems, retrieving from memory, constructing explanations — that produce deep understanding. This phenomenon, which the authors term *cognitive atrophy*, mirrors concerns raised in the [[efficiency-gain-illusion-ai-overreliance]] and [[cognitive-offloading]] literatures: learners may feel more productive while learning less. **Loss of agency and dependency.** Beyond cognitive effects, the paper identifies a broader threat to learner agency. Prolonged reliance on AI tutors can produce [[over-reliance]] — a state where students lose confidence in their own reasoning abilities and become dependent on AI assistance even for tasks they could complete independently. This dependency dynamic connects to the [[correct-answer-trap-ai-tutor]] problem and the finding that [[over-reliance]] — students give up more quickly when AI help is available. **Emotional and well-being risks.** The authors highlight underexplored emotional dimensions: AI tutor interactions can erode self-efficacy when students compare themselves unfavorably to flawless AI outputs, contribute to [[ai-fatigue-academic-contexts]], and diminish the relational aspects of learning that [[teacher-role]] frameworks seek to preserve. **Ethical concerns.** The paper catalogs significant ethical risks including [[academic-integrity]] erosion, questionable [[privacy]] practices in educational AI systems, and the broader societal implications of [[ai-making-us-stupid]] — a provocative framing that challenges the edtech optimism narrative. ## Implications The paper's most important contribution is its call for *critical AI literacy* as a prerequisite for AI tutor deployment. Rather than banning AI tutors, the authors argue for what they call "critically informed use" — a framework where students, educators, and institutions actively interrogate AI tools rather than passively accepting them. This aligns with the [[critical-thinking]] tradition in education and extends it to the domain of [[ai-literacy-power-knowledge]]. For system designers, the paper implies that [[ai-tutor-safety-harms]] frameworks must expand beyond immediate harm prevention to include long-term developmental impacts. An AI tutor that never harms a student in the moment but gradually erodes their cognitive independence should be considered unsafe. This reframes [[pedagogical-safety]] as encompassing not just what the tutor does but what kind of learner it produces. The authors' emphasis on student voice — centering learner perspectives in design decisions — connects to the [[student-experience]] literature and the growing recognition that [[generative-ai-guardrails-harm-learning]] must be co-designed with the very populations they aim to protect. For educators and policymakers, this paper provides intellectual ammunition for resisting uncritical AI adoption. It suggests that [[regulation]] of educational AI should consider not just data privacy and bias but also cognitive and developmental outcomes — a position that resonates with calls in the [[ai-governance-education]] community for holistic evaluation frameworks. ## Connected Concepts - [[over-reliance]] - [[ai-governance-education]] - [[cognitive-load-theory]] - [[pedagogical-safety]] - [[regulation]] - [[student-experience]] ## Connected Articles - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[ai-making-us-stupid]] — Is AI making us stupid? - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[efficiency-gain-illusion-ai-overreliance]] — The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks - [[generative-ai-guardrails-harm-learning]] — Generative AI without guardrails can harm learning: Evidence from high school mathematics - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-fallibility-warning-help-seeking]] — Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System ## Citation Favero, L., Pérez-Ortiz, J.-A., Käser, T., & Oliver, N. (2025). [Critical AI Tutors: Empower or Enslave?](https://arxiv.org/abs/2507.06878). arXiv:2507.06878. --- ## [Feedback futures: beyond the limits of human and GenAI capacities](https://edtechdev.github.io/aied/articles/feedback-futures-genai/) ## Summary This editorial synthesises the seven papers of the AEHE 51(5) special issue on feedback in the age of generative AI. Its central claim: the question is **not whether GenAI feedback is useful, but how human and GenAI feedback can be combined to sustainably support learning rather than merely improve immediate performance**. Teacher and student feedback literacy are necessary but not sufficient — what is also required is deliberate pedagogic design, institutional conditions that prioritise learning over efficiency, and the purposeful development of human judgement that cannot be delegated to the tool. The issue's papers include four also ingested into this wiki: [[learner-centered-feedback-ai]], [[chatgpt-feedback-engagement-genai]], [[genai-teacher-feedback-comparison]], and [[care-full-feedback-genai]]. ## Five tensions in GenAI feedback The editors distill five recurring tensions from the special issue: 1. **Usefulness, trust, and uptake.** GenAI is valued for speed, clarity, and accessibility, while teacher feedback is trusted for contextual understanding, disciplinary expertise, accountability, and human connection. Uptake depends on experienced care, recognition, and presence — relational conditions GenAI struggles to reproduce. 2. **Immediate task achievement vs longer-term learning.** GenAI excels at helping students complete the task at hand but may orient them toward performance/avoidance goals rather than mastery — echoed in [[chatgpt-feedback-engagement-genai]]'s finding of weak metacognitive engagement and one-off interactions. 3. **Agency vs dependency.** Agency can be extended via iterative prompting and comparison, but may become "thinner" when students stay active at the level of interaction while ceding evaluative work to the system. 4. **Access, avoidance, and advantage.** GenAI is unlikely to benefit all students equally; non-users cite trustworthiness, preference for human feedback, and academic integrity. Access alone does not guarantee educative uptake. 5. **Teacher judgement and labour redistribution.** GenAI redistributes rather than removes teacher labour — teachers still assess accuracy, tone, relationality, and pedagogical value, and poorly designed tools can *increase* workload (see [[learner-centered-feedback-ai]]). ## Conditions of partnership - GenAI and human feedback are **complementary but only if the complementarity is specified** — via sequencing, comparison, editing, and governance across multiple sources and moments ("an ecology of configured encounters"). - The subtler risk is **displacement, not replacement**: the speed of GenAI feedback may gradually displace the slower, careful work of educational judgement without anyone consciously deciding it should. - **Human agency must be proactively maintained rather than assumed** — humans remain responsible for judging, adapting, and rejecting GenAI responses. - The editors warn against both "AI slop" and "human slop" (low-quality, standardised feedback from comment banks and templates), arguing GenAI may entrench the latter if ungoverned (see [[care-full-feedback-genai]]). ## Feedback literacy for GenAI contexts - In a GenAI-enhanced essay-writing study in the issue, **feedback literacy predicted performance while frequency of GenAI use, trust, and prior knowledge did not** — the learner's capacity to seek, evaluate, and act on feedback shapes GenAI's educational value. - Existing feedback literacy frameworks (Carless & Boud 2018; Molloy et al. 2020) must be **extended** with GenAI-specific capacities: evaluative judgement (GenAI output can look authoritative while being hallucinatory), metacognitive skill (monitoring when GenAI supports vs narrows vs substitutes thinking), and ethical decision-making (when and how GenAI use supports rather than substitutes for students' own intellectual work). - Teacher feedback literacy (design, relational, pragmatic dimensions; Carless & Winstone 2023) is less well theorised: design requires workable GenAI+human feedback workflows; relational oversight cuts both ways (teacher feedback is often perceived as more negative/risky than GenAI — see [[genai-teacher-feedback-comparison]]); pragmatically, what must remain human-led is not only connection but **accountable judgement**. ## Governance and assessment design - Policy cannot be purely top-down: acceptable GenAI support varies across courses and disciplines, and ambiguous institutional direction pushes responsibility down to individual teachers, individualising the challenge. - Assessment reform should make learning processes visible rather than treating final products as direct evidence; institutions must be deliberate about when GenAI use is itself part of what is being assessed. - Research should move beyond self-report toward in-situ methods (think-aloud, trace/log data, stimulated recall) and toward how learners **orchestrate feedback across sources** (human, GenAI, artefactual) that differ in credibility and value. Equity research should track not just tool access but educationally productive use. ## Connected Concepts - [[ai-literacy]] - [[equity-in-ai-education]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[ai-education]] - [[generative-ai]] - [[rag]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative a... - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers' practices and perceptions - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform ## Citation Zhan, Y., Wood, J., Carless, D., & Yan, Z. (2026). Feedback futures: beyond the limits of human and GenAI capacities. [*Assessment & Evaluation in Higher Education*](https://doi.org/10.1080/02602938.2026.2672830), 51(5), 811–820 --- ## [Principled AI Education Framework](https://edtechdev.github.io/aied/articles/finkelstein-principled-ai-education-2025/) > **Principled AI Education Framework** — A principled way to think about AI in education: guidance for educators and policy makers on action based on goals, models of human learning, and use of technologies. Rather than focusing solely on the promise and peril of AI or its immediate implementation, this framework advances a third path — connecting broad educational goals to actionable practices through a set of explicit, scholarship-grounded principles. **Noah Finkelstein**, University of Colorado, Boulder — Drafted July 2025, updated July 2026. ## Key Findings Finkelstein argues that the critical question facing education in the AI era is: *what skills, habits of mind, and practices must we maintain as within the human purview to preserve our humanity and societies?* He identifies three essential educational objectives for the modern era: **discernment** (the ability to ask appropriate questions, validate answers, and act on results), **empathy** (understanding and sharing perspectives of others, underpinning collaboration), and **sense of self** (developing agency, purpose, and belonging within a field or cultural system). The framework organizes principles around three interconnected dimensions: **Goals of Education.** Three broad classes of goals span why educational systems exist: (1) developing individuals through higher-order cognitive functions — reasoning, argumentation, communication, metacognition, empathy, and identity; (2) societal infrastructure — democracies depend on an educated populace for basic skills and common culture; (3) supporting disciplines and professional fields — education prepares students to contribute to and transform academic and professional communities. Current AI discourse often conflates these goals or reduces education to performance metrics alone. **Roles of Educators, Learners, and Technologies.** The framework clarifies who does what: educators shape goals, design learning environments, and model professional practices; learners develop skills, identities, and capacities; technologies play supporting roles that must be deliberately chosen. A core principle is that technology should augment rather than displace human capacities — when AI bypasses student thinking (e.g., generating essays or solving problems that students should grapple with), it undermines the very goals education exists to serve. **Learning Practices.** Finkelstein maps principles onto specific educational activities: designing curricula (principle 2.1 — align technology use with stated learning goals), structuring practice and feedback (principle 2.2 — ascertain which practices AI assists versus circumvents), synthesizing across courses (principle 2.3 — support or block cross-disciplinary integration), collaborative co-design of learning environments with students (principle 2.4), and evaluating educators and systems (principle 2.5 — use technology for formative feedback and evidence collection). The paper includes concrete scenarios illustrating how multiple principles interact: ubiquitous AI access without training risks reducing education to performance without learning; attempting to ban AI ("back to blue books") sends contradictory messages about professional practice; technology-enhanced skills development can help but risks exacerbating equity gaps if it displaces human interaction; and having students train or validate AI systems in their subject domain develops metacognition, discernment, and understanding of AI limitations. ## Implications for AI in Education Finkelstein's framework directly challenges the prevailing discourse that oscillates between uncritical AI adoption and reactionary prohibition. By grounding decisions in explicit principles derived from decades of learning sciences scholarship, educators and policy makers gain a stable foundation for action even as AI technologies evolve rapidly. The framework is deliberately flexible — designed for varied educational contexts across disciplines and institution types while maintaining fidelity to enduring educational values. The emphasis on **discernment**, **empathy**, and **sense of self** as non-negotiable educational outcomes has profound implications for AI tool design. Systems that optimize for immediate task completion at the expense of student reflection, that reduce collaborative learning to isolated human-AI interaction, or that erode students' sense of belonging and agency fail the principled test — regardless of their technical sophistication. This aligns with findings from [[transfer-of-learning]] showing that AI-assisted performance gains frequently fail to transfer to unassisted contexts, and with [[desirable-difficulties]] research demonstrating that easier practice can harm long-term retention. The framework also speaks to the growing evidence gap documented in [[stanford-evidence-base-ai-k12-2026]]: without principled foundations, the rush to implement AI in education proceeds without adequate causal evidence of effectiveness. Finkelstein's approach provides a normative compass — even where empirical evidence is thin, principles grounded in learning science can guide responsible implementation while research catches up. For course design, the framework connects directly to [[scaffolding]] practices and the [[zone-of-proximal-development]]: AI tools should provide support calibrated to learner readiness, not do the work for students. It also intersects with [[ai-metacognition-stem-review]] findings on how AI can either support or undermine metacognitive development depending on implementation choices. ## Connected Concepts - [[ai-education]] - [[cognitive-load-theory]] - [[desirable-difficulties]] - [[generative-ai]] - [[metacognition]] - [[scaffolding]] - [[zone-of-proximal-development]] - [[ai-literacy]] - [[curriculum-design]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[transfer-of-learning]] — AI Learning Transfer - [[ai-metacognition-stem-review]] — AI Tools Scaffolding Metacognition in STEM - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries ## Citation Finkelstein, N. (2025). [*A principled way to think about AI in education: guidance for educators and policy makers on action based on goals, models of human learning, and use of technologies*](https://arxiv.org/abs/2510.01467). arXiv:2510.01467. --- ## [Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education](https://edtechdev.github.io/aied/articles/flowcode-ai-creative-coding/) Building upon found examples is a popular way people learn to code, especially in creative coding communities where sharing projects and remixing are common practices. But effectively doing so requires being able to 1) understand how existing code works, and 2) extend it by writing code that implements your own ideas, practices that can be challenging for new creative coders. We explored how to support these two processes through the design of Flowcode, a creative coding programming environment that integrates a flowchart for visualizing code structure and a chat interface tailored to support learning to code over vibe coding. We share how we iterated on the design of Flowcode over two studies with new creative coders, reflecting on the roles visualization and friction may play in enabling productive AI-use in computing education. Flowcode pairs a code-structure flowchart with a learning-oriented (not vibe-coding) chat to help novice creative coders understand and extend found examples; two studies show visualization and productive friction can steer AI use toward learning rather than bypass. This work connects to core wiki themes: [[scaffolding]] [[generative-ai]] [[feedback-loop]] [[active-learning]] [[student-experience]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[scaffolding]] - [[generative-ai]] - [[feedback-loop]] - [[active-learning]] - [[student-experience]] ## Connected Articles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[llm-diverse-explanations-programming]] — Exploring the Value of Diverse LLM Explanations in Introductory Programming - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata ## Citation Tseng, Seoror, Adda, Factor, Darabi, Matschke, Fu, Lin, Maram & Sinha (2026). [Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education](https://arxiv.org/abs/2607.06721). arXiv:2607.06721. arXiv preprint. --- ## [Adoption-Ready Project-Based Learning for Computing Education: The FORAP Framework and a Multi-Scale Project Portfolio](https://edtechdev.github.io/aied/articles/forap-pjbl-computing-education/) Presents FORAP (Framework for Organizing Reusable and Adaptable Project-Based Learning projects) and a portfolio of 14 adoption-ready PjBL packages for computing education. The framework addresses the gap between PjBL's known benefits and its slow classroom adoption by making projects reusable, adaptable, and scalable across contexts. Anchors in [[active-learning]] and [[active-learning]] practice for [[k-12]] and [[higher-ed]], connecting to [[cs-education]] and [[stem-education]] implementation. It complements [[scaffolding]] approaches by providing structures teachers can adopt without rebuilding from scratch, lowering barriers to [[teacher-role]] adoption of hands-on computing pedagogy. ## Connected Concepts - [[active-learning]] - [[active-learning]] - [[k-12]] - [[higher-ed]] - [[cs-education]] - [[stem-education]] - [[scaffolding]] - [[teacher-role]] ## Connected Articles - [[visual-query-tracer-declarative-logic-learning]] — Evaluating a Visual Query Tracer and Builder for Learning Declarative Logic Programming - [[concept-catalyst-engineering-scaffolds]] — Creating Learning Scaffolds for Engineering Design Using Concept Catalyst - [[q-learning-lab-rl-teaching]] — Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis - [[llm-design-problems-hot-pjbl]] — LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[human-ai-co-mentorship]] — Human-AI Co-Mentorship in Project-Based Learning: A Case Study in Financial Forecasting ## Citation Ahmad D. Suleiman, Jan DeWaters, David C. Shepherd, Turgay Korkmaz, Faraz Hussain, Yu Liu, Daqing Hou (2026). [Adoption-Ready Project-Based Learning for Computing Education: The FORAP Framework and a Multi-Scale Project Portfolio](https://arxiv.org/abs/2607.11129). arXiv:2607.11129. arXiv preprint. --- ## [Trust-utility gap in introductory physics education: Students' adoption, domain-specific skepticism, and preferences for AI integration](https://edtechdev.github.io/aied/articles/fouad-bentley-trust-utility-gap-physics-2026/) > **Synthesis:** Fouad & Bentley (2026) survey 81 introductory physics students and find a striking 50-percentage-point trust-utility gap: 91% use AI for coursework but only 41% trust AI physics explanations — evidence of domain-calibrated skepticism, not uncritical adoption. Students spontaneously identified where AI fails in physics (visual-spatial reasoning, circuits, abstract reasoning), aligning with known benchmarks. 65% prefer optional over mandatory AI integration. A mixed-methods survey of 81 introductory physics undergraduates reveals a 50-point trust-utility gap: 91% [95% CI: 83%–96%] use AI for physics coursework, but only 41% [95% CI: 30%–52%] trust AI-generated physics explanations. Thematic analysis identified eight themes; the most distinctive finding was that 40% of qualitative respondents spontaneously articulated where AI fails — visual-spatial reasoning, circuit analysis, and abstract physical reasoning — aligning with known benchmarks. Students show domain-calibrated skepticism rather than uncritical adoption, and 65% prefer optional AI integration. - 91% use AI for physics, only 41% trust it — 50-point trust-utility gap - Students show domain-specific skepticism, not naive acceptance - 40% spontaneously identified AI failure modes: visual-spatial reasoning, circuits, abstract physics - Student trust calibration tracks AI competence boundaries (consistent with Kortemeyer benchmarks) - 65% prefer optional over mandatory AI integration - Verification gap identified: self-reported vs actual verification behavior ## Connected Concepts - [[physics-education]] - [[ai-literacy]] - [[trust-calibration]] - [[student-experience]] - [[higher-ed]] - [[stem-education]] ## Connected Articles - [[becker-chatgpt-typology-physics-2026]] - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] ## Citation Fouad, E., & Bentley, I. (2026). [Trust-utility gap in introductory physics education: Students' adoption, domain-specific skepticism, and preferences for AI integration](https://doi.org/10.1103/bdl1-6bn4). --- ## [FOXGLOVE: Comparing Goal-Oriented Writing Feedback from Experts and LLMs](https://edtechdev.github.io/aied/articles/foxglove-writing-feedback-experts-llms/) Introduces **FOXGLOVE**, a dataset of 696 feedback comments by trained writing instructors on 69 twelfth-grade argumentative essays, paired with 1,644 comments from four frontier LLMs — totaling 2,340 comments with expert quality ratings. Provides the first systematic comparison of LLM and expert feedback on three pedagogically critical dimensions: **goal-orientation, anchoring to specific sentences, and prioritization**. **Key findings:** Instructors and LLMs distribute feedback similarly across revision goals and essay positions, but diverge significantly on which specific sentences receive feedback. Models write more complex feedback and use fewer questions than human instructors. LLM feedback receives higher quality ratings on most dimensions — but much of this advantage is attributable to lengthier comments inflating perceived quality. This work directly informs the design of [[formative-assessment|AI writing feedback systems]], highlighting the need to evaluate [[ai-feedback-quality|feedback quality]] beyond surface-level ratings and to consider pedagogical factors like [[writing-education|feedback anchoring and prioritization]]. Relevant to both [[k-12|secondary]] and [[higher-ed|higher education]] writing instruction. ## Connected Concepts - [[formative-assessment]] - [[ai-feedback-quality]] - [[writing-education]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[icle-plus-plus-essay-scoring]] — ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[cyberscholar-genai-writing-feedback]] — Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models ## Citation Liu, Y., Song, Y., Gallagher, J., Sterman, S., & August, T. (2026). FOXGLOVE: Understanding Goal-Oriented and Anchored Writing Feedback from Experts and LLMs on Argumentative Essays. [arXiv:2606.06271](https://arxiv.org/abs/2606.06271). --- ## [Framing the 5% Problem: Teachers'' Perspectives on Persistence in Educational Technology](https://edtechdev.github.io/aied/articles/framing-5-percent-problem-teachers-persistence/) # Framing the 5% Problem: Teachers' Perspectives on Persistence in Educational Technology Borchers (2026) reports on a 90-minute participatory design workshop with 12 U.S. middle school mathematics teachers using i-Ready Math weekly. Thematic analysis identified four recurring dimensions of low student persistence: motivation and buy-in, cognitive roadblocks, resilience under challenge, and contextual barriers. Teachers emphasized the need to identify where students become stuck and recognize silent disengagement, prioritizing support for diagnosis and timely instructional response over aggregate usage metrics. This reframes the '5% problem' from an analytics gap to a situated instructional challenge, suggesting that [[teacher-role]] and [[teacher-ai-adoption-confidence]] must be centered when designing [[adaptive-learning]] systems for [[k-12]] contexts. The work connects to [[student-experience]] research and engagement-metrics by arguing that teacher-facing systems supporting interpretation of persistence through both cognitive and contextual evidence are needed. ## Connected Concepts - [[teacher-role]] - [[adaptive-learning]] - [[k-12]] - [[student-experience]] ## Connected Articles - [[teacher-ai-adoption-confidence]] ## Citation Conrad Borchers (2026). [Framing the 5% Problem: Teachers' Perspectives on Persistence in Educational Technology](https://arxiv.org/abs/2606.22294). arXiv:2606.22294. EC-TEL 2026 --- ## [From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments](https://edtechdev.github.io/aied/articles/from-answer-generators-to-reasoning-facilitators-ai-tutors/) The rapid integration of [[llm]]s into [[intelligent-tutoring]] threatens to reduce mathematical learning to mere answer generation. This paper presents a design framework for AI tutors that act as reasoning facilitators rather than answer generators, specifically targeting high-stakes exam preparation environments. Through a mixed-methods study of junior-high students preparing for the Zhongkao exam, the authors find that students actively resist traditional Socratic dialogue under time pressure and repurpose "answer-first" shortcuts as diagnostic checkpoints, and that features such as layered worked examples, step-linked visual grounding, and metacognitive scaffolding lower the interaction cost of reasoning repair. The framework provides concrete guidelines for designing [[student-experience]] patterns that prioritize deep understanding over superficial completion in [[k-12]] mathematics. ## Key Findings - The paper combines a **generative study, usability analysis, and 12-participant field deployment** of **AITutor**, an interactive system that translates theoretical pedagogical mechanisms into concrete user interface features for junior-high students preparing for high-stakes exams (Zhongkao). - Mixed-methods triangulation of **7,379 telemetry events, 8 contextual observations, and 10 interviews** revealed that students actively resist traditional Socratic dialogue under time pressure, repurposing "answer-first" shortcuts as vital diagnostic checkpoints. - Features like **layered worked examples, step-linked visual grounding, and metacognitive scaffolding** lowered the interaction cost of reasoning repair. - Design implications include verifying that generated methods belong to the junior-high syllabus (blocking advanced vector-based or calculus methods students cannot use in exams), **dynamic geometry coordination** (auto-highlighting auxiliary lines on the diagram synchronously with textual steps), and **step-specific follow-up buttons** ("Explain this step," "Simpler method") to minimize interaction friction. - The authors also propose **automated wrong-book generation**: segmenting captured problems by knowledge point into a delayed-retrieval review list, transforming immediate transfer tasks into spaced weekend practice. ## The Reasoning-Centered Product Loop The study contributes a broader framework for educational AI called the **Reasoning-Centered Product Loop**, organized around orienting learners' cognitive investment — making answer access an entry point into reasoning rather than an endpoint — and visualizing to coordinate mental models across representations. Its goal is to structurally support the inspection, local repair, curriculum verification, and delayed retrieval of mathematical reasoning "in the wild." ## Implications for AI in Education The findings push back on the assumption that withholding answers is always the right tutoring strategy: in time-pressured settings, the final answer can help students decide whether to invest effort in self-explanation, error search, or full solution reading. The design question is how to make answer access an entry point into reasoning rather than an endpoint — a principle that generalizes beyond high-stakes exam preparation to [[scaffolding]]-oriented [[ai-tutoring]] design more broadly. ## Connected Concepts - [[llm]] - [[intelligent-tutoring]] - [[scaffolding]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations]] — Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[data-comics-for-education-evaluating-effectiveness-benefits-ethics]] — Data Comics for Education: Evaluating Effectiveness, Benefits, and the Ethics of AI-Assisted Creation - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI ## Citation Yuming Feng, Yuan Tian, Erica Zhao (2026). [From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments](https://arxiv.org/abs/2607.01692). arXiv:2607.01692. --- ## [From emotion regulation to academic success: A self-determination theory-based emotional agent-mediated approach](https://edtechdev.github.io/aied/articles/from-emotion-regulation-to-academic-success-a-self-determination-theory-based-em/) > **Synthesis:** Emotion regulation has been recognized as a key factor affecting students' academic success. This study proposed a self-determination theory (SDT)-based emotional agent framework, implementing an emotional agent (EmoAgent) capable of proactively detecting students' emotional states and providing emotional regulation strategies. An 8-week quasi-experiment with 173 sixth graders found that the SDT-based emotional agent-mediated approach significantly outperformed the conventional agent approach in academic achievement, engaged students in positive emotional experiences, and moderated the negative effect of negative emotions on academic success. ## Key Findings Emotion regulation has been recognized as a key factor affecting students' academic success. This study proposed a self-determination theory (SDT)-based emotional agent framework, implementing an emotional agent (EmoAgent) capable of proactively detecting students' emotional states and providing emotional regulation strategies. An 8-week quasi-experiment with 173 sixth graders found that the SDT-based emotional agent-mediated approach significantly outperformed the conventional agent approach in academic achievement, engaged students in positive emotional experiences, and moderated the negative effect of negative emotions on academic success. ## Connected Concepts - [[affective-computing]] - [[agentic-ai]] ## Connected Articles - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] - [[ai-in-the-wild-college]] ## Citation Zheng, X.-L., Lyu, Z.-Y., Wang, S.-Y., Wang, F., Kong, X., Hwang, G.-J., et al. (2026). [*From emotion regulation to academic success: A self-determination theory-based emotional agent-mediated approach*](https://doi.org/10.1111/bjet.70083). *British Journal of Educational Technology.* --- ## [A Guiding Framework for K-12 Teachers in Creating AI-powered Learning Technologies through Vibe Coding](https://edtechdev.github.io/aied/articles/gaide-vibe-coding-k12-teachers/) Large language models generate code from natural language prompts, enabling vibe coding, which allows non-programmers to develop computational solutions. Vibe coding for teachers amplifies the teachers-as-designers paradigm, improving technology integration while fostering AI literacy. However, structured guidance on supporting this process is lacking. We propose GAIDE (A Guiding Framework for AI-Integrated Design for Educators), a framework that supports K-12 teachers in creating AI-powered learning technologies through vibe coding. The initial framework, built on Design Thinking and INTERACT, was validated through a CORDTRA interaction analysis of three teachers and four faculty mentors in an eight-week workshop to derive the final framework. Additionally, the qualitative analysis of pre- and post-interviews found an enhancement of teachers' AI literacy. Findings highlight the potential of learning-by-creating for professional development. GAIDE, a Design-Thinking-based framework, supports K-12 teachers (non-programmers) in building AI-powered learning tools via vibe coding; an eight-week workshop raised participating teachers' AI literacy, showing learning-by-creating as a professional-development model. This work connects to core wiki themes: [[k-12]] [[teacher-role]] [[ai-literacy]] [[generative-ai]] [[faculty-development]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[k-12]] - [[teacher-role]] - [[ai-literacy]] - [[generative-ai]] - [[faculty-development]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[llm-cultural-relevance-k12]] — LLMs for Culturally Relevant K-12 Pedagogy - [[concept-catalyst-k12-teacher]] — Concept Catalyst: Exploring Scrutable Interfaces to Structure K-12 Teacher Interactions with Generative AI - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators ## Citation Song, Choi, Kim, Kim, Weisberg & Moon (2026). [A Guiding Framework for K-12 Teachers in Creating AI-powered Learning Technologies through Vibe Coding](https://arxiv.org/abs/2607.05406). arXiv:2607.05406. arXiv preprint. --- ## [Game-Based and Gamified Robotics Education: A Comparative Systematic Review and Design Guidelines](https://edtechdev.github.io/aied/articles/game-based-gamified-robotics-education-review-2026/) > **Synthesis:** Mubarrat, Shao, and Min (2026) present the first PRISMA-aligned systematic review and comparative synthesis of game-based learning (GBL) and gamification in robotics education. Analyzing 95 studies from 12,485 records across four databases (2014–2025), they coded each study's approach, learning context, skill level, modality, pedagogy, and outcomes (κ = .918). Three patterns emerged: (1) approach–context–pedagogy coupling (GBL more prevalent in informal settings while gamification dominated formal classrooms and favored project-based learning); (2) an emphasis on introductory programming and modular kits with limited adoption of advanced software; and (3) a comparative synthesis of outcomes leading to design guidelines. ## Key Findings - Robotics education fosters **computational thinking, creativity, and problem solving** but remains challenging due to technical complexity; GBL and gamification offer engagement benefits, yet their comparative impact was previously unclear. - This is the **first PRISMA-aligned systematic review** comparing GBL and gamification in robotics education, analyzing **95 studies from 12,485 records** across four databases (2014–2025). - **Pattern 1 — approach–context–pedagogy coupling:** GBL is more prevalent in informal settings, while **gamification dominated formal classrooms** (p < .001) and favored **project-based learning** (p = .009). - **Pattern 2 — emphasis on introductory programming and modular kits**, with limited adoption of advanced software (~17%) and advanced hardware. - The review produced **comparative design guidelines** for applying GBL vs. gamification in robotics education based on context and pedagogy. ## Study Design & Method This is a **PRISMA-aligned systematic literature review** with a comparative synthesis. The authors screened 12,485 records across four databases (2014–2025) and analyzed 95 studies of game-based learning and gamification in robotics education. Each study was coded for approach (GBL vs. gamification), learning context, skill level, modality, pedagogy, and outcomes, with high inter-coder reliability (Cohen's κ = .918). Statistical comparisons examined the coupling between approach, context, and pedagogy, and the relative emphasis on skill levels and tool adoption. ## Implications for AI in Education The review provides evidence-based guidance for using [[game-based-learning]] and [[gamification]] in [[educational-robotics]]. It shows that GBL suits informal settings while gamification works well in formal classrooms and supports [[project-based-learning]], helping educators choose the right engagement strategy for their context. It connects to [[computational-thinking]], [[programming-education]], and both [[k-12]] and [[higher-ed]] robotics teaching, and highlights that robotics education emphasizes introductory programming and modular kits, with room for more advanced software adoption. ## Limitations As a systematic review, its conclusions depend on the quality and reporting of the 95 included studies; heterogeneous methods and outcome measures across studies complicate direct comparison. The 2014–2025 scope predates some recent advances in LLM-powered robotics education. The review focuses on GBL/gamification, so robotics education outside these approaches is not the focus. ## Connected Concepts - [[educational-robotics]] - [[game-based-learning]] - [[gamification]] - [[project-based-learning]] - [[computational-thinking]] - [[programming-education]] - [[k-12]] ## Connected Articles - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks: Project-Based Robotics - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[computational-thinking-educational-robotics-secondary-2026]] — Computational Thinking and Educational Robotics ## Citation Mubarrat, S. T., Shao, T., & Min, B.-C. (2026). *Game-based and gamified robotics education: A comparative systematic review and design guidelines*. arXiv:2601.22199. doi:10.1145/3772318.3791338. --- ## [Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration](https://edtechdev.github.io/aied/articles/gaze-informed-ai-children/) > **Zekun Wu, Man Su, Huiyong Li, Tomohiro Nagashima, Anna Maria Feit** — submitted 1 Jul 2026 **Zekun Wu, Man Su, Huiyong Li, Tomohiro Nagashima, Anna Maria Feit** — submitted 1 Jul 2026 Ollie, a gaze-informed AI assistant for children's picture exploration, uses eye-tracking to trigger LLM narrative descriptions; within-subject experiment shows gaze-informed assistance keeps children's attention longer and guides exploration effectively. ## Key Contributions - Ollie, a gaze-informed AI assistant for children's picture exploration, uses eye-tracking to trigger LLM narrative descriptions; within-subject experiment shows gaze-informed assistance keeps children's attention longer and guides exploration effectively. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[child-safety-genai]] — Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[mixed-reality-engineering-learning]] — Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning ## Citation Zekun Wu, Man Su, Huiyong Li, Tomohiro Nagashima, Anna Maria Feit (2026). [Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration](https://arxiv.org/abs/2607.00445). arXiv:2607.00445. submitted 1 Jul 2026 --- ## [Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS)](https://edtechdev.github.io/aied/articles/genai-academic-search-workshop/) **Yifan Liu, Jaime Arguello, Orland Hoeber, Chang Liu et al.** — cs.IR, cs.AI, cs.HC This report summarizes the CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS), which examined how GenAI is reshaping academic search systems and research practices. Three thematic clusters emerged: foundations and principles (guiding theories, design principles for human-centered GenAI-enhanced search), applications and opportunities, and search-as-learning — the idea that academic search systems should foster higher-order cognitive processes including synthesis, critical evaluation, and knowledge construction. The workshop emphasized transparency, credibility, research integrity, and long-term scholarly needs. The 'search-as-learning' theme is most directly relevant to [[ai-literacy]] and [[ai-education]] domains, as it reframes academic search not merely as information retrieval but as a learning activity. This connects to [[formative-assessment]] and [[self-regulated-learning]] research exploring how AI-mediated research tools shape student learning processes. ## Connected Concepts - [[ai-literacy]] - [[ai-education]] - [[formative-assessment]] - [[self-regulated-learning]] ## Connected Articles - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[reshaping-cs-education-genai]] — Reshaping Undergraduate Computer Science Education in the Generative AI Era ## Citation Liu, Y., Arguello, J., Hoeber, O., et al. (2026). [*Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS)*](https://arxiv.org/abs/2606.08936). arXiv:2606.08936. --- ## [From Prompt to Embodied Simulation: Using Generative AI to Create AR Physics Learning Tools](https://edtechdev.github.io/aied/articles/genai-ar-physics-simulation-prompt-2026/) > **Synthesis:** Levy et al. (2026) show how a structured natural-language prompt can generate a browser-based, hand-controlled **augmented-reality (AR) physics simulation** — spread your thumb and index finger and a virtual lamp changes color — and describe its use in an introductory physics class. Computer simulations have a long record of supporting physics learning by making abstract concepts interactive, and generative AI now lowers the barrier to creating customized, embodied, interactive simulations. The work connects to [[physics-education]], [[generative-ai]], and [[simulation]] and illustrates the growing role of [[prompt-engineering]] in educational content creation. ## Lowering the Barrier to Embodied Simulation Computer simulations have a long and well-documented record of supporting physics learning, making abstract physical concepts interactive and inviting students to play with parameters. This paper shows how a structured natural-language prompt can generate a browser-based, hand-controlled AR physics simulation. ## The AR Simulation Spread your thumb and index finger in the air, and a virtual lamp in the room changes color — a browser-based, hand-controlled augmented-reality physics simulation generated entirely from a structured natural-language prompt. ## Use in Instruction The paper describes the simulation's use in an introductory physics class, illustrating how generative AI and prompt engineering can produce customized, embodied, interactive physics learning tools without programming expertise. ## Connected Concepts - [[physics-education]] - [[generative-ai]] - [[simulation]] - [[generative-ai]] - [[stem-education]] - [[prompt-engineering]] - [[personalized-learning]] - [[scaffolding]] - [[teacher-role]] - [[multimodal]] ## Connected Articles - [[ai-generated-smartphone-circular-motion-lab-2026]] - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[becker-chatgpt-typology-physics-2026]] - [[hashmi-socratic-physics-chatbot-2025]] - [[agentschool-multi-agent-simulation-education-2026]] ## Citation Levy, O., Glazer, J., Finkelstein, N. D., & Ben-Zion, Y. (2026). [*From prompt to embodied simulation: Using generative AI to create AR physics learning tools*](https://arxiv.org/abs/2607.24709). arXiv:2607.24709. --- ## [Development and applications of Generative AI in architectural design studios](https://edtechdev.github.io/aied/articles/genai-architectural-design-studios/) Examines the integration of deep generative models into architectural design education. The findings, based on students' views and observations in design studios, suggest that GenAI supports the exploration of creative ideas — serving as visual stimuli and inspirational resources in early design stages — while also highlighting the competencies students need to differentiate between GenAI models and use them effectively. ## Key Findings - A longitudinal study in architectural design studios led to the development of the **GAI-A platform**, a user-friendly GenAI interface refined through a cyclical process of experimentation, feedback collection, and evaluation. - Students used GenAI as **visual stimuli and inspirational resources**, supporting exploration of the unknown in the early stages of the design process. - A second, combinatorial pattern of use (analogous to Boden's 'combinatorial' method) was effective as **solution space enhancement**: generating alternatives that expand the solution space during the design and development phase, with students using GenAI to advance their preliminary solution ideas. - The authors stress that avoiding **design fixation** requires competencies and discernment to differentiate between GenAI models and use them effectively. - The study also cites evidence (Wadinambiarachchi et al., 2024) that exposure to GenAI images can narrow designers' focus by tying them to specific aesthetics, and notes that several of its own findings require further research. ## Study Design & Method The research combined the development of a GenAI interface with its implementation in design education: GenAI models were used in design studios, feedback was collected, the GAI-A platform was developed and evaluated, and it was subsequently implemented in design education. Findings draw on students' views and observations, including semi-structured interviews with a focus group conducted in week 6 (W6) and week 13 (W13) of studio work, with coded design processes illustrating how GenAI entered the design workflow. ## Implications for AI in Education For [[design-thinking]]-oriented disciplines, the results suggest GenAI is best framed as an ideation resource whose value depends on pedagogical orchestration: students need guidance in model selection and critical discernment to exploit generative alternatives without falling into design fixation or aesthetic lock-in. These competencies extend the [[teacher-role]] in [[higher-ed]] studio pedagogy from transmitting craft to coaching how, when, and why to delegate creative exploration to generative models. ## Connected Concepts - [[higher-ed]] - [[design-thinking]] - [[socratic-ai-dialogue]] - [[teacher-role]] - [[math-education]] - [[prompt-engineering]] - [[affective-tutoring]] - [[plagiarism-detection]] ## Connected Articles - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[ai-team-teaching-talk-analytics]] — AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata ## Citation Leman Figen Gül, Burak Delikanlı, Oğulcan Üneşi, Ertuğrul Ömer Gül (2026). [Development and applications of Generative AI in architectural design studios](https://arxiv.org/abs/2607.24752). arXiv:2607.24752. cs.HC. --- ## [Gen-AI-tecture: using generative AI to support architectural students in design tasks](https://edtechdev.github.io/aied/articles/genai-architecture-education/) Kapsalis (2026) presents one of the first empirical studies of generative AI integration in architectural design education, using a locally executed, discipline-specific tool within a mixed-methods focus-group design. The study addresses three objectives: creativity impact, inclusivity enhancement, and employability preparation. Results showed enhanced creative fluency, broadened participation across diverse learner profiles, and strengthened student confidence in AI-supported workflows. This work extends the [[generative-ai]] education literature beyond text-based domains (coding, writing) into visual-spatial design disciplines. The finding that gen-AI tools broadened participation is particularly significant for [[equity]] research — students who traditionally struggled with manual drafting or 3D modeling gained new entry points. This connects to [[ai-literacy]] discussions about AI as an accessibility tool and [[personalized-learning]] frameworks that emphasize multiple pathways to competence. The emphasis on locally executed (non-cloud) AI is also noteworthy for [[educational-measurement]] and privacy-conscious deployment. The study operationalizes [[constructivist]] principles by positioning AI as a tool for learner-led meaning-making within human-AI networks. For [[faculty-development]], the paper provides evidence-based guidance on gen-AI integration in studio-based disciplines, an area where the [[stanford-evidence-base-ai-k12-2026]] and related literature have been thin. ## Connected Concepts - [[generative-ai]] - [[equity]] - [[ai-literacy]] - [[personalized-learning]] - [[educational-measurement]] - [[constructivist]] - [[faculty-development]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] ## Citation Timo Kapsalis (2026). [Gen-AI-tecture: using generative AI to support architectural students in design tasks](https://arxiv.org/abs/2605.21361). arXiv:2605.21361. arXiv:2605.21361 [cs.HC] — Submitted to Journal of Architectural Education. --- ## [Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment](https://edtechdev.github.io/aied/articles/genai-assessment-governance/) # GenAI as a Design Variable: Assessment Governance Framework This paper proposes a principled framework grounded in Evidence-Centered Design (ECD) that treats [[generative-ai]] as a design variable within STEM assessment arguments rather than an external threat. This represents a significant evolution beyond the binary debate of 'ban AI vs. allow AI' that has dominated discussions about [[academic-integrity]] in education. ## Three Governance Stances The framework articulates three context-dependent governance stances based on how GenAI interacts with the assessment's validity argument: 1. **Restrict** — warranted when GenAI would contaminate the inferential chain between student work products and targeted unaided proficiency. This preserves the validity of assessments designed to measure independent competence. 2. **Scaffold** — warranted when bounded GenAI support can assist with peripheral demands without revealing the target construct, preserving inferential interpretability. This aligns with [[scaffolding]] approaches in [[intelligent-tutoring]] systems. 3. **Require** — warranted when the target construct is disciplinary AI interaction competency itself. Tasks elicit process artifacts (prompts, critiques, revisions) that make student reasoning observable and scorable, distinguishing it from AI-generated output. ## Empirical Validation Two task designs deployed in an introductory physics course demonstrated that disciplinary AI interaction competencies are observable in student response artifacts and can be scored using defensible rubrics grounded in student data and expert knowledge. This connects to [[automated-grading]] and [[automatic-short-answer-grading]] research on making student reasoning visible and scorable. ## Implications for Policy By situating GenAI governance within validity arguments, the framework offers actionable guidance for preserving learning integrity while preparing students for AI-enabled workplaces. This has direct relevance for [[educational-policy-ai]] decisions about assessment design and [[ai-literacy]] standards. The framework also complements emerging work on [[ai-tutor-safety-harms|AI tutor safety]] by providing structured decision-making about when AI interaction is educationally appropriate. ## Connected Concepts - [[generative-ai]] - [[academic-integrity]] - [[scaffolding]] - [[intelligent-tutoring]] - [[automated-grading]] - [[educational-policy-ai]] - [[ai-literacy]] ## Connected Articles - [[automatic-short-answer-grading]] - [[ai-tutor-safety-harms]] ## Citation Gao, Y., Chen, Z., Li, M., & Zhai, X. (2026). [Generative AI as a design variable: An evidence-centered framework for principled governance in STEM assessment](https://arxiv.org/abs/2605.24837). arXiv:2605.24837. cs.CY. --- ## [Generative AI Availability, Grades, and Student Satisfaction at a Large University](https://edtechdev.github.io/aied/articles/genai-availability-grades-satisfaction/) This large-scale observational study tests the "GenAI substitution hypothesis" — the concern that students offload cognitive effort to [[generative-ai]] and earn inflated grades without learning. Using syllabus and administrative data from a large U.S. university (2015–2025; 156,135 students; 87,936 course offerings), the authors measure each course's GenAI susceptibility with a human-validated LLM pipeline that extracts assessment types from syllabi, then apply a difference-in-differences design comparing outcomes before and after ChatGPT's release while modeling COVID-19 effects as persistent or transient. They find **no significant differential effect** of GenAI availability on grades overall or among previously lower-performing students, and no significant effect on self-reported understanding; effects on subject interest are significant only under a transient-pandemic assumption. The findings temper alarm about grade inflation and satisfaction erosion, complementing [[ai-availability-student-motivation]] and the mixed picture in [[generative-ai-reduced-study-time-math]]. The null result is notable against theoretical worries about [[cognitive-offloading]] and situates [[student-experience]] concerns in [[higher-ed]] on firmer empirical ground. ## Connected Concepts - [[generative-ai]] - [[cognitive-offloading]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[ai-availability-student-motivation]] - [[generative-ai-reduced-study-time-math]] ## Citation Dumlao, Wang, Xie, Hu, Bar, Chaney, Gold & Teplitskiy (2026). [Generative AI Availability, Grades, and Student Satisfaction at a Large University](https://arxiv.org/abs/2607.21534). arXiv:2607.21534. arXiv preprint (cs.CY). --- ## [Generative AI Can Harm Teaching](https://edtechdev.github.io/aied/articles/genai-can-harm-teaching-rct-2026/) > The null average performance effect masks strong offsetting heterogeneity — and the exam had severe ceiling compression (control mean 89.2/100, 47% ≥ 95), which also limits power. The belief reversal is striking: it contradicts "familiarity breeds acceptance" and suggests an arc from initial awe at AI's instant responses to awareness of its unintended effects. > Sungu, Lira & Duckworth (2026) ran one of the first large-scale RCTs of a teacher-facing generative AI tool and found it can *harm* students: providing teachers an AI teaching assistant **reduced student intrinsic motivation by 0.11 SD** and — among lower-performing teachers — **cut student achievement by 0.13 SD**. The pattern is a **principal–agent problem**: teachers (agents) gain labor savings from AI delegation while students (principals) bear the cost of displaced relational teaching and scaffolding. ## The experiment - **538 teachers across 24 Turkish K-12 schools** randomized at school-department level; analytical sample 193 teachers / 2,816 students / 14,198 student-course observations - Treatment: custom **GPT-4o chatbot** with Turkish Ministry of Education curriculum database + 1-hour training (one arm added weekly usage-stat reminders); control = business-as-usual - Pre-registered; ITT; semester-length (spring 2025) ## Results | Outcome | Average effect | Heterogeneity | |---|---|---| | Student intrinsic motivation | **−0.111 SD** (p=.015) | Heavy baseline AI users: −0.182 (p=.015); light users: −0.052 (ns) | | Student confidence | −0.090 SD (p=.097) | Lower-performing teachers: −0.183 (p=.012); higher: −0.022 (ns) | | Academic performance | −0.019 SD (ns, ceiling-compressed) | **Below-median teachers' students: −0.129 (p=.005)**; above-median: +0.054 (ns) | | Teacher beliefs about AI's effect on learning | +0.126 SD (ns) | Heavy prior users became *more pessimistic* (−0.379); light users *more optimistic* (+0.458) | The null average performance effect masks strong offsetting heterogeneity — and the exam had severe ceiling compression (control mean 89.2/100, 47% ≥ 95), which also limits power. The belief reversal is striking: it contradicts "familiarity breeds acceptance" and suggests an arc from initial awe at AI's instant responses to awareness of its unintended effects. ## Why the harm happens: usage patterns - **66% of teacher conversations were teaching-material production** (lecture prep 32%, homework/exam 22%, syllabus 9%); only 16% instructional support; 18% general - **Shallow use: median 2 prompts, mean 4.7 messages** per session — teachers accepted outputs with minimal iteration - Interpretation: **task delegation, not pedagogical collaboration** — the tool was a generator of finished artifacts rather than an iterative partner, limiting the pedagogical reflection that separates augmentation from substitution ## Connected Concepts - [[generative-ai]] - [[k-12]] - [[student-experience]] - [[teacher-ai-competency]] - [[teacher-role]] - [[rag]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-expertise-pathways-sysadmin]] — Unanticipated Effects of Generative AI on Expertise Pathways and Performance Perception in System Administration - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming ## Citation Sungu, Lira & Duckworth (2026). [*Generative AI Can Harm Teaching*](https://ssrn.com/abstract=7007339) --- ## [Integrating Generative AI into Cybersecurity Education: A Study of OCR and Multimodal LLM-Assisted Instruction](https://edtechdev.github.io/aied/articles/genai-cybersecurity-ocr-multimodal-instruction-2025/) > **Synthesis:** Patel et al. (2025) present an LLM-assisted instructional integration with a virtual cybersecurity lab platform, addressing workforce reskilling needs driven by the digital transformation of Fourth Industrial Revolution (4IR) systems. Recognizing that the workforce must be reskilled and upskilled for STEM skills such as robotics, automation, AI, and security, the authors integrated a generative-AI instructional assistant into a prior experiential learning platform. The system assists trainees by acting as an instructional assistant, helping students build skill sets while performing experiential learning exercises. ## Key Findings - The rapid digital transformation of **4IR systems is transforming workforce needs**, increasing skill-set gaps, especially for older workers, with growing emphasis on robotics, automation, AI, and security skills. - **Generative AI can aid workforce building** by acting as an instructional assistant that helps trainees build skills during experiential learning exercises. - The paper presents a **generative-AI-based instructional assistant integrated into a virtual cybersecurity lab platform**, leveraging OCR and multimodal LLM capabilities to assist instruction. - The approach supports **experiential, hands-on learning** in cybersecurity education. ## Study Design & Method This is a **research-category full paper** describing the design and integration of an LLM-assisted instructional assistant into a virtual cybersecurity lab platform. The system uses generative AI (including OCR and multimodal LLM capabilities) to act as an instructional assistant within an experiential learning environment, guiding trainees through exercises. The paper describes the architecture, integration, and use of the system to support skill development in cybersecurity, responding to workforce reskilling demands. ## Implications for AI in Education The paper demonstrates how [[generative-ai]] can serve as an [[ai-tutoring|instructional assistant]] within experiential, hands-on learning environments, applied here to [[cs-education]] education. It connects to [[llm|large language models]] as [[llm|learning assistants]], to [[higher-ed]] and workforce reskilling, and to [[stem-education]] skills such as automation and AI. For educators, it shows how multimodal LLM capabilities (including OCR) can support lab-based and exercise-driven learning, reducing the instructional burden while enabling experiential skill building. ## Limitations The paper focuses on a specific cybersecurity lab context, and the described study's empirical evaluation scope is not fully detailed in the abstract. The emphasis is on workforce reskilling in cybersecurity, so generalizability to other disciplines and to traditional academic settings may be limited. The reliance on LLM assistance raises considerations about accuracy and oversight in instructional content. ## Connected Concepts - [[generative-ai]] - [[llm]] - [[cs-education]] - [[higher-ed]] - [[experiential-learning]] - [[ai-tutoring]] ## Connected Articles - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM: LLMs and Robotic Simulation - [[jeon-isd-agent-bench-2026]] — ISD Agent Bench - [[teachbench-llm-teaching-evaluation]] — TeachBench: Evaluating LLM Teaching Ability ## Citation Patel, K., Lin, Y.-Z., Raul, G., Shih, B. P.-J., Redondo, M. W., Saber Latibari, B., Pacheco, J., Salehi, S., & Satam, P. (2025). *Integrating generative AI into cybersecurity education: A study of OCR and multimodal LLM-assisted instruction*. arXiv:2509.02998. --- ## [Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education](https://edtechdev.github.io/aied/articles/genai-declaration-frameworks-higher-education/) As [[generative-ai]] disrupts [[higher-ed]], institutions increasingly require students to declare AI use. However, generic binary declarations (e.g., "I used GenAI") fail to capture the nuanced application of these tools across different academic tasks. Micallef & Petrovska argue that establishing transparency is key to protecting [[academic-integrity]], promoting [[ai-literacy]], and shifting the focus from policing to professional practice. The paper contributes a design artifact: a framework of two task-specific declaration structures — one for writing-focused activities and one for coding assessments — developed for a Computer Science department based on an existing taxonomy of GenAI usage. By categorising AI usage across specific cognitive and developmental stages (e.g., structural planning vs. textual content generation, or code improvement vs. code generation), the framework encourages students to reflect on their own learning process and clarifies the boundary between acceptable assistance and [[academic-integrity|academic misconduct]]. This domain-specific approach is proposed as a foundation for fostering more honest assessment in Computer Science and other disciplines, aiming to better prepare students for professional environments where documenting GenAI workflows might be an essential job requirement. The work connects to broader discussions about [[ai-higher-ed-bridge-gap]] and the tension between [[educational-policy-ai|institutional regulation]] and [[student-experience|student practice]]. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[academic-integrity]] - [[ai-literacy]] - [[educational-policy-ai]] - [[student-experience]] ## Connected Articles - [[ai-higher-ed-bridge-gap]] ## Citation Micallef, N., & Petrovska, O. (2026). [Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education](https://arxiv.org/abs/2606.13389). arXiv:2606.13389. --- ## [From Unified to Differentiated Materials: Generative AI–Supported Adaptation of EAP Reading Materials](https://edtechdev.github.io/aied/articles/genai-differentiated-eap-reading-materials-2026/) > **Synthesis:** Gao (2026) examined whether generative-AI-supported adaptation of English for Academic Purposes (EAP) reading materials chiefly changes passage-level structural complexity or text-embedded functional support. Using a role-prompted workflow (barrier analysis, adaptation, fidelity checking, validation) and a 3×3 between-subjects design (N=135; proficiency × material condition), the study found that GenAI-supported differentiation operates primarily at the level of proficiency-specific support-layer design — glosses, sentence unpacking, rhetorical cues, claim-evidence notes, and critical prompts — rather than broad changes in passage-level structural complexity. Differentiated-AI materials most strongly outperformed unified-AI versions among high-proficiency learners (d = 1.40), with a moderate advantage among low-proficiency learners (d = 0.60) and a small difference for intermediate learners. ## Key Findings - Instructors rated the generated materials favorably for academic fidelity (M = 4.24), proficiency appropriateness (M = 4.36), and teachability (M = 4.31), with acceptable inter-rater reliability, ICC(2,k) = 0.84. - Automated structural-complexity indicators showed limited separation between proficiency versions (η²p = 0.043), and leave-one-out discriminant analysis classified intended proficiency labels at only 11.1% — well below the 33.3% balanced-task benchmark. - Material differences were clearest in functional support: glosses, sentence unpacking, rhetorical cues, claim–evidence notes, and critical prompts, rather than in rewritten sentence complexity. - Differentiated-AI exceeded unified-AI most strongly among high-proficiency learners (d = 1.40), followed by a moderate advantage among low-proficiency learners (d = 0.60) and a small difference among intermediate learners (d = 0.16). - The omnibus reading-comprehension interaction identified this heterogeneity, F(4, 126) = 7.43, p < 0.001, η²p = 0.191; immediate unsupported application did not differ by material condition or interaction. ## Study Design & Method The study deployed a four-module role-prompted GenAI workflow — barrier analysis, adaptation, academic-fidelity checking, and validation — designed to make generation auditable and protect the source text's disciplinary meaning. A 3×3 between-subjects experiment (N = 135; n = 15 per cell) crossed proficiency level (low, intermediate, high) with material condition (original, unified-AI, differentiated-AI). Three EAP instructors evaluated 15 anonymized material versions. Analyses included automated structural-complexity indicators, leave-one-out discriminant analysis, within-proficiency planned contrasts (comparing differentiated-AI with unified-AI while holding proficiency constant), and omnibus outcome models. Learner-outcome analysis gave priority to within-stratum planned contrasts as the cleaner evidence, while the omnibus Proficiency × Material model described heterogeneity across strata. ## Implications for AI in Education The findings argue that GenAI's value for [[language-learning]] materials lies not in blanket simplification but in targeted, proficiency-sensitive support-layer design that preserves academic fidelity — terminology, stance, hedging, citation relations, and argumentative structure. This reframes GenAI as a tool for [[personalized-learning|differentiation]] where the key design decision is what support each proficiency level needs (lexical/syntactic access for low-proficiency readers, discourse organization for intermediate readers, stance/evidence evaluation for high-proficiency readers) rather than a single harder-or-easier rewrite. For teachers, the study offers a concrete, auditable role-prompted workflow for generating and validating materials, and it underscores the continued necessity of teacher oversight against hallucinated content, flattened stance, and over-simplification in [[generative-ai]] output. ## Limitations The learner-outcome and process indicators were collected in the same session, making the process estimates descriptive rather than causal. Immediate unsupported application did not differ by condition, limiting evidence on transfer. The expert-rating rubric and the specific EAP genre constrain generalization to other disciplines and material types. Sample sizes per cell (n = 15) are modest, and the within-proficiency comparisons, while cleaner, rest on single-instructor-generated material versions. ## Connected Concepts - [[language-learning]] - [[generative-ai]] - [[personalized-learning]] - [[scaffolding]] - [[instructional-design]] - [[teacher-role]] - [[curriculum-design]] ## Connected Articles - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[ai-tools-arab-english-classrooms]] — AI Tools in Arab University English Classrooms - [[learner-centered-feedback-ai]] — Enhancing Learner-Centered Feedback with AI - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task - [[genai-linguistic-diversity-academic-writing]] — Generative AI and Linguistic Diversity in Academic Writing ## Citation Gao, X. (2026). [*From unified to differentiated materials: Generative AI–supported adaptation of EAP reading materials*](https://doi.org/10.3389/fpsyg.2026.1887565). --- ## [Unanticipated Effects of Generative AI on Expertise Pathways and Performance Perception in System Administration](https://edtechdev.github.io/aied/articles/genai-expertise-pathways-sysadmin/) > **Rana Abou Khamis, Hala Assal, Ashraf Matrawy** — arXiv preprint (2026). ## Synthesis Drawing on 14 semi-structured interviews with IT professionals, this study explores how GenAI integration reshapes professional practice in system administration — troubleshooting, scripting, and system verification. Two unanticipated socio-technical findings: 'compression of traditional expertise pathways' — GenAI acts as both mentor-like tutor and 'ladder-shortening' tool, accelerating unfamiliar-domain task performance while reducing exposure to the foundational build-fail-debug cycles that historically built expertise. 'Performance perception shift': AI-assisted speed resets organizational and self-expectations, creating a 'two-speed culture' within teams and 'productivity guilt' — a metacognitive cost of AI-augmented work. The findings extend concerns about cognitive offloading and skill decay from classroom settings to professional practice, with implications for professional training programs and workplace learning. ## Connected Concepts - [[human-in-the-loop-ai]] - [[student-experience]] - [[ai-misuse-learning-harm]] - [[open-source]] - [[over-reliance]] - [[critical-thinking]] - [[lifelong-learning]] - [[student-misconceptions-ai]] ## Connected Articles - [[cognitive-commons-ai-expertise-regeneration]] — The Tragedy of the Cognitive Commons: How AI Could Disrupt the Regeneration of Professional Expertise - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning - [[absent-cognitive-baseline-2026]] — The Absent Cognitive Baseline: Theorizing a Structural Gap in AI-Native College Students' Academic Self-Assessment - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding ## Citation Abou Khamis, R., Assal, H., & Matrawy, A. (2026). [Unanticipated effects of generative AI on expertise pathways and performance perception in system administration](https://arxiv.org/abs/2607.28650). arXiv:2607.28650. --- ## [Human-centered GenAI feedback design in higher education: a multisite experiment on direct, reflective, and hybrid approaches to scientific argumentation](https://edtechdev.github.io/aied/articles/genai-feedback-design-multisite-experiment/) > **Synthesis:** A multisite, cluster-randomized field experiment (1,176 first-year undergraduates, 48 sections, 4 universities, 3 science domains) compares four feedback designs for scientific argumentation: peer-only, direct GenAI, reflective GenAI (self-evaluation then AI critique), and hybrid (self-evaluation + peer + GenAI). The hybrid condition produced the highest argument-quality gains and clearest advantage on conceptual learning; reflective and hybrid designs both outperformed direct GenAI on delayed AI-free transfer. Findings suggest that GenAI's educational value depends less on AI access than on preserving student agency, evaluative judgment, and ownership during revision. ## Study Design Ateş conducted a multisite, cluster-randomized, longitudinal field experiment in introductory university science courses: - **1,176 first-year undergraduates** from 48 course sections across 4 universities - **3 science domains** — biology, chemistry, physics - **4 feedback conditions** randomized at the section level: 1. Peer feedback only (control) 2. Direct GenAI-supported feedback — AI critique delivered to students 3. Reflective GenAI-supported feedback — self-evaluation first, then AI critique 4. Hybrid design — self-evaluation → peer feedback → GenAI critique ## Key Findings | Outcome | Direct GenAI | Reflective GenAI | Hybrid | |---|---|---|---| | Immediate argument-quality gain | Better than peer | — | **Highest** | | Feedback uptake | — | Stronger | Stronger | | Self-regulated learning | — | Stronger | Stronger | | Conceptual learning | — | Positive (n.s.) | **Clearest advantage** | | Delayed AI-free transfer | — | **Outperformed direct** | **Outperformed direct** | - **Direct GenAI** improved immediate argument quality over peer feedback but showed weaker transfer - **Reflective and hybrid designs** produced stronger feedback uptake and self-regulated learning - **Hybrid condition** showed the clearest advantage on conceptual learning - **Both reflective and hybrid** outperformed direct on delayed AI-free transfer - **Multilevel mediation**: feedback uptake and self-regulated learning partially explained these advantages ## Why Design Matters The paper argues that feedback becomes educationally valuable not through comment delivery alone, but when learners: 1. Interpret critique 2. Compare it against criteria 3. Judge its relevance 4. Use it to improve subsequent work Direct GenAI feedback may encourage passive uptake — students outsource evaluative judgment to the system. Reflective and hybrid designs preserve **epistemic agency**: the student must first evaluate their own work, compare peer/AI inputs, and decide how to revise. The core insight: **GenAI's educational value depends less on AI access per se than on whether feedback environments preserve student agency, evaluative judgment, and ownership during revision.** ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[self-regulated-learning]] - [[rag]] ## Connected Articles - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory ## Citation Ateş, H. (2026). Human-centered GenAI feedback design in higher education: A multisite experiment on direct, reflective, and hybrid approaches to scientific argumentation. [*International Journal of Educational Technology in Higher Education*](https://doi.org/10.1186/s41239-026-00614-9), *23*(38) --- ## [Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025)](https://edtechdev.github.io/aied/articles/genai-higher-education-systematic-review-2026/) > **Synthesis:** This PRISMA-guided systematic review synthesizes 125 peer-reviewed studies (2022–2025) on generative AI in higher education, documenting exponential adoption (92% student usage by 2025), four primary application domains, and persistent challenges around academic integrity, bias, hallucination, faculty readiness, and digital equity. ## Key Findings 1. **125 studies, PRISMA 2020.** The review synthesizes empirical evidence across 125 peer-reviewed studies following PRISMA 2020 guidelines, covering Scopus, Web of Science, ERIC, IEEE, and Google Scholar. 2. **Exponential GenAI adoption.** Student GenAI usage reached 92% by 2025, reflecting exponential growth since ChatGPT's release in November 2022. 3. **Four primary application domains.** Applications cluster into automated feedback and assessment, personalised learning support, critical skill development, and research assistance. 4. **Balanced benefits and challenges.** Benefits include enhanced learning efficiency, accessibility, and personalisation; challenges include academic integrity, algorithmic bias, hallucination rates, faculty readiness gaps, and digital equity concerns. ## Implications This broad systematic review provides a comprehensive landscape of [[generative-ai]] in [[higher-ed]], useful as a synthesis anchor for the wiki's coverage of GenAI applications. The 92% student-usage figure and the four-domain taxonomy ([[automated-assessment]], [[personalized-learning]], critical skills, research assistance) offer a useful organizing structure. The persistent challenges — [[academic-integrity]], [[bias-mitigation]]/algorithmic bias, [[hallucination-risk]], faculty readiness, and [[equity]]/[[equity]] — echo themes across the wiki and connect to [[ai-literacy]] and [[faculty-development]] needs. The review's identification of pedagogical innovations (new assessment designs, AI literacy curricula, human-AI collaborative models) aligns with [[human-ai-collaboration]] and [[human-in-the-loop-ai]] threads. For policy and practice, the review underscores that the faculty readiness gap and equity concerns remain the binding constraints on responsible GenAI integration, connecting to [[educational-policy-ai]] and [[ai-governance-education]]. ## Connected Concepts - [[academic-integrity]] - [[ai-governance-education]] - [[ai-literacy]] - [[automated-assessment]] - [[bias-mitigation]] - [[educational-policy-ai]] - [[equity]] - [[faculty-development]] - [[generative-ai]] - [[hallucination-risk]] - [[higher-ed]] - [[human-ai-collaboration]] - [[personalized-learning]] ## Connected Articles - [[ai-uk-higher-education-policy-2026]] — AI in UK higher education policy - [[pchl-he-framework-genai-content-creation-2026]] — PCHL-HE framework - [[learnity-graphs-lifelong-learning-framework-2026]] — Learnity graphs framework - [[elementary-writing-genai-systematic-review-2026]] — Elementary writing GenAI review - [[amponsah-ai-acceptance-science-teachers-2026]] — AI acceptance science teachers ## Citation Rathnayake, P. B. (2026). [*Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025)*](https://osf.io/ky4uj). EdArXiv preprint. --- ## [The impact of generative artificial intelligence on academic development of Chinese students in humanities and social sciences](https://edtechdev.github.io/aied/articles/genai-impact-chinese-students-hss/) # The Impact of GenAI on Chinese HSS Students' Academic Development This large-scale survey of humanities and social sciences (HSS) students in China examines how [[generative-ai]] reshapes academic development across four dimensions: usage patterns, effects on learning processes and performance, challenges, and preferred curricular integration approaches. Over half of respondents reported enhanced learning motivation, independent thinking, and creativity, though a substantial minority saw little change or decline. A larger majority reported academic performance gains, though the authors caution these may partly reflect limitations in conventional [[assessment]] practices. ## Key Findings **Usage patterns:** HSS students use GenAI primarily for writing assistance, information synthesis, and idea generation — tasks that align closely with HSS learning outcomes expressed through written and interpretive forms. Variations emerged by discipline and duration of GenAI experience, with modest gender differences. **Learning processes:** More than half of students perceived enhanced motivation, independent thinking, and creativity. However, a substantial minority reported little change or even decline, suggesting that [[personalized-learning]] approaches to GenAI integration may be necessary. **Academic performance:** A notably larger majority reported academic gains, though these may partly reflect assessment practices ill-equipped to distinguish AI-assisted from independent work — a challenge related to [[academic-integrity]]. **Challenges:** Limited accuracy and [[over-reliance]] emerged as the most pressing concerns. While an overwhelming majority valued ethical considerations, only slightly more than half were satisfied with [[privacy]] protections. Students favored partial or optional GenAI integration into curricula. ## Implications The study highlights the need for [[higher-ed]] institutions to develop nuanced policies that balance GenAI's potential benefits against risks of over-reliance and assessment validity. Disciplinary differences suggest that [[faculty-development]] programs should tailor guidance to specific fields rather than adopting one-size-fits-all approaches. ## Connected Concepts - [[generative-ai]] - [[assessment]] - [[personalized-learning]] - [[academic-integrity]] - [[over-reliance]] - [[privacy]] - [[higher-ed]] - [[faculty-development]] ## Connected Articles - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[genai-architecture-education]] — Gen-AI-tecture: using generative AI to support architectural students in design tasks - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) ## Citation Fan, L., & Liu, F. (2026). [The impact of generative artificial intelligence on academic development of Chinese students in humanities and social sciences](https://arxiv.org/abs/2606.24104). arXiv:2606.24104. --- ## [Generative AI and linguistic diversity in academic writing and publishing: Perspectives from World Englishes](https://edtechdev.github.io/aied/articles/genai-linguistic-diversity-academic-writing/) Structured scholarly dialogue among five sociolinguists examining how GenAI tools influence academic writing practices, reinforce or disrupt linguistic hierarchies, and impact the legitimacy of diverse English varieties in global scholarly communication. Raises concerns about linguistic homogenization and the marginalization of World Englishes. ## Key Findings - The article is a **structured scholarly dialogue** among five sociolinguists from World Englishes and adjacent fields, organized around **five guiding questions** covering GenAI's broad influence in academic writing and publishing (AWP), its potential biases toward dominant Englishes, institutional responsibilities, peer review practices, and ethical frameworks. - Contributors see potential for GenAI to **democratise writing processes**, while raising concerns that it may **homogenise linguistic styles, privilege dominant English varieties, and flatten nuance** in scholarly writing. - The dialogue foregrounds themes of **linguistic (in)justice, researcher agency, and institutional responsibility**, with contributors calling for equity-informed policies, critical AI literacy, and inclusive co-design in GenAI development. - Drawing on Blommaert's notion of "orders of indexicality," contributors argue that language varieties are evaluated through hierarchies of value, and that GenAI **reinforces these hierarchies by automating what counts as 'good' writing** — so apparent improvements in quality may conceal the algorithmic enforcement of standardised norms. - The authors conclude that while GenAI may reinforce existing hierarchies, it **can also serve as a site of resistance**, depending on how it is designed, governed, and used within scholarly communities committed to linguistic diversity. ## Study Design & Method Rather than an empirical study, the piece is a dialogic scholarly intervention: five contributors with expertise in World Englishes and adjacent fields respond to five guiding questions, with the article structured around their exchanges. The authors contrast this approach with recent interview-based work (Moorhouse et al., 2025) that reports policy ambiguity among applied-linguistics journal editors, who largely restricted acceptable GenAI use to language polishing while treating transparency as essential. ## Implications for AI in Education For [[writing-education]] and English for Research Publication Purposes, the dialogue implies that GenAI tools are not linguistically neutral: they encode standards that can marginalise minoritised varieties, so questions of [[equity]] must be built into how academic writing instruction prepares students to use these tools. The calls for critical [[ai-literacy]] and inclusive co-design connect directly to curricula, suggesting students should be taught to use GenAI reflexively rather than defer to its standardising output. ## Connected Concepts - [[language-learning]] - [[ai-literacy]] - [[writing-education]] - [[equity]] - [[automated-essay-scoring]] - [[student-experience]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[ai-tools-arab-english-classrooms]] — AI tools in Arab University English classrooms: Looking back and forward - [[will-skill-not-tool-chinese-university-students-acceptance-of-generative-ai-for-]] — Will, Skill, Not Tool: Chinese university students' acceptance of generative AI for academic writing in informal English medium instruction settings - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[genai-architecture-education]] — Gen-AI-tecture: using generative AI to support architectural students in design tasks - [[post-covid-ict-career-aspirations]] — Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills? ## Citation Kingsley Ugwuanyi, Christian Mair, Sender Dovchin, Iker Erdocia, Maria Kuteeva (2026). [Generative AI and linguistic diversity in academic writing and publishing: Perspectives from World Englishes](https://arxiv.org/abs/2607.28505). arXiv:2607.28505. cs.CL. --- ## [Generative AI Literacy Training Improves Intelligence Analysts’ Discrimination of Real and AI-Generated Images](https://edtechdev.github.io/aied/articles/genai-literacy-image-discrimination/) Kamali et al. (2026) evaluate a Generative AI Literacy training intervention designed to improve intelligence analysts' ability to distinguish real photographs from AI-generated images. In a controlled experiment, trained analysts significantly outperformed untrained controls on image discrimination tasks, with gains persisting on challenging edge cases. This is an important contribution to [[ai-literacy]] research because it demonstrates that detection skills are teachable even among domain experts, challenging the assumption that AI-generated content is fundamentally undetectable. The training draws attention to specific visual artifacts and generative model signatures, providing a template for broader [[professional-training]] curricula. The study connects to [[equity]] concerns around AI-generated misinformation, since disparities in detection ability can compound existing information-access inequalities. It also informs [[generative-ai]] regulation discussions by providing evidence that literacy interventions are a viable complement to technical watermarking approaches. The work extends [[ai-literacy-continuum-higher-education]] by showing that literacy training is relevant not just for students but for professionals across domains. ## Connected Concepts - [[ai-literacy]] - [[professional-training]] - [[equity]] - [[generative-ai]] ## Connected Articles - [[ai-literacy-continuum-higher-education]] ## Citation Negar Kamali, Candice Rockell Gerstner, Jessica Hullman, Matthew Groh (2026). [Generative AI Literacy Training Improves Intelligence Analysts’ Discrimination of Real and AI-Generated Images](https://arxiv.org/abs/2606.28510). arXiv:2606.28510. cs.HC / cs.AI / cs.CY. --- ## [Development and evaluation of artificial intelligence literacy training for teacher education students](https://edtechdev.github.io/aied/articles/genai-literacy-training-teacher-education-dbr-2026/) > **Synthesis:** Le, Huynh, Dang, Pham, Nguyen, and Nguyen (2026) develop and evaluate a design-based research (DBR) intervention providing GenAI literacy training for teacher education students. Arguing that existing AI literacy programs overemphasize technical knowledge and pre-GenAI tools, the study integrates contemporary AI competency frameworks into a workshop prototype. The workshop was piloted with 14 master's students and evaluated with 29 teacher education students. Findings indicate the intervention effectively enhanced overall AI competency self-efficacy, with positive emotional responses from 87.1% of participants and shifts toward critical exploration of GenAI tools for teaching. ## Key Findings - Design-based research produced design principles for GenAI literacy training in teacher education, addressing the gap between technical AI knowledge and pedagogical application - Pilot with 14 master's students followed by evaluation with 29 teacher education students showed enhanced overall AI competency self-efficacy - 87.1% of pilot participants expressed positive emotional responses (curiosity, inspiration, excitement); 12.9% expressed concerns about AI's societal impact - Half (51.1%) reported shifting perspectives about learner and teacher roles; 40% expressed intentions to critically integrate GenAI into future teaching - AI ethics gains shifted from significant to marginally significant, suggesting pedagogical and assessment competencies may require longer-term development ## Connected Concepts - [[ai-literacy]] - [[teacher-ai-competency]] - [[faculty-development]] - [[pedagogical-llm-training]] - [[teacher-role]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[hcap-human-centric-ai-pedagogy-framework-2026]] - [[teacher-education-ai-literacy-sdt-2026]] - [[ai-tpack-preservice-math-teachers]] - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] ## Citation Le, T. H., Huynh, L., Dang, B., Pham, H.-H., Nguyen, N. T. V., & Nguyen, A. (2026). [*Development and evaluation of artificial intelligence literacy training for teacher education students*](https://doi.org/10.1111/bjet.70047). British Journal of Educational Technology. --- ## [When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education](https://edtechdev.github.io/aied/articles/genai-marketing-education-roles-2026/) > **When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education** — Uses multipronged analysis (syllabi review, educator survey, qualitative interviews) and Role Theory + Community of Inquiry model to propose three GAI roles in education: tutor (grasping theoretical concepts), teammate (brainstorming and problem-solv... [[generative-ai]] [[higher-ed]] pedagogy [[instructional-design]] [[teacher-role]] [[professional-training]] Uses multipronged analysis (syllabi review, educator survey, qualitative interviews) and Role Theory + Community of Inquiry model to propose three GAI roles in education: tutor (grasping theoretical concepts), teammate (brainstorming and problem-solving), and tool. Each role influences teaching, social, and cognitive presence differently. Identifies ethical considerations: data privacy, plagiarism, AI dependency, and assessment fairness. Provides concrete examples for GAI integration in courses. ## Abstract Generative Artificial Intelligence (GAI) is increasingly being integrated into marketing education and is reshaping the skillsets required in marketing careers. Building on Role Theory and the Community of Inquiry (CoI) model, we propose that GAI can assume three roles in marketing education: tutor, teammate, and tool. Each role influences teaching, social, and cognitive presence differently, shaping the learning experience and preparing workplace-ready marketing graduates. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[instructional-design]] - [[teacher-role]] - [[professional-training]] ## Connected Articles - [[ai-interior-design-malaysia-2026]] — Artificial Intelligence as Catalyst and Contested Terrain: Transforming Interior Design Practice, Pedagogy, and Professional Regulation in Malaysia - [[talebzadeh-ai-green-education-2026]] — The Role of Artificial Intelligence in Green Education: Optimizing Teacher Workflow and Enhancing Pedagogical Design under Sustainable Development Pedagogy (SDP) Constraints - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[ai-literacy-legal-translation-2026]] — AI Literacy for Legal Translation: Developing Digital Resilience - [[learnity-graphs-lifelong-learning-framework-2026]] — Rethinking Higher Education: From Fixed Curricula to Learnity Graphs ## Citation Unnati Narang, Vishal Sachdev, & Ruichun Liu (2026). [When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education](https://arxiv.org/abs/2608.03973). arXiv:2608.03973. Journal of Public Policy & Marketing, 44(3), 473-489 (2025). --- ## [A meta-analysis of the effect of generative AI on productivity and learning in programming](https://edtechdev.github.io/aied/articles/genai-meta-analysis-programming-learning/) > Maier, Gunzenhäuser & Schweisthal (2026) conduct a **meta-analysis synthesizing evidence** on how generative AI tools affect both programming productivity and learning outcomes. This is a **confidence: high** paper due to its synthesis design across multiple studies, addressing the central tension between short-term efficiency gains and long-term skill development. # Generative AI Meta-Analysis: Programming Productivity vs. Learning ## Core Contribution Maier, Gunzenhäuser & Schweisthal (2026) conduct a **meta-analysis synthesizing evidence** on how generative AI tools affect both programming productivity and learning outcomes. This is a **confidence: high** paper due to its synthesis design across multiple studies, addressing the central tension between short-term efficiency gains and long-term skill development. ## Key Findings The meta-analysis examines the **productivity-learning trade-off** that sits at the heart of [[transfer-of-learning]] debates: when AI tools boost immediate coding output, do they simultaneously undermine the development of foundational programming skills? This directly connects to the broader question of whether [[tutoring-specific-vs-general-ai]] matters — general AI coding assistants may optimize for productivity at the expense of learning. ## Significance for AIED This paper provides the **highest-level evidence synthesis** currently available on the impact of generative AI in programming education. It bridges [[rct]]-level findings with practical implications for: - **Curriculum design:** When should AI tools be introduced in programming courses? - **Assessment:** How do we [[assessment-validity|validly assess]] programming skill when AI is available? - **Policy:** What guidance should [[regulation]] frameworks provide for AI in CS education? The meta-analytic approach provides more robust conclusions than individual studies like [[collaborative-ai-tutoring]] (ProPACT) or [[agentic-education-coding]] work on teaching Claude Code. It also connects to [[programming-its]] research on traditional intelligent tutoring for coding, providing a baseline for comparing AI-augmented approaches. The findings have implications beyond programming — the productivity-learning tension applies to [[writing-education]], [[language-learning]], and any domain where AI can produce acceptable output without the learner developing underlying skills (the [[llm-fallacy-misattribution]] problem). ## Open Questions - Does the productivity-learning trade-off vary by student skill level (novices vs. experts)? - How do findings generalize beyond programming to other [[stem-education]] domains? - What instructional designs mitigate the learning cost while preserving productivity gains? ## Connected Concepts - [[rct]] - [[assessment-validity]] - [[regulation]] - [[writing-education]] - [[language-learning]] - [[stem-education]] ## Connected Articles - [[transfer-of-learning]] - [[tutoring-specific-vs-general-ai]] - [[collaborative-ai-tutoring]] - [[agentic-education-coding]] - [[programming-its]] - [[llm-fallacy-misattribution]] ## Citation Schweisthal, A.S.M.M.G.J., on, A.M.O.T.E.O.G.A., Manuel, S.M.M.G.J.S., 1,2, S.A.S.F., Munich, L.M., Munich, M.C.F.M.L., searched, O.G.C.A.O.P.A.L.W.S., & us-, A.S.W.R.T.C.G.W.U.P. (2026). [*A meta-analysis of the effect of generative AI on productivity and learning in programming*](https://arxiv.org/abs/2605.04779). contrast, we find no statistically significant effect of GenAI assistance on learning outcomes (g = 0 --- ## [Generative AI (GenAI) as a mindtool that supports generative learning (GL)](https://edtechdev.github.io/aied/articles/genai-mindtool-generative-learning/) > **Synthesis:** Generative AI (GenAI) as a mindtool that supports generative learning (GL) ## Key Findings - The paper argues that **Generative AI can serve as a Mindtool** — a knowledge representation tool — that facilitates Generative Learning (GL) rather than short-circuiting it. - **Generative Learning** asks students to actively generate information and make connections between new and existing knowledge; this active participation promotes deeper understanding, long-term retention, and critical thinking and problem-solving abilities, with the "generation" process being where learners make sense of material rather than passively receiving it. - The authors propose **eight pedagogical roles** for GenAI: learning strategy/study buddy, collaborative thinking tool, possibility engine, Socratic opponent, personal tutor, exploratory research engine, motivator, and dynamic assessor. - Each role maps to a distinct learning function, from supporting knowledge organization and comprehension monitoring, to challenging students to refine arguments, to evaluating knowledge in real time so activities can be tailored to current understanding. - The paper ends with the conceptualization of a **pedagogical framework** for designing and supporting generative learning activities (GLAs) with GenAI. ## The Eight Pedagogical Roles The roles span the learning process. As a **learning strategy or study buddy**, GenAI supports knowledge organization and comprehension monitoring at varying degrees of complexity; as a **collaborative thinking tool**, it fosters teamwork and project-based activities by encouraging the sharing, discussion, and integration of spatial representations of content into a more cohesive knowledge structure; as a **possibility engine**, it generates alternative responses that let students explore different ways of expressing ideas; and as a **Socratic opponent**, it challenges students to develop and refine their arguments. The remaining roles extend beyond dialogue: a **personal tutor** provides personalized feedback, an **exploratory research engine** helps students explore and interpret data, a **motivator** proposes games and challenges to engage learners, and a **dynamic assessor** evaluates student knowledge in real time, allowing for tailored generative learning activities based on current understanding. ## Evidence from Research The authors ground the mindtool framing in three strands of evidence. First, the eight generative learning strategies formalized by Fiorella and Mayer (2015) — **summarizing, mapping, drawing, imagining, self-testing, self-explaining, teaching, and enacting** — operationalize Wittrock's (1974) original generative learning theory, and each maps onto roles GenAI can scaffold. Second, Makransky et al.'s (2025) experiments with a theory-informed chatbot (ChatTutor) that scaffolds student-produced explanations show the value of explicit generative scaffolding over raw LLM use: in Study 1 (N = 175; ChatTutor = 51, ChatGPT-4 = 49, teaching-as-usual = 75), immediate post-test differences were non-significant, F(2,172) = 1.89, p = .155, but four weeks later ChatTutor users retained substantially more conceptual knowledge (M = 7.26 of 9) than ChatGPT users (M = 6.72) or teaching-as-usual (M = 5.93). Third, concept mapping — a primary generative strategy — shows large benefits versus passive learning (college students d ≈ 0.72; grades 4–8 d ≈ 0.68; grades 9–12 d ≈ 0.74), and can now be co-constructed iteratively with GenAI-assisted tools. The authors also note that generative strategies are not universally effective: their fit with learners' age and cognitive capacity matters, and strategic scaffolding — such as partially completed concept maps or metacognitive prompts — makes advanced strategies accessible to all learners. ## The A2-GLD Framework To put these ideas into practice, the paper proposes the **AI-Augmented Generative Learning Design model (A2-GLD)**, a five-phase pedagogical framework that combines GL theory, GenAI affordances, and instructional scaffolding: 1. **Prime the learning task** — teacher-initiated; activates prior knowledge and curiosity through prompting questions, AI-generated visuals, and analogies (e.g., "What do you already know about ecosystems?"). 2. **Enact or Visualize** — learner-initiated; builds internal mental models through non-verbal generative activities such as concept maps, diagrams, and gesture-based explanations, optionally animated via AI tools. 3. **Explain with AI** — learner-driven; students verbalize understanding in their own words with AI feedback (e.g., "Great start! Can you clarify what you mean by energy transfer?"). 4. **Human Scaffolding** — instructor-driven; teachers review AI–student interaction artifacts, facilitate peer discussion, and embed metacognitive prompts (e.g., "What confused you the most in the AI chat?"). 5. **Reflect and Transfer** — learner-driven; strengthens metacognition and transfer of learning beyond the activity. The alternating driver of each phase (teacher → learner → learner → teacher → learner) is deliberate: it keeps the instructor in the loop while preserving the learner's active generation of knowledge, and it positions GenAI as a configurable design material — a "primed" tool customized for each activity — rather than a fixed answer machine. ## Implications for AI in Education The framework gives [[instructional-design]] practitioners a vocabulary for using GenAI to enhance rather than replace learning: each role is tied to a specific generative learning function, and the resulting pedagogical model is intended to guide the design of GLAs. This connects to [[constructivist]] and [[self-regulated-learning]] traditions, where the learner's active knowledge construction — supported, not performed, by the tool — is the point of the activity, and it offers a counterweight to answer-generating uses of GenAI in the classroom. The authors frame the underlying goal in terms of learning with, not from, the technology: using GenAI as a mindtool can sharpen inference-making and critical thinking while avoiding the accumulation of inert (unusable) knowledge, and the challenge is to protect learners' creativity and reasoning rather than outsource them. The A2-GLD phases give faculty a concrete route to "design for AI" — preparing customized GenAI tools for specific generative learning activities, with the instructor explicitly present in the Human Scaffolding phase. ## Connected Concepts - [[instructional-design]] - [[self-regulated-learning]] - [[curriculum-design]] - [[math-education]] - [[constructivist]] - [[prompt-engineering]] - [[formative-assessment]] - [[affective-tutoring]] ## Connected Articles - [[llm-intervention-design-cs-review]] — A review of intervention designs of LLM Integration in Undergraduate Computer Science Education - [[learnity-graphs-lifelong-learning-framework-2026]] — Rethinking Higher Education: From Fixed Curricula to Learnity Graphs - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[genai-marketing-education-roles-2026]] — When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education ## Citation Dabbagh, N., & Fake, H. (2026). [*Generative AI (GenAI) as a mindtool that supports generative learning (GL)*](https://doi.org/10.1016/j.caeai.2026.100626). --- ## [Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability](https://edtechdev.github.io/aied/articles/genai-minoritized-knowledges-disability/) ## Overview This paper argues that [[generative-ai]] systems in [[higher-ed]] are not epistemically neutral — they actively marginalize non-hegemonic ways of knowing. Drawing on educational sciences, critical technology studies, and disability studies, Tali-Otmani demonstrates how predominantly Anglophone and Western-centric training data reinforces epistemic coloniality. The situation of persons with disabilities provides a particularly clear illustration, where technological architectures confine them to reductive stereotypes or exclude them from the design process entirely. ## Double Marginalization The paper identifies a **double marginalization** for disabled learners: first, their epistemologies are underrepresented in AI training data; second, they are excluded from the design process that shapes AI tools used in education. This compounds existing [[equity]] challenges documented in [[digital-literacy-illusion]] research, where surface-level AI proficiency masks deeper exclusion. The concept extends [[equity]] frameworks by focusing on knowledge production rather than just access. ## Epistemic Coloniality Training data predominantly sourced from Anglophone, Western academic traditions means that [[generative-ai]] outputs reflect and reinforce those epistemic frameworks. When deployed in higher education without critical awareness, these systems can present non-Western or disability-centered knowledge as less legitimate. This connects to concerns in [[genai-assessment-governance]] about whose standards govern AI use in educational settings. ## Proposed Hybridization Tali-Otmani explores whether a **researcher-machine hybridization** could preserve epistemic plurality. Rather than rejecting AI tools outright, she examines whether collaborative human-AI processes might surface marginalized perspectives. However, she warns against treating [[bias-mitigation]] through algorithmic correction as a purely palliative strategy — structural limitations persist when the underlying training data and design processes remain unchanged. ## Implications for AIED For the [[special-education]] AIED community, this paper raises the stakes beyond accessibility to knowledge legitimacy. AI tools for disabled learners must not only be accessible but must also amplify rather than suppress their epistemological contributions. This requires rethinking [[ai-literacy]] to include critical awareness of whose knowledge is being represented and validated. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[equity]] - [[equity]] - [[bias-mitigation]] - [[special-education]] - [[ai-literacy]] ## Connected Articles - [[digital-literacy-illusion]] - [[genai-assessment-governance]] ## Citation Tali-Otmani, F. (2026). [*Generative artificial intelligence and the marginalization of minoritized knowledges in higher education: The case of disability*](https://arxiv.org/abs/2605.26769). arXiv:2605.26769. --- ## [Examining the Impact of Generative AI on Student Motivation and Engagement: The Mediating Role of Autonomy-Support and Autonomous Motivation in Education](https://edtechdev.github.io/aied/articles/genai-motivation-engagement-2026/) > **Synthesis:** Ahmed and Sultan (2026) investigated how perceived autonomy, competence, relatedness, expectancy, and value influence autonomy support for AI use, autonomous motivation, and ultimately student motivation and engagement in GenAI-supported learning. Integrating Self-Determination Theory, Expectancy-Value Theory, and the Technology Acceptance Model, and analyzing data from 297 undergraduates and postgraduates at King Saud University (Saudi Arabia) with PLS-SEM, they found that autonomy support and autonomous motivation significantly increased student motivation, which emerged as the strongest predictor of student engagement. Perceived expectancy showed no significant influence, and perceived competence did not significantly affect autonomy support. ## Key Findings - Perceived autonomy, perceived relatedness, and perceived value significantly enhanced autonomy support for AI use, while perceived autonomy, competence, and relatedness positively influenced autonomous motivation for AI use. - Autonomy support and autonomous motivation significantly increased student motivation, which subsequently emerged as the strongest predictor of student engagement. - Perceived expectancy showed no significant influence on either autonomy support or autonomous motivation, and perceived competence did not significantly affect autonomy support. - The study integrates SDT, EVT, and TAM within a single framework to explain student engagement in generative AI-supported learning environments, addressing the gap of single-theory models. - It provides context-specific evidence from Saudi higher education, extending AI-in-education research beyond Western and technologically advanced contexts. ## Study Design & Method A quantitative research design was used with data from 297 undergraduate and postgraduate students at King Saud University. The proposed model was analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM). Constructs spanned perceived autonomy, competence, relatedness, expectancy, and value (from SDT and EVT); autonomy support for AI use and autonomous motivation for AI use (contextualized mediators); and student motivation and student engagement (outcomes). The integrated framework allowed the authors to trace how psychological need satisfaction and technology acceptance perceptions translate into motivation and engagement in AI-supported learning. ## Implications for AI in Education The findings indicate that the motivational payoff of [[generative-ai]] depends on satisfying students' basic psychological needs — [[self-determination-theory|autonomy, competence, and relatedness]] — and on supporting autonomous motivation, rather than on expectancy or simple perceptions of competence. For educators and instructional designers, this argues for GenAI integration that enhances learner autonomy (flexible paths, student choice), competence (feedback and skill development), and relatedness (collaborative, inclusive opportunities), since these psychological supports drive the [[motivation]] that most strongly predicts [[engagement-metrics|engagement]]. The study offers practical guidance for implementing GenAI in ways that promote meaningful, sustainable [[student-experience|student engagement]] in higher education. ## Limitations The cross-sectional design limits causal inference, and the sample is drawn from a single Saudi university, bounding generalizability to other national and institutional contexts. All measures are self-report, and the study's cross-sectional PLS-SEM analysis cannot establish temporal ordering among mediators and outcomes. The integration of three theories, while comprehensive, relies on the specific operationalization of autonomy support and autonomous motivation for AI use developed for this context. ## Connected Concepts - [[generative-ai]] - [[motivation]] - [[self-determination-theory]] - [[engagement-metrics]] - [[higher-ed]] - [[personalized-learning]] - [[student-experience]] ## Connected Articles - [[ai-availability-student-motivation]] — "Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns - [[same-ai-different-pathways]] — Same AI, Different Pathways: Unpacking Mechanisms of AI-Mediated Learning - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning When Using Generative AI - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI ## Citation Ahmed, A., & Sultan, A. (2026). [*Examining the impact of generative AI on student motivation and engagement: The mediating role of autonomy-support and autonomous motivation in education*](https://doi.org/10.3389/fpsyg.2026.1852265). --- ## [From Enhancement to Over-Reliance: A Mixed-Method Study of Generative AI and Sustainable Learning Performance](https://edtechdev.github.io/aied/articles/genai-over-reliance-learning-2026/) > **Synthesis:** Gao, Sun, and Khan (2026) developed a dual-pathway model examining both the positive and negative effects of generative AI use on sustainable learning performance, integrating AI literacy, self-regulated learning, cognitive offloading, and individual differences (polychronicity). Using a mixed-method design with three-wave time-lagged survey data from 623 Chinese university students plus educator interviews, they found that AI literacy significantly enhances critical AI evaluation, which — along with self-regulated learning — promotes effective AI use. Effective AI use positively influences sustainable learning performance but also increases AI over-reliance, which negatively affects learning outcomes, with polychronicity moderating key relationships. ## Key Findings - AI literacy significantly enhances critical AI evaluation, which, along with self-regulated learning, promotes effective AI use. - Effective AI use positively influences sustainable learning performance but also increases AI over-reliance, which negatively affects learning outcomes — the dual, or two-edged, nature of AI use. - Polychronicity (multitasking tendency) moderates key relationships, shaping both AI dependency and learning effectiveness: students high in polychronicity are at greater risk of over-reliance. - The qualitative findings from educator interviews support and explain the quantitative results, highlighting the behavioral mechanisms underlying AI-supported learning. - The study's central contribution is recognizing two parallel processes coexisting in AI use: creating sustainable learning opportunities while also fostering cognitive over-dependency on AI. ## Study Design & Method The study employed a mixed-method design combining three-wave time-lagged survey data from 623 university students in China with qualitative interviews with educators. The quantitative strand used PLS-SEM to test the dual-pathway model linking AI literacy, critical AI evaluation, self-regulated learning, effective AI use, AI over-reliance, and sustainable learning performance, with polychronicity as a moderator. The qualitative strand used thematic analysis of educator interviews to deepen and explain the observed relationships. The theoretical framework integrated the AI literacy framework, Self-Regulated Learning Theory, and Cognitive Offloading Theory. ## Implications for AI in Education The findings reframe AI's impact on learning as inherently dual — capable of both enhancement and [[over-reliance]] — and locate the determining factors in how students engage with the technology. For educators, this argues for building [[ai-literacy]] (the capacity to evaluate AI output critically and use it meaningfully) and [[self-regulated-learning]] skills so students use AI as a scaffold rather than a substitute, and for attending to individual differences such as [[motivation|polychronicity]] that shape dependency risk. The study connects AI use to [[cognitive-offloading]] risks, warning that habitual delegation of cognitive processing can erode the internal effort needed for deep learning, and offers implications for designing interventions that maximize the enhancement pathway while mitigating [[ai-misuse-learning-harm|over-reliance]]. ## Limitations The cross-sectional, self-report survey design limits causal inference despite the time-lagged structure. The sample is drawn from Chinese university students, bounding generalizability. Polychronicity and the AI-literacy/AI-evaluation constructs rely on self-report operationalizations. The qualitative strand, while informative, is limited to educator perspectives and does not directly capture students' behavioral mechanisms. ## Connected Concepts - [[generative-ai]] - [[over-reliance]] - [[ai-literacy]] - [[self-regulated-learning]] - [[cognitive-offloading]] - [[ai-misuse-learning-harm]] - [[higher-ed]] ## Connected Articles - [[ai-autonomous-learning-accomplishment-2026]] — AI-Assisted Autonomous Learning and Reduced Academic Accomplishment - [[ai-use-critical-thinking-medical-students-2026]] — From AI Use to Critical Thinking Among Medical Students - [[lodge-loble-cognitive-offloading-2026]] — Artificial Intelligence, Cognitive Offloading and Implications for Learning - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning When Using Generative AI - [[genai-skill-bypass-literacy]] — The GenAI Skill Bypass: Mapping Divergent Pathways of Use ## Citation Gao, L., Sun, Y., & Khan, S. U. (2026). [*From enhancement to over-reliance: A mixed-method study of generative AI and sustainable learning performance*](https://doi.org/10.3389/fpsyg.2026.1847369). --- ## [Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks](https://edtechdev.github.io/aied/articles/genai-patient-education-transplant-handbooks/) Li, Padman and Krishnan audit 102 US transplant-center patient handbooks that serve as grounding corpora for generative AI patient-education assistants. They show large institutional heterogeneity in the underlying education materials, undermining the premise that grounding a genAI assistant in local content yields consistent guidance: patients at different institutions can receive materially different AI-mediated answers to the same question. The study extends grounding-quality concerns familiar from [[retrieval-augmented-tutoring-algorithm-kite]] into health education, and connects to AI-driven medical training work such as [[medeasy-ai-standardized-patients]], [[adaptive-virtual-patient-psychotherapy-training]] and [[medgame-llm-medical-education-gamification]]. The equity implication (institution-dependent quality of AI-mediated education) parallels [[equity-in-ai-education]]. ## Connected Concepts - [[equity-in-ai-education]] ## Connected Articles - [[retrieval-augmented-tutoring-algorithm-kite]] - [[medeasy-ai-standardized-patients]] - [[adaptive-virtual-patient-psychotherapy-training]] - [[medgame-llm-medical-education-gamification]] ## Citation Yubo Li, Rema Padman, Ramayya Krishnan (2026). [Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks](https://arxiv.org/abs/2607.22606). arXiv:2607.22606. --- ## [Efficacy of an Intensive Generative AI Professional Development Program on Pedagogical Content Knowledge (AI-PCK) and the Comparative Analysis of Learning Gain between Experienced and Pre-service Teachers](https://edtechdev.github.io/aied/articles/genai-pd-ai-pck-learning-gain-2026/) > **Synthesis:** This quasi-experimental study of an intensive 8-hour generative-AI professional development program with 163 teachers and pre-service teachers found significant gains across all five AI-PCK components (overall *d* = 2.36), with pre-service teachers showing statistically higher learning gains than experienced teachers (*p* = 0.033). ## Key Findings 1. **Intensive 8-hour PD, 163 participants.** A quasi-experimental pretest-posttest design assessed an intensive 8-hour GenAI professional development program on five core AI-PCK components. 2. **Large significant effects.** The PD program was significantly effective across all five AI-PCK components (*p* < 0.001), with the highest effect size in the Rubric Assessment component (*d* = 2.19) and an overall effect size of *d* = 2.36. 3. **Pre-service teachers gained more.** Pre-service teachers demonstrated a statistically higher overall PCK learning gain than experienced teachers (*p* = 0.033). 4. **Design implications.** The intensive model is highly efficacious for rapid AI-PCK enhancement, but the differential learning gain highlights the need to integrate AI-PCK components into core teacher-training curricula and to design specialized support. ## Implications This study provides strong evidence that intensive, focused professional development can rapidly build [[teacher-ai-competency]] in [[generative-ai]]-related pedagogical content knowledge. The very large effect sizes (*d* = 2.36 overall) support the [[efficacy-study]] case for structured, short-format training programs in a fast-moving area where teachers urgently need updated skills. The differential gain favoring [[professional-training]] candidates suggests that embedding AI-PCK into pre-service curricula may be particularly efficient, while experienced teachers may need tailored, specialized support rather than one-size-fits-all PD. This connects directly to [[professional-training]], [[faculty-development]], and [[professional-training]] debates about how to upskill educators for AI-integrated teaching. For [[professional-training]] and institutional [[educational-policy-ai]], the findings argue for systematic rather than ad-hoc AI teacher preparation, and for attention to differentiated learning needs across the teacher-career spectrum. ## Connected Concepts - [[efficacy-study]] - [[faculty-development]] - [[generative-ai]] - [[professional-training]] - [[teacher-ai-competency]] - [[professional-training]] - [[professional-training]] - [[professional-training]] ## Connected Articles - [[ai-tpack-teacher-multi-agent-workflow]] — AI-TPACK teacher multi-agent workflow - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review - [[ai-uk-higher-education-policy-2026]] — AI in UK higher education policy - [[talebzadeh-ai-green-education-2026]] — AI in green education ## Citation Talebzadeh, H. (2026). [*Efficacy of an Intensive Generative AI Professional Development Program on Pedagogical Content Knowledge (AI-PCK) and the Comparative Analysis of Learning Gain between Experienced and Pre-service Teachers*](https://osf.io/njztk). EdArXiv preprint. --- ## [Distinguishing performance gains from learning when using generative AI](https://edtechdev.github.io/aied/articles/genai-performance-vs-learning/) ## Core Argument This *Nature Reviews Psychology* piece draws a critical distinction that has been under-theorized in AIED research: - **Performance gains** — immediate improvements in task completion, efficiency, or output quality when using AI tools. - **Learning** — durable understanding that requires deep cognitive processing (elaboration, critical analysis) and metacognitive processing (planning, monitoring, reflection). The authors argue that generative AI easily boosts performance but often bypasses the mental processes essential for genuine knowledge acquisition. This challenges the common assumption in [[ai-tutor-effectiveness-review]] research that improved task performance equals improved learning. ## Implications for AIED Design This distinction has profound implications for [[scaffolding]] design. Systems that optimize for immediate performance may undermine learning. They must be designed to promote cognitive engagement — for example, through [[socratic-method]] dialogue, constrained feedback, or requiring student articulation before revealing AI output. The piece connects to [[over-reliance]] research showing that AI assistance can reduce independent problem-solving and to the [[pedagogy-ai-mistakes]] paradigm that uses AI errors as learning opportunities. The performance-vs-learning gap is now causally demonstrated in a field RCT: [[generative-ai-guardrails-harm-learning]] shows that an unguarded GPT-4 tutor raised practice performance +48% but *reduced* later unassisted exam scores by 17%, while a guardrailed "hint-not-answer" tutor eliminated the harm. ## Theoretical Framework Generative AI can function as either a cognitive tool (amplifying thinking) or a cognitive crutch (replacing thinking). This maps onto the [[self-regulated-learning]] cycle — performance gains without metacognitive engagement short-circuit the planning-monitoring-evaluating loop. ## Connected Concepts - [[scaffolding]] - [[socratic-method]] - [[over-reliance]] - [[self-regulated-learning]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[pedagogy-ai-mistakes]] - [[generative-ai-guardrails-harm-learning]] ## Citation Yan, L., Greiff, S., Lodge, J. M., & Gašević, D. (2026). Distinguishing performance gains from learning when using generative AI. [Nature Reviews Psychology](https://arxiv.org/abs/2605.13731), 4(7), 435-436. arXiv:2605.13731. --- ## [A Comparative Analysis of Institutional and Course Generative AI Policies within Higher Education: Implications for Instruction in Computing Education](https://edtechdev.github.io/aied/articles/genai-policies-higher-ed-computing/) > **Synthesis:** A comparative content analysis of institutional GenAI policies and computing-course syllabi in U.S. research-intensive universities, revealing a gap between broadly pro-use institutional guidance and guarded, often prohibition-heavy classroom-level uptake. ## Key Findings - **Data set:** secondary analysis of **116 institutional GenAI policies** (from 131 R1 U.S. universities screened) and **98 computer-science course syllabi** (from 54 R1 institutions), content-coded and then mapped across levels. - **Institutional guidance is broadly pro-use:** a majority of universities (N = 73, 63%) encourage GenAI use, with 41% (N = 48) offering detailed classroom guidance, while just over a quarter (N = 31, 27%) discourage it. More than half (N = 64, 55%) stipulate syllabus statements using a range-of-use framing ("embrace," "limit," "prohibit"). - **Course-level uptake is far more guarded:** almost all syllabi (92%, N = 90) give explicit use guidelines, but **half (50%, N = 49) outright prohibit GenAI use**, 41% (N = 40) permit only partial use for specified activities, and few communicate encouragement (7%, N = 7). - **The top-down vs. bottom-up gap:** institutions are comparatively supportive of GenAI while individual computing instructors often restrict or prohibit it, leaving instructors to improvise local policies that may not align with institutional guidance — the central coordination problem for [[ai-governance-education]]. - **Shared emphasis on transparency:** both levels stress citation and acknowledgement — 83% (N = 81) of syllabi require citing GenAI use (over two-thirds treat uncredited use as an honor-code or academic-integrity violation), and 38% (N = 44) of institutions provide formal citation guidance, most often referencing APA. - **Institutional guidance is uniquely curricular and ethical:** institutions, not syllabi, emphasize curriculum design (50% ask instructors to reflect on teaching and assessment; 29% encourage GenAI for lesson planning) and ethics, including Diversity/Equity/Inclusion (52%), privacy (57%), and classroom ethics discussions (53%). ## Study Design & Method The study performs a **secondary/document analysis** comparing institutional policy documents against computing-course syllabi. Institutional guidance was collected from 116 R1 universities (Carnegie-classified research-intensive, collected through late 2023); course guidance came from 98 computer-science syllabi at 54 R1 universities (collected March–May 2024), chosen because CS is an early and heavy adopter of GenAI. Two researchers coded the institutional corpus with a 13-code codebook (inter-rater reliability established on a random sample); three researchers coded the syllabi with a 16-code codebook. A three-step comparative analysis then examined coverage comprehensiveness, mapped institutional codes against course-level codes to identify similarities and gaps, and presented practice examples for key codes. ## Key Results - **Alignment is partial.** Among 131 R1 institutions, 116 had institutional guidelines, 54 had course-level guidelines, and only 47 had both — meaning a large share of institutions with top-level policy had no detectable course-level translation in computing syllabi. - **Use-permission asymmetry.** Institutions mostly *allow and encourage* (63%) while courses mostly *restrict or prohibit* (50% outright, plus 41% partial-use); few syllabi explicitly encourage use (7%). - **Range-of-consent categories** recur at both levels (e.g., "embrace/limit/prohibit" vs. "no use/some use/unlimited use/required use"), but the *direction* differs — institutional guidance skews permissive, course guidance skews restrictive. - **Anthropomorphism appears only at course level:** 39% (N = 38) of syllabi describe GenAI in human terms — as a "tutor," "coach," or "assistant" — reflecting faculty and student mental models of the tools. - **CS as the policy canary:** institutional guidance concentrates on STEM and especially computer science (48% of institutions mention CS), consistent with GenAI's strength in programming and its training on coding corpora (StackOverflow, GitHub, online repositories). ## Implications for AI in Education For [[ai-governance-education]], the finding that institutional guidance is pro-use while classroom practice is restrictive suggests that effective governance requires **translating policy into concrete instructor support**, not just issuing top-down documents — otherwise instructors improvise inconsistent local rules. For computing education specifically, the study positions CS as the canary discipline for GenAI adoption: high student uptake forces instructors to confront policy gaps first. The authors characterize both levels as sending "mixed messages" because institutions and educators are **playing catch-up to student use**, compounded by fast-moving, vendor-controlled access. The gap between policy intent and classroom practice connects to [[educational-policy-ai]] and [[equity]] debates about who sets AI-use rules and how consistently they apply across courses and students, and to [[teacher-ai-adoption-confidence]] — uneven instructor readiness to translate guidance into everyday practice. ## Limitations - The data set is **restricted to R1 universities in the U.S.** (Carnegie classification), limiting generalizability to other institution types (liberal arts colleges, community colleges) and to higher-education systems outside the U.S., where regulatory environments and institutional cultures around GenAI may differ substantially. - The course-level analysis covers **computer science only** (a deliberate design choice to study early adopters), so findings may not transfer to other disciplines; the authors note most prior literature spans multiple domains and call for cross-discipline comparisons. - Institutional policies and course syllabi were **collected in different periods** (late 2023 vs. spring 2024), during which GenAI tools and guidance evolved rapidly. ## Connected Concepts - [[higher-ed]] - [[ai-education]] - [[ai-governance-education]] - [[educational-policy-ai]] - [[equity]] ## Connected Articles - [[teacher-ai-adoption-confidence]] ## Citation Ganguly, A., Johri, A., McDonald, N., Ali, A., et al. (2026). [A Comparative Analysis of Institutional and Course Generative AI Policies within Higher Education: Implications for Instruction in Computing Education](https://arxiv.org/abs/2607.12296). arXiv:2607.12296. --- ## [Associations Between Generative AI–Based Pronunciation Feedback and Willingness to Communicate in English: The Mediating Role of English Pronunciation Self-Efficacy](https://edtechdev.github.io/aied/articles/genai-pronunciation-feedback-wtc-2026/) > **Synthesis:** Lu et al. (2026) examined, through the lens of Social Cognitive Theory, whether Chinese university EFL learners' perceptions of generative-AI-based pronunciation feedback relate to their willingness to communicate (WTC) in English, with English pronunciation self-efficacy as a hypothesized mediator. Using a cross-sectional survey of 1,701 learners, covariance-based structural equation modeling, and bias-corrected bootstrapping, they found that positive perceptions of GenAI pronunciation feedback were significantly associated with greater WTC in English, and that pronunciation self-efficacy partially mediated this relationship — the indirect effect accounted for 69.9% of the total effect while the direct effect remained significant. ## Key Findings - Positive perceptions of generative-AI-based pronunciation feedback were significantly associated with greater willingness to communicate in English among Chinese EFL learners. - English pronunciation self-efficacy partially mediated the association: the indirect effect accounted for 69.9% of the total effect, while the direct effect of GenAI feedback perception on WTC remained significant. - The results suggest pronunciation self-efficacy is an important but not exclusive mechanism — GenAI feedback may also relate to WTC through other routes such as reduced communication anxiety and enhanced psychological safety. - The study conceptualizes GenAI feedback as a technology-mediated environmental influence and pronunciation self-efficacy as a personal cognitive factor within [[self-regulated-learning|Bandura's Social Cognitive Theory]], extending the framework to the Chinese EFL context and to intelligent feedback technologies. ## Study Design & Method The authors used a cross-sectional survey design with a convenience sample of 1,701 Chinese university EFL learners. Covariance-based structural equation modeling (CB-SEM) and bias-corrected bootstrapping were used to test the hypothesized relationships and the mediating effect of pronunciation self-efficacy on the perception-of-GenAI-feedback → WTC path. The analysis distinguished the indirect effect (through self-efficacy) from the direct effect of feedback perception on willingness to communicate, and quantified the indirect effect's share of the total effect. ## Implications for AI in Education The findings position GenAI-based pronunciation feedback as a promising, low-pressure supplement to traditional teacher and peer feedback for [[language-learning]], particularly in contexts (like China) where limited authentic English interaction and "mute English" make pronunciation anxiety a barrier to speaking. Because GenAI feedback is immediate, repeatable, personalized, and perceived as less judgmental, it may build learners' pronunciation self-efficacy and, through it, their readiness to speak. For practitioners, this supports integrating AI pronunciation tools as ongoing speaking practice that complements — not replaces — classroom feedback, while the partial mediation highlights that confidence and psychological safety, not just accuracy, are central to why such feedback helps learners communicate. ## Limitations The cross-sectional design precludes causal inference, and the convenience sample of Chinese university EFL learners bounds generalizability to other populations and contexts. All constructs (perceptions of feedback, pronunciation self-efficacy, WTC) are self-report measures collected at a single time point, so the mediating role of self-efficacy is inferred from covariance rather than manipulated. The study focuses on perceptions of GenAI feedback rather than objective feedback quality or its actual behavioral effects on speaking performance. ## Connected Concepts - [[language-learning]] - [[generative-ai]] - [[ai-feedback-quality]] - [[self-regulated-learning]] - [[motivation]] - [[feedback-loop]] - [[personalized-learning]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] — AI-Guided Learning: Research on Knowledge and Skill Acquisition - [[multimodal-ai-feedback-learning]] — LLM-Based Multimodal AI Feedback Produces Equivalent Learning - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[cyberscholar-genai-writing-feedback]] — Generative AI Feedback, English Writing and Teacher Rubrics - [[genai-feedback-design-multisite-experiment]] — Human-Centered GenAI Feedback Design in Higher Education ## Citation Lu, Y., Yang, Y., Cui, T., Yang, Z., Cai, Y., & Jing, B. (2026). [*Associations between generative AI–based pronunciation feedback and willingness to communicate in English: The mediating role of English pronunciation self-efficacy*](https://doi.org/10.3389/fpsyg.2026.1918980). --- ## [Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS)](https://edtechdev.github.io/aied/articles/genai-reliance-types-scale/) As generative AI (GenAI) becomes embedded in undergraduate academic writing, *how* students rely on these tools — not merely whether they use them — has emerged as a core question for [[academic-integrity]], [[student-experience]], and educational equity. This study develops and validates the **Generative AI Reliance Types Scale (GenAI-RTS)**, a 20-item instrument measuring four theoretically derived reliance types: **Strategic**, **Instrumental**, **Dependent**, and **Dialogic**. Confirmatory factor analysis supports a five-factor structure in which Strategic Reliance splits into *Deliberate Use* and *Critical Evaluation* (CFI = .92, RMSEA = .08; DWLS CFI = .98, RMSEA = .07), with subscale reliability (ω) ranging .75–.90. Validation follows the multi-source framework of the *Standards for Educational and Psychological Testing*, combining a survey of 382 undergraduates at a U.S. Minority-Serving Institution with interviews with 14 purposively sampled students. The **Critical Evaluation** facet — students scrutinizing GenAI output rather than accepting it — directly operationalizes [[ai-literacy]] in writing contexts, while the **Dependent** type maps onto documented patterns of [[over-reliance]] on [[generative-ai]] tools. The instrument gives researchers and instructors a validated way to measure reliance modes in [[writing-education]] and across [[higher-ed]], moving beyond binary “use/no-use” measures toward nuanced, equity-aware assessment of GenAI integration. ## Connected Concepts - [[academic-integrity]] - [[student-experience]] - [[ai-literacy]] - [[over-reliance]] - [[generative-ai]] - [[writing-education]] - [[higher-ed]] ## Connected Articles - [[genai-usage-design-students-survey]] — A study of GenAI usage by Design Students: Analysis of Survey Results and Journals of AI practices at the Politecnico di Milano in 2025/2026 - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation in Introductory Programming ## Citation Shahin Hossain, Tukhbita Afroz Nawmi (2026). [Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS)](https://arxiv.org/abs/2607.14301). arXiv:2607.14301. --- ## [GenAI as a runaway object in higher education: A socio-cultural view on AI-influenced academic practice in mathematics](https://edtechdev.github.io/aied/articles/genai-runaway-object-math-higher-ed/) > **Synthesis:** GenAI as a runaway object in higher education: A socio-cultural view on AI-influenced academic practice in mathematics ## Key Findings - A qualitative study of **ten mathematics academics at a Swedish university**, analyzed through **Cultural-Historical Activity Theory (CHAT)**, examines how GenAI reshapes the interrelated activity systems of teaching and research. - GenAI is conceptualized as a **"runaway object"** — a technology that evolves unpredictably, disrupts boundaries, and reconfigures roles, tools, and epistemic norms. - Five dominant themes characterize emerging change across both activity systems: **boundary fluidity, shifting objects, epistemic concerns, distributed innovation, and reconfigured roles**. - In the research activity system, GenAI was most commonly described as a peripheral support tool used to streamline tasks, with most participants reporting it had not yet penetrated the core processes of scientific knowledge development; in teaching, participants described a shift in the teacher's role from content expert toward facilitator and mediator of critical reasoning, with students portrayed as early adopters who bring new tools, answers, and expectations into the classroom. - Adoption unfolded **largely through informal and distributed processes**, often associated with student use and peer networks rather than institutional policy, and extended across both teaching and research — a pattern the authors read as evidence of the uneven, hard-to-control diffusion characteristic of runaway objects. - Epistemic concerns were especially pronounced in teaching: participants described a tension between the convenience of AI-generated outputs and the deeper purpose of learning mathematics, warning of a "fake sense of being good at something" when polished output replaces genuine struggle, and of surface-level understanding of methods and assumptions. - The findings suggest GenAI is beginning to influence **how academic work is carried out and evaluated**, contributing to the blurring of established boundaries and to shifts in how academics orient their work. ## Study Design & Method The study draws on qualitative data from ten academics in a mathematics department at a Swedish university. Data were collected in the early weeks of the spring semester of the 2024–2025 academic year (January–March), at an early stage of GenAI adoption; the primary source was semi-structured individual interviews of approximately one hour each, audio-recorded and fully transcribed. Analysis followed a thematic approach inspired by Braun and Clarke (2006), informed by CHAT constructs (Engeström, 1987), combining inductive coding grounded in participants' accounts with deductive, activity-theoretical coding; themes were assessed for robustness through recurrence across the ten interviews. Using CHAT as its analytical lens, the study treats teaching and research as interrelated activity systems and examines how GenAI acts as a runaway object within them. By highlighting early-stage dynamics, the analysis demonstrates the analytical value of this lens for examining how GenAI is being negotiated within mathematics teaching and research in [[higher-ed]]. ## Implications for AI in Education For [[higher-ed]] institutions and [[math-education]] departments, the findings suggest that formal policy is lagging behind practice: because adoption is driven by students and peer networks, institutions may be designing governance for a technology that is already reshaping academic work from below — and participants themselves called for shared institutional frameworks so that "every teacher" is not forced to invent their own approach. The five themes give [[faculty-development]] and institutional strategy a vocabulary for responding — acknowledging epistemic concerns and reconfigured roles rather than treating GenAI as a neutral tool — and for deciding where [[educational-policy-ai]] should intervene. The observed shifts in [[teacher-role]] — from content expert to mediator of critical engagement, with students sometimes more fluent in the tools than their instructors — point to concrete faculty-development needs around assessing meaningful engagement and maintaining pedagogical control in AI-supported classrooms. ## Limitations The study is a qualitative, activity-theoretical case study at a single institution within the Swedish higher education context, where relatively high levels of institutional trust and proactive engagement with AI are evident; the observed dynamics therefore reflect a specific configuration of rules, norms, and community relations and are not representative of other settings. Data were collected at one point in time in the early stage of GenAI adoption, so findings are context-bound and in flux — practices that currently appear peripheral, supportive, or informal, particularly in research, may become more embedded or differently regulated over time. The authors also caution that the findings identify empirical patterns consistent with a runaway-object perspective rather than fully demonstrating GenAI as a runaway object in the strong sense proposed by Engeström. ## Connected Concepts - [[math-education]] - [[generative-ai]] - [[higher-ed]] - [[stem-education]] - [[faculty-development]] ## Connected Articles - [[ai-tpack-preservice-math-teachers]] - [[epistemic-proactivity-math]] ## Citation Bakogianni, D., Liljekvist, Y., & Bui, P. (2026). [*GenAI as a runaway object in higher education: A socio-cultural view on AI-influenced academic practice in mathematics*](https://doi.org/10.1016/j.caeai.2026.100637). --- ## [The GenAI Skill Bypass: Mapping Divergent Pathways of University Students and Staff AI Literacy](https://edtechdev.github.io/aied/articles/genai-skill-bypass-literacy/) Higher education institutions are increasingly expected to ensure that both students and staff develop Generative AI (GenAI) literacies. In response, they are introducing professional development programs and embedding GenAI skills within student curricula. However, current educational frameworks typically assume a linear progression of GenAI literacy, implying that foundational technical understanding must precede creative application. This paper challenges such an assumption through a psychometric analysis of a taxonomy-based self-assessment instrument (n = 158). We applied Rasch measurement theory and Guttman ordering to map the latent perceived order of difficulty of GenAI skills across students, academics, and professional staff. Results reveal a fundamental divergence in perceived competence profiles: while academics follow a more traditional linear path, students exhibit an inverted profile, frequently mastering high-level creation tasks before acquiring foundational conceptual understanding. Furthermore, the correlation of skill difficulty between students and academics was weak (r = 0.188). We argue that this skill bypass creates a fragile sense of fluency, where high self-efficacy in prompting masks low literacy in AI mechanics. These findings challenge the one-size-fits-all curricula and provide the empirical basis for diagnostic-driven, modular interventions that foster genuine human-AI synergy. Rasch analysis of n=158 GenAI-literacy self-assessments reveals students show an inverted skill profile (mastering creation before conceptual foundations), weak correlation with academics (r=0.188); a skill bypass gives fragile fluency where prompting self-efficacy masks low AI-mechanics literacy, arguing against one-size-fits-all curricula. This work connects to core wiki themes: [[ai-literacy]] [[higher-ed]] [[over-reliance]] [[generative-ai]] [[faculty-development]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[over-reliance]] - [[generative-ai]] - [[faculty-development]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-making-us-stupid]] — Is AI making us stupid? - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs ## Citation Oliveira, English, Ryan, Misiejuk, dal Ponte, Lopez-Pernas & Saqr (2026). [The GenAI Skill Bypass: Mapping Divergent Pathways of University Students and Staff AI Literacy](https://arxiv.org/abs/2607.05411). arXiv:2607.05411. arXiv preprint. --- ## [\"It is a temptation to get it to do the work…\" Student Experiences of Navigating the Generative AI Landscape in UK Higher Education: A Cross-Institutional Survey with International Comparison](https://edtechdev.github.io/aied/articles/genai-student-experiences-uk-he-survey-2026/) > **Synthesis:** The StudentXGenAI Project surveyed more than 7,000 students across 7 UK institutions (September–December 2025) on GenAI use in their studies, comparing findings with a companion Australian survey. A significant minority of students conscientiously object to GenAI use, while most users are honest most of the time and try to avoid submitting direct GenAI outputs — yet students still use GenAI throughout the entire learning and assessment process, creating a persistent tension between use and integrity. ## Key Findings 1. **Conscientious objectors.** Despite discourse that "everyone is using it," a significant minority of students deliberately refuse to use GenAI. 2. **Mostly honest but pervasive use.** For students who do use GenAI, the majority are honest most of the time and try to avoid submitting direct GenAI outputs, yet they use it for a range of tasks throughout the entire learning and assessment process. 3. **International convergence.** The UK and Australian findings are strikingly similar yet nuanced, suggesting shared student norms and tensions across institutions and countries. ## Implications For [[student-experience]] and [[higher-ed]] practice, the survey provides large-scale evidence that students are not uniformly outsourcing work — many regulate their own use, and a notable minority object outright. This complicates simple [[academic-integrity]] narratives and supports evidence-based, student-voice-driven [[educational-policy-ai]] rather than blanket prohibition. The tension between honest self-regulation and pervasive use across the assessment lifecycle bears on [[assessment]] design and on [[ethics]] education: institutions need policies that acknowledge legitimate [[generative-ai]] use while protecting learning. The multi-institution, internationally compared design makes this a strong empirical anchor for [[ai-governance-education]] and institutional practice. ## Connected Concepts - [[student-experience]] - [[higher-ed]] - [[generative-ai]] - [[academic-integrity]] - [[equity-in-ai-education]] - [[ethics]] - [[educational-policy-ai]] - [[motivation]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI writing - [[ai-assessment-human-tutors]] — AI assessment compared with human tutors - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] — Students' engagement with GenAI (self-determination) - [[ai-higher-ed-workforce-survey]] — AI in higher-ed workforce survey - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review ## Citation Gow, S., Illingworth, S., Fabian, K., & Goddard, C. (2026). [*"It is a temptation to get it to do the work…" Student experiences of navigating the generative AI landscape in UK higher education: A cross-institutional survey with international comparison*](https://osf.io/preprints/edarxiv/bsx47_v2/). EdArXiv preprint. --- ## [Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness](https://edtechdev.github.io/aied/articles/genai-teacher-feedback-comparison/) ## Summary The largest study in the AEHE 51(5) special issue: a **cross-sectional survey across four Australian universities** (≈192,000 invited; 10,132 volunteered; this paper analyses **6,960 students** who answered the feedback items). It combines quantitative comparison of perceived helpfulness/trustworthiness of GenAI vs teacher feedback with **thematic analysis of 8,642 open-ended responses** (11,903 coded instances, 48 codes). Core conclusion: **GenAI and teacher feedback serve different needs — complementary but not interchangeable**. ## Quantitative findings - **Usage split:** 49.7% (n = 3,461) of students had sought feedback from GenAI; 50.3% had not. - **Helpfulness:** 83.9% rated AI feedback somewhat/very helpful vs 82.2% for teacher feedback — a small but significant teacher advantage (mean diff 0.14, t(3327) = −7.17, p < .001, d = −0.13). - **Trustworthiness:** the striking gap — **90.5% rated teacher feedback somewhat/very trustworthy vs 60.1% for AI** (mean diff 0.93, t(3327) = −49.30, p < .001, d = −0.89, a large effect). 58.3% rated teacher feedback *very* trustworthy vs 8.8% for AI. ## Qualitative findings — why they differ When comparing GenAI to teacher feedback, students said GenAI was **more**: accessible/easy (99.3% of access codes), fast, voluminous, understandable, objective (less biased), and positive in tone — and **less risky** (99.3% of relational-risk codes: less vulnerable, no loss of social status). It aided sense-making and could be used before submission. Teacher feedback was **more**: relevant, contextualised (95.2% of contextualisation codes), specific, in-depth, personal/relational, and expert — but also more frequently **negative in tone** (e.g. dismissive, insulting) and more likely to produce negative feelings (85.3% of negativity codes). The two most frequent comparison themes were the nature of the feedback **information** (54.8% of coded instances; quality, reliability, relevance, contextualisation) and feedback **processes** (22.7%; access, timing, effort, sense-making). ## Why students did NOT use GenAI for feedback (n = 3,405 comments) - **28.1%** were unaware it was possible or did not know how (a support/equity gap, not a preference). - **28.7%** cited trustworthiness/reliability concerns. - **22.5%** cited values: preference for human connection (4.3%) or no perceived need (6.3%). - Smaller shares: academic integrity (9.1%), privacy (2.7%), wanting to preserve effortful learning (18 of 24 effort codes). ## Implications - Students already self-initiate GenAI feedback at scale, so institutions should actively support **how** students engage with it (feedback literacy, evaluative judgement — cf. [[ai-literacy]]). - The trust gap (90.5% vs 60.1%) is not simply an accuracy verdict; it partly reflects source-credibility heuristics (Lipnevich & Smith 2008; Nazaretsky et al. 2024) and may be context-dependent (language-form advice vs deeply contextual course guidance). - The "less risky" property makes GenAI feedback valuable for feedback seeking, especially for anxious students — complementing [[chatgpt-feedback-engagement-genai]]'s "calm, stress-free" affective finding — while teacher feedback retains contextual expertise and relational recognition. - Directly challenges replacement narratives: GenAI is an **additional source**, not a substitute for the teacher-learner relationship (cf. [[ai-generated-feedback-higher-ed]], which found equivalent *outcomes* under strong assessment architecture — outcomes and *perceptions* can diverge). ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[student-experience]] - [[generative-ai]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative a... - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers' practices and perceptions - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education ## Citation Henderson, M., Bearman, M., Chung, J., Fawns, T., Buckingham Shum, S., Matthews, K. E., & de Mello Heredia, J. (2026). Comparing Generative AI and teacher feedback: Student perceptions of usefulness and trustworthiness. [*Assessment & Evaluation in Higher Education*](https://doi.org/10.1080/02602938.2025.2502582), 51(5), 863–878 --- ## [Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use](https://edtechdev.github.io/aied/articles/genai-tutor-engagement-patterns/) > **Authors:** Youjie Chen, Xixi Shi, Xinyu Liu, Shuaiguo Wang, Tracy Xiao Liu, Dragan Gašević **Year:** 2026 **Venue:** arXiv (cs.CY) > Large-scale analysis (N=11,406 students, 200 classes, 10 institutions) of GenAI tutor engagement identifies four session-level engagement types — Deep, Shallow, Routine-Learning, and Exam-Driven — with 10.4% of sessions being shallow copy-paste use and deeper engagement more common at selective institutions. ## Summary Large-scale analysis (N=11,406, 200 classes, 10 institutions) of GenAI tutor engagement. Four engagement types; 10.4% shallow with copy-paste. Students at selective institutions more likely to engage deeply. This study analyzes de-identified interaction logs from a commercial LMS with an integrated GenAI Tutor used across ten post-secondary institutions during the Spring 2025 semester. Among 11,406 students, 6,932 (60.8%) engaged with the tutor at least once, generating a median of 5 conversation sessions per user across 113,255 segmented conversation sessions. Using clustering on behavioral, cognitive, and temporal session features, the authors identified four engagement types, then used process mining at the student level to examine how learners transitioned between them over time. ## Key Contributions - A **two-stage learning-analytics pipeline** that identifies conversation-session-level engagement types via clustering and aggregates them into student-level engagement patterns via process mining (First-Order Markov Model), applicable to other human-AI interaction data. - **Four session-level engagement types** — Deep, Shallow, Routine-Learning, and Exam-Driven — grounded in behavioral, cognitive, and temporal features of student-GenAI conversations. - Evidence on **contextual variation** across institution selectivity and course discipline, reflecting equity concerns about GenAI use in educational settings. - Insights intended to inform learning-analytics interventions and institutional guidance for supporting student learning with GenAI Tutors in more effective and equitable ways. ## The Four Engagement Types - **Deep Engagement (14.0% of sessions)**: multiple conversation turns, longer durations, more words, and a high prevalence of understanding-oriented queries; distributed relatively evenly across the semester. - **Shallow Engagement (10.4%)**: few turns, short durations, few words, with copy-pasting behaviors and direct answer-seeking requests prevalent; somewhat more likely to occur during class. - **Routine-Learning Engagement (44.5%)**: concentrated in daytime and the first half of the semester alongside coursework; few turns but reasonable word counts, with more understanding-oriented questions and fewer direct answer requests. - **Exam-Driven Engagement (31.2%)**: concentrated in the final weeks as exams approached; few, short turns with the least copy-pasting, suggesting fewer needs for formalistic homework-style answers. ## Contextual Variation & Implications Context mattered. At highly selective universities, the proportion of deep engagement (19.36% vs. 12.42%) and routine-learning engagement (58.88% vs. 40.41%) was significantly higher, while exam-driven engagement was lower (10.65% vs. 36.98%). STEM courses showed significantly more shallow (15.41% vs. 8.56%) and routine-learning engagement and less exam-driven engagement than non-STEM courses, while STEM students were overall less likely to adopt the tutor (51% vs. 71% adoption). Adoption and usage intensity were not significantly associated with institutional selectivity. For [[learning-analytics]] and [[equity]] in [[higher-ed]], the takeaway is that "engagement" is not a single behavior: interventions and regulations should target specific patterns — such as shallow, copy-paste-heavy use — rather than treating all GenAI Tutor use alike, especially since students who engaged shallowly were more likely to remain in that mode over time, pointing to a risk group for [[over-reliance]] without deep processing. ## Connected Concepts - [[higher-ed]] - [[affective-computing]] - [[socratic-method]] - [[affective-tutoring]] - [[administrator]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[help-seeking]] ## Connected Articles - [[students-llm-usage-critical-thinking]] — Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration: A Longitudinal Analysis of Prompting Strategies - [[knowledge-gap-detection-ai-tas]] — Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[lets-chat-chatbot-outreach-2026]] — Let''s Chat: Leveraging Chatbot Outreach for Improved Course Performance ## Citation Youjie Chen et al. (2026). [Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use](https://arxiv.org/abs/2602.00447). arXiv:2602.00447. cs.CY. --- ## [A study of GenAI usage by Design Students: Analysis of Survey Results and Journals of AI practices at the Politecnico di Milano in 2025/2026](https://edtechdev.github.io/aied/articles/genai-usage-design-students-survey/) This survey of design students at the Politecnico di Milano (2025/2026), paired with AI-use journals kept during research assignments, examines how [[generative-ai]] enters the design process. Reported use is very frequent and concentrated in the early, ideation-heavy stages of projects. Crucially, high GenAI adoption did not reduce students' perceived project ownership or creativity, a finding relevant to [[higher-ed]] policy and the [[student-experience]] of AI-augmented studio work. The study speaks to ongoing [[ai-literacy]] efforts and to [[academic-integrity]] debates about authorship and process transparency in design and [[writing-education]], offering educators a discipline-specific evidence base for [[generative-ai]] guidance rather than blanket restriction. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[student-experience]] - [[ai-literacy]] - [[academic-integrity]] - [[writing-education]] ## Connected Articles - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[student-regulatory-awareness-genai]] — Knowing the Rules Is Not Enough: Student Regulatory Awareness and Use of GenAI in Higher Education - [[genai-declaration-frameworks-higher-education]] — Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education - [[student-rationalization-ai-writing]] — It''s OK Because...": The Wild West of Student Rationalization of AI Use in Academic Writing ## Citation Stefana Broadbent, et al. (2026). [A study of GenAI usage by Design Students: Analysis of Survey Results and Journals of AI practices at the Politecnico di Milano in 2025/2026](https://arxiv.org/abs/2607.17094). arXiv:2607.17094. arXiv:2607.17094 [cs.CY]. --- ## [Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing](https://edtechdev.github.io/aied/articles/gender-bias-transfer-llm-writing/) > **Ariyan Hossain, Kazi Kamruzzaman Rabbi, Farig Sadeque, S M Taiabul Haque** (2026). arXiv cs.CL ## Key Findings - In a controlled study with **N = 123 participants**, students wrote career plan essays for paired biographical profiles differing only in gender under three conditions: **no AI assistance, neutral LLM assistance, or gender-biased LLM assistance**. - Students in the biased condition produced essays with a **significantly larger agentic gap and more gender-stereotypic occupation suggestions** than those in the control and neutral conditions. - Bias transfer was **asymmetric**: agency was suppressed in female-target essays while male-target writing remained largely unaffected. - A verification study of **N = 1,600 LLM-generated essays** (two-way ANOVA, R² = .399) confirmed that a gender-biased prompt induces gender-differentiated language while a neutral prompt does not, with significant main effects of biography gender, prompt condition, and their interaction. - The authors frame the findings as evidence of **bias propagation in AI-assisted writing**, calling for fairness-aware design in educational AI tools. ## Study Design & Method The researchers first verified that a gender-biased prompt induces gender-differentiated language in LLM-generated essays while a neutral prompt does not, using a corpus of 1,600 generated essays analyzed with a two-way ANOVA. They then recruited 123 participants in a controlled environment to write career plan essays for paired biographical profiles differing only in gender, under three assistance conditions. Outcomes included an agentic gap measure (the difference in agency expressed in female- versus male-target essays) and the gender stereotypicality of occupation suggestions. Normality was confirmed across all gender × condition cells (Shapiro-Wilk, all p > .05), supporting parametric tests. ## Relevance to AI in Education This paper contributes directly to understanding how AI systems interact with learners in authentic educational settings. It provides causal evidence that gender-biased LLM prompts transfer bias into student writing, with asymmetric effects suppressing female agency — and it notes that student adoption of AI writing tools has accelerated sharply, rising from 66% to 92% between 2024 and 2025 in cited survey data. The results highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools. ## Implications for AI in Education For [[writing-education]] and [[bias-mitigation]] efforts, the study implies that the fairness of AI-assisted writing depends not only on model outputs but on the **prompts and scaffolding** students are given: if a biased system or prompt shapes how students describe themselves and their plans, the effects can carry into human-authored text. This has consequences for [[equity]] in assessment and for [[academic-integrity]] discussions, where the focus is often on detecting AI use rather than on auditing the values the AI transmits. ## Connected Concepts - [[bias-mitigation]] - [[equity]] - [[prompt-engineering]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[plagiarism-detection]] - [[administrator]] - [[teacher-ai-competency]] ## Connected Articles - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability - [[humanlike-ai-collaborative-writing]] — It Felt a Bit Eerie": Exploring Humanlike Interactions During Collaborative Writing with an Artificial Agent - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[paternalistic-filter-llm-history-education]] — The Paternalistic Filter: Epistemic Injustice and Differential Refusal in LLM-Mediated History Education for Marginalized Romanian Students - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[llm-reliance-types-undergrad]] — Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers: A Mixed-Methods Study at a Minority-Serving R1 University ## Citation Ariyan Hossain, Kazi Kamruzzaman Rabbi, Farig Sadeque, S M Taiabul Haque (2026). [Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing](https://arxiv.org/abs/2606.15914). arXiv:2606.15914. arXiv cs.CL. --- ## [Gender Differences in AI Literacy Workshop Outcomes and Deepfake Engagement](https://edtechdev.github.io/aied/articles/gender-differences-ai-literacy-deepfake/) > Examines gender differences in AI literacy, safety awareness, and STEM career aspirations among Australian secondary students (Years 7, 8, 10; N=199) from two co-educational government schools after a one-day AI literacy workshop. Male students reported higher STEM career interest; female students were more likely to use AI for schoolwork and seek AI advice. Males were more likely to have created deepfake content. Both genders improved post-intervention, but females showed richer gains: wider conceptual understanding, greater confidence, and increased AI/CS career interest partially narrowing the gender gap. - Single-day AI literacy workshops partially narrow gender STEM gaps; males need targeted deepfake safety education. ## Key Findings - Pre-workshop, **male students reported significantly higher STEM career interest** across all three domains measured (AI, computer science, and engineering), controlling for year level and school. - **Female students were significantly more likely to use AI for schoolwork and to seek advice from AI tools**. - Gender-differentiated patterns emerged in deepfake behavior: **males were significantly more likely to have created or shared deepfake content**. - Both genders improved in AI knowledge after the one-day workshop, but **females showed a richer profile of gains**: wider conceptual understanding, greater confidence, and meaningful increases in AI and computer science career interest that partially narrowed the gender STEM gap. - The authors conclude that even single-day workshops can narrow gender gaps in STEM aspirations and AI confidence, and highlight the need for **deepfake safety education targeted at male students**. ## Study Design & Method The study surveyed Australian secondary students in **Years 7, 8, and 10 (Npre = 199, Npost = 136)** from two co-educational government schools who participated in a one-day AI literacy workshop. Statistical regression methods controlling for year level and school were used to examine gender differences in AI literacy, safety awareness, and STEM career aspirations before and after the intervention. ## Implications for AI in Education For [[k-12]] [[ai-literacy]] curricula, the findings support **gender-responsive curriculum design**: engagement patterns differ by gender before instruction — females already using AI for schoolwork, males more engaged with deepfake content — so safety education and career aspiration support should be differentiated rather than uniform. The partial narrowing of the STEM career gap after a single-day intervention suggests short, well-designed workshops can be an effective [[equity]] lever in [[stem-education]], while the deepfake findings point to a concrete safety target for male students. ## Connected Concepts - [[ai-literacy]] - [[administrator]] - [[teacher-ai-competency]] - [[equity]] - [[bias-mitigation]] - [[agentic-ai]] - [[k-12-ai-education]] - [[k-12]] ## Connected Articles - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[dyslexlens-dyslexic-learners-ai]] — DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[post-covid-ict-career-aspirations]] — Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills? ## Citation Jake Renzella, Christian Bergh, Natasha Banks, Alexandra Vassar (2026). [Gender Differences in AI Literacy Workshop Outcomes and Deepfake Engagement](https://arxiv.org/abs/2606.14718). arXiv:2606.14718. arXiv preprint. --- ## [Generate-Then-Validate: Question Generation for Education](https://edtechdev.github.io/aied/articles/generate-then-validate-question-gen/) > **Synthesis:** A novel generate-then-validate pipeline for educational question generation that reduces LLM hallucination by 62% compared to direct generation, validated on STEM datasets with 89% accuracy and a 23% improvement over baseline LLMs on relevance metrics. The two-stage approach first generates candidate questions, then validates them against domain constraints and pedagogical criteria. ## Approach The paper introduces a two-stage pipeline for automated educational question generation: 1. **Generate** — an LLM produces candidate questions based on source material and specified learning objectives 2. **Validate** — a separate validation module checks each candidate against domain constraints, factual accuracy, and pedagogical quality criteria This architecture addresses a core limitation of direct generation: LLMs produce plausible-sounding but factually incorrect or pedagogically inappropriate questions at high rates. The validation stage acts as a quality filter, discarding or flagging candidates that fail domain-specific checks. ## Key Findings - **62% reduction in hallucination** compared to direct LLM generation - **89% accuracy** on STEM datasets (physics, chemistry, biology) - **23% improvement** over baseline LLMs on relevance and pedagogical alignment metrics - The validate stage catches factual errors, inappropriate difficulty levels, and misaligned learning objectives ## Significance Automated question generation reduces manual authoring burden for educators and enables adaptive assessment at scale. The generate-then-validate approach is particularly relevant for STEM domains where factual precision is critical and hallucinated content can mislead learners. This work connects to the broader [[automated-question-generation]] and [[automated-assessment]] literature. ## Connected Concepts - [[automated-question-generation]] - [[automated-assessment]] ## Connected Articles - [[code-gen]] — CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[nsmq-riddles-science-math-benchmark]] — NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[multimodal-item-parameter-estimation-2026]] — Multimodal Item Parameter Estimation using Simulated Response Probabilities - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation ## Citation F, A.W.Y.S.J.C.P. (2026). [*Generate-Then-Validate: Question Generation for Education*](https://arxiv.org/abs/2512.10110). (LAK 2026), April 27-May 01 Wei, Y., Stamper, J., et al. (2025). *Generate-Then-Validate: A Novel Question Generation Approach*. arXiv preprint arXiv:2512.10110. --- ## [Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment](https://edtechdev.github.io/aied/articles/generative-ai-education-productivity-gaps/) > **Synthesis:** In a [[rct|randomized controlled trial]] with 1,174 participants, Cruces et al. find that [[generative-ai|generative AI]] substantially narrows education-based productivity gaps, closing approximately three-quarters of the initial performance difference between higher- and lower-education workers. Critically, gains are not purely from delegation — lower-education participants retain part of their improvement after AI is removed, and follow-up performance improves when intensive AI use is combined with sustained effort. This study provides causal evidence that [[ai-education|AI tools]] can serve as productivity equalizers in workplace tasks. ## Experimental Design The study employed a randomized online experiment with 1,174 adults aged 25-45 completing workplace-style problem-solving tasks: - **Treatment group:** Access to a generative AI assistant during the main task - **Control group:** No AI assistance - **Follow-up module:** Both groups completed an unassisted module to measure learning retention Chat logs were analyzed to understand differential AI usage patterns across education levels. ## Key Findings | Measure | No AI | With AI | Gap Reduction | |---------|-------|---------|---------------| | Education-based performance gap | 0.548 SD | 0.139 SD | ~75% | | Lower-ed AI gains | — | Substantial | Large | | Higher-ed AI effectiveness | — | More effective per interaction | Moderate | - **Gap narrowing:** Without AI, higher-education participants outperform by 0.548 SD; with AI, the gap shrinks to 0.139 SD - **Differential usage:** Lower-education participants obtain substantial assistance; higher-education participants use AI more effectively - **Learning retention:** Treated participants do not perform worse once AI is removed; lower-education participants retain part of their improvement - **Active engagement matters:** Follow-up performance improves only when intensive AI use is combined with sustained effort ## Implications for Education and Workforce This study provides some of the strongest causal evidence yet that generative AI can serve as a **productivity equalizer** across education levels. However, the re-emergence of gaps in unassisted settings and the differential quality of AI use suggest that [[ai-literacy]] and [[professional-training|skill development]] remain critical. Educational institutions should focus on teaching effective AI collaboration strategies, not just tool access. ## Connected Concepts - [[rct]] - [[generative-ai]] - [[ai-education]] - [[ai-literacy]] - [[professional-training]] ## Connected Articles - [[skill-diversity-worker-resilience]] — Navigating the skill diversity frontier: How skill complexity explains worker resilience - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[genai-literacy-image-discrimination]] — Generative AI Literacy Training Improves Intelligence Analysts’ Discrimination of Real and AI-Generated Images - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability ## Citation Cruces, G., Fernandez Meijide, D., Galiani, S., Galvez, R., & Lombardi, M. (2026). [*Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment*](https://arxiv.org/abs/2608.04198v1). arXiv:2608.04198v1. --- ## [Generative AI-enhanced learning experiences for computational thinking: A systematic scoping review and design guidelines](https://edtechdev.github.io/aied/articles/generative-ai-enhanced-learning-experiences-for-computational-thinking-a-systema/) > **Synthesis:** This systematic scoping review examines the use of GenAI to support the teaching of computational thinking skills. Results reveal a young but rapidly growing research field, with most interventions focusing on undergraduate students and basic programming tasks. GenAI is typically used as a coder, tutor, debugger, or ideator, with mixed effects on learning outcomes. A key challenge is the tension between overreliance by beginners and under-utilization by advanced learners. Seven design guidelines are derived to effectively integrate GenAI for CT while minimizing risks. ## Key Findings This systematic scoping review examines the use of GenAI to support the teaching of computational thinking skills. Results reveal a young but rapidly growing research field, with most interventions focusing on undergraduate students and basic programming tasks. GenAI is typically used as a coder, tutor, debugger, or ideator, with mixed effects on learning outcomes. A key challenge is the tension between overreliance by beginners and under-utilization by advanced learners. Seven design guidelines are derived to effectively integrate GenAI for CT while minimizing risks. ## Connected Concepts - [[generative-ai]] - [[computational-thinking]] ## Connected Articles - [[liang-genai-systematic-review-human-ai-2026]] - [[learning-by-chatting-genai-impact]] - [[genai-usage-design-students-survey]] - [[genai-mindtool-generative-learning]] ## Citation Ouaazki, A., Shibani, A., Knight, S., & Holzer, A. (2026). [*Generative AI-enhanced learning experiences for computational thinking: A systematic scoping review and design guidelines*](https://doi.org/10.1016/j.caeai.2026.100608). *Computers and Education: Artificial Intelligence.* --- ## [Generative AI without guardrails can harm learning: Evidence from high school mathematics](https://edtechdev.github.io/aied/articles/generative-ai-guardrails-harm-learning/) This landmark field experiment is among the first randomized controlled trials to causally demonstrate that **unguarded generative-AI tutoring can harm skill acquisition**, not merely fail to help. Conducted with **nearly 1,000 high-school math students** across ~50 classes at a large school in Turkey (Fall 2023–2024), the study compares three arms assigned at the classroom level: a **control** arm (textbook/notes only), **GPT Base** (a ChatGPT-like GPT-4 chat interface), and **GPT Tutor** (GPT-4 with teacher-designed guardrails — hints instead of answers, plus the correct solution and common mistakes baked into the prompt). Each 90-minute session had three parts: a teacher lecture, an AI-assisted practice period (treatment only here), and an unassisted closed-book exam on conceptually similar problems. The core finding: access to generative AI sharply improves **performance** during practice but, without guardrails, **degrades learning** once the tool is removed — a tradeoff driven by students using the AI as a "crutch" to copy answers rather than to learn. ### Performance vs. Learning Tradeoff (intention-to-treat) Relative to control (practice mean 0.284; exam mean 0.321, normalized grades): | Arm | Assisted practice (perf) | Unassisted exam (learning) | | --- | --- | --- | | **GPT Base** | **+48%** (β = 0.137\*\*) | **−17%** (β = −0.054\*) | | **GPT Tutor** | **+127%** (β = 0.361\*\*) | ≈ 0 (β = −0.004, n.s.) | - GPT Base significantly **improves** practice performance but **significantly worsens** later unassisted exam performance — students who never had AI access actually outperformed them. - GPT Tutor's guardrails **largely eliminate the learning harm** (point estimate near zero), though they do not produce a positive learning effect either. ### Mechanism: Students Use GPT Base as a Crutch Two candidate explanations for the harm were tested — (1) GPT Base's errors mislead students, and (2) students offload thinking by copying answers. The evidence strongly favors the second: - **Error analysis:** When GPT Base's logical-error rate on a practice problem is higher, it hurts *practice* performance but shows **no spillover** to the corresponding unassisted exam problem — so students aren't being systematically misled into exam errors. - **Engagement analysis:** Students in the GPT Base arm send far fewer messages and overwhelmingly just "ask for the answer" or restate the question (superficial conversations dominate). In the GPT Tutor arm, a growing share of conversations are *substantive* (asking for help, attempting answers independently), and this improves within the very first session. - GPT Base answered correctly only **51% of the time** on the 57 practice problems (42% logical errors, 8% arithmetic errors) — yet students still copy its outputs. ### Students Don't Perceive the Harm Students in the GPT Base arm performed *worse* on the exam but did **not** report learning or performing less; GPT Tutor users *perceived* they performed better than control even though exam scores were statistically indistinguishable. This perceived-vs-actual-learning mismatch parallels the "feeling of learning" literature and means self-report is an unreliable gauge of AI's learning impact. ### Secondary Results - **Skill-gap narrowing is temporary:** Both AI arms reduced grade dispersion (HHI) during practice (biggest help to weakest students), but the effect **does not persist** on the unassisted exam. - **Limited heterogeneity:** Little evidence of differential effects by student ability, resources, or effort on exam performance. - **Robustness:** Intention-to-treat (including noncompliers), alternative specifications, and absenteeism checks all confirm the pattern. ## Design Lesson: What the Guardrails Did GPT Tutor differed from GPT Base in two ways: (1) the prompt instructed it to **give hints, not answers**, and (2) it was seeded with **teacher-authored problem-specific information** (correct solution, common mistakes, feedback guidance) — making its hints accurate and checkable. This labor-intensive prompt design is what neutralized the crutch effect. The authors note GPT Tutor remains *passive* (it doesn't proactively probe misconceptions) and call for combining pedagogical software tutors with generative AI, plus "co-pilot" models that assist human tutors rather than replace them. ## Implications - **For the wiki:** This is the strongest *causal, field-deployed* evidence for the [[over-reliance]] and [[cognitive-offloading]] phenomena, and a direct, preregistered demonstration of the [[genai-performance-vs-learning]] distinction (performance ↑, learning ↓). It complements [[generative-ai-reduced-study-time-math]] (population-scale behavioral evidence) and [[over-reliance]] (lab causal evidence), and motivates [[scaffolding]] and [[socratic-ai-dialogue]] designs that preserve learning. - **For practice/policy:** Because ChatGPT-like tools are already widely used by students off-platform, the GPT Base harm is immediately relevant. Deployments where learning matters need explicit guardrails (hint-not-answer prompts, teacher-authored scaffolding, answer-checking). ## Limitations (per authors) Single topic (math), single high school in Turkey, Fall 2023 (early GPT-4 era), short-term outcomes only; writing and other subjects lack the objective grading used here. Generalizability to newer models and other contexts requires further study. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[scaffolding]] - [[socratic-ai-dialogue]] ## Connected Articles - [[genai-performance-vs-learning]] - [[generative-ai-reduced-study-time-math]] ## Citation Bastani, H., Bastani, O., Sungu, A., Ge, H., Kurucu, Ö., & Mushi, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. [Proceedings of the National Academy of Sciences](https://www.pnas.org/doi/10.1073/pnas.2422633122), 122(26). https://doi.org/10.1073/pnas.2422633122 --- ## [Generative AI interactive textbook in electrotechnics: A four-year comparative study on student performance and inclusion](https://edtechdev.github.io/aied/articles/generative-ai-interactive-textbook-in-electrotechnics-a-four-year-comparative-st/) > **Synthesis:** This four-year comparative study presents results of implementing a Generative-AI Interactive Textbook built on GPT-4, integrated into an Electrical Engineering course. With a sample of 736 students, results suggest effects vary by assessment type: statistically significant improvement in mid-term assessment was consistently observed in multi-year analyses, while final assessment results showed more variable patterns. The study presents a practical method for implementing intelligent textbooks including targeted prompts, didactic anchoring to higher levels of Bloom's taxonomy, and principles to ensure GenAI develops cognitive abilities rather than replacing them. ## Key Findings This four-year comparative study presents results of implementing a Generative-AI Interactive Textbook built on GPT-4, integrated into an Electrical Engineering course. With a sample of 736 students, results suggest effects vary by assessment type: statistically significant improvement in mid-term assessment was consistently observed in multi-year analyses, while final assessment results showed more variable patterns. The study presents a practical method for implementing intelligent textbooks including targeted prompts, didactic anchoring to higher levels of Bloom's taxonomy, and principles to ensure GenAI develops cognitive abilities rather than replacing them. ## Connected Concepts - [[generative-ai]] ## Connected Articles - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] - [[effects-of-ai-chatbot-supported-cooperative-flipped-classroom-on-student-collabo]] ## Citation Fecko, B., Dziak, J., Vince, T., & Molnár, J. (2026). [*Generative AI interactive textbook in electrotechnics: A four-year comparative study on student performance and inclusion*](https://doi.org/10.1016/j.caeai.2026.100611). *Computers and Education: Artificial Intelligence.* --- ## [Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build](https://edtechdev.github.io/aied/articles/generative-ai-reduced-study-time-math/) This landmark study provides the **first large-scale behavioral and outcome evidence** that [[generative-ai]] has fundamentally altered how students study and what they retain. Using a ten-year panel of **3.2 million ALEKS learning interactions** and complementary placement-assessment data, the authors employ a quasi-experimental design that exploits within-curriculum variation in AI susceptibility: text-based word problems that can be transcribed into AI prompts serve as the treated group, while graph-based problems requiring interactive platform manipulation serve as the comparison. ### Time-on-Task Decline Learning time on AI-susceptible problems declined at a rate of **2.8% per quarter** among college students after ChatGPT's release, cumulating to a **26.9% reduction** over eleven quarters. The effect varied by age group: - High school: **31.3%** cumulative decline - Middle school: **9.0%** cumulative decline - Grade 5: No detectable change ### Proctoring Eliminates the Effect The divergence in study time **vanishes entirely under proctoring** for college students, ruling out general efficiency improvements as an explanation. This strongly implicates off-platform AI use rather than any platform, cohort, or curriculum change. ### Learning Outcome Impact - **Proctored retention items**: 25% cumulative decline in odds of correct response - **Non-proctored assessment**: Large opposite-signed increase — impossible to attribute to anything other than AI assistance ### The "Cognitive Surrender" Concept The authors introduce the term **cognitive surrender** to describe students offloading thinking to generative AI, producing a measurable population-level decline in durable knowledge. This represents a fundamental shift from [[cognitive-offloading]] as a deliberate metacognitive strategy to a passive, unreflective dependency. ## Implications for the Wiki This study provides the strongest empirical evidence yet for the [[over-reliance]] phenomenon documented across the wiki. It extends findings from [[genai-performance-vs-learning]] and [[cognitive-shift-ai-education]] by demonstrating effects at population scale with objective behavioral measures rather than self-report. The findings have direct implications for: - [[academic-integrity]] policy and assessment governance - [[ai-literacy]] curriculum design - learning-gains measurement methodology - Educational [[regulation]] and AI policy ## Connected Concepts - [[generative-ai]] - [[cognitive-offloading]] - [[over-reliance]] - [[academic-integrity]] - [[ai-literacy]] - [[regulation]] ## Connected Articles - [[genai-performance-vs-learning]] - [[cognitive-shift-ai-education]] ## Citation Rismanchian, S., Uzun, H., Matayoshi, J., Cosyn, E., & Kurd-Misto, E. (2026). [Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build](https://arxiv.org/abs/2605.21629). arXiv:2605.21629. --- ## [Stop Writing for Me: Generative Refusal in AI Tools for Thought](https://edtechdev.github.io/aied/articles/generative-refusal-ai-tools-for-thought/) Position paper exploring "Generative Refusal" — AI tools that strategically withhold text generation to demand user articulation, functioning as a Maieutic Partner rather than a cognitive offloading tool. Argues that in domains where the labor of articulation is central to craft, AI should enhance rather than bypass human cognition. ## Key Findings - The paper critiques current GenAI design paradigms that prioritize **cognitive offloading** — generating text on the user's behalf — which risks eroding the constructive thought process essential to artistic training. - It proposes **Generative Refusal**: strategically withholding text generation to demand user articulation, positioning AI as a **Maieutic Partner** that returns cognitive friction to the user and shifts the interaction from delegation to active articulation. - The framework is instantiated in **Actor's Note**, a character-journaling tool for actor training that generates **context-aware questions instead of draft text**. - A field study of Actor's Note suggests the constraint **significantly reduced cognitive burden** while fostering a **residual effect of internalized questioning habits**. - The motivating context is character journaling — a foundational tool for thought in actor training that is valuable but hard to sustain: actors frequently abandon it not from lack of material but because the high cognitive load of initiating reflection after exhausting rehearsals creates a barrier, often manifesting as the "blank page" problem. ## The Maieutic Interaction Framework Instead of bypassing cognition, the Maieutic Interaction Framework returns cognitive friction to the user: the system withholds drafts and generates questions that prompt reflection, inverting the efficiency-oriented default in which generative tools write for users. For creativity support and [[writing-education]], the framework suggests that the *absence* of generated text can be a deliberate design feature rather than a deficiency, protecting the constructive thought process that the labor of articulation is meant to build. ## Implications for AI in Education The position paper generalizes beyond theater: wherever the labor of articulation is the point of the learning activity, AI tools should be designed to protect human cognition against "the tendency of generative efficiency." This connects directly to [[cognitive-offloading]] and [[over-reliance]] concerns in education — if tools always complete the writing, students lose the constructive thought process the task was designed to build — and supports [[scaffolding]] designs in which AI prompts, questions, and withholds rather than completes. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[scaffolding]] - [[open-source]] - [[writing-education]] - [[reducing-ai-misuse]] - [[math-education]] - [[creativity]] ## Connected Articles - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding - [[ai-tools-arab-english-classrooms]] — AI tools in Arab University English classrooms: Looking back and forward - [[generative-ai-guardrails-harm-learning]] — Generative AI without guardrails can harm learning: Evidence from high school mathematics - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[genai-performance-vs-learning]] — Distinguishing performance gains from learning when using generative AI - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning ## Citation Sora Kang (2026). [Stop Writing for Me: Generative Refusal in AI Tools for Thought](https://arxiv.org/abs/2607.24751). arXiv:2607.24751. cs.HC. --- ## [Generativism: Toward a Learning Theory for the Age of Generative Artificial Intelligence](https://edtechdev.github.io/aied/articles/generativism-learning-theory/) **Li & Zheng (2026)**. Li & Zheng argue that the four dominant learning theories — behaviorism, cognitivism, constructivism, and connectivism — show significant conceptual limitations as [[generative-ai]] proliferates in [[higher-ed|educational settings]]. They propose **Generativism**, a new learning theory for the generative AI age, which posits that learning increasingly occurs through the iterative co-construction of knowledge between human learners and AI systems. The theory is built on four core principles: 1. **Epistemic Partnership** — Humans and AI collaboratively construct knowledge through iterative dialogue and co-reasoning, extending [[human-ai-collaboration]] research. 2. **Distributed Agency** — Learning is a shared process where both human and AI exert influence over knowledge outcomes, building on [[cognitive-offloading-llm-synthesis-writing]] and [[cognitive-shift-ai-education]]. 3. **Generative Literacy** — Learners must develop skills to critically interpret, evaluate, and guide AI-generated content, complementing [[ai-literacy]] frameworks. 4. **Adaptive Metacognition** — Learners monitor and regulate their own cognition *and* the AI's contributions during collaborative learning, extending [[scaffolding]] and [[self-regulated-learning]]. Generativism has profound implications for [[intelligent-tutoring|instructional design]], [[formative-assessment|learning and assessment]], and [[teacher-role|expertise development]] in contexts where generative AI plays an integral role in cognition. The framework challenges existing [[principled-ai-education|educational AI principles]] and offers a foundation for rethinking how [[llm|large language models]] reshape the fundamental nature of learning itself. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[human-ai-collaboration]] - [[ai-literacy]] - [[scaffolding]] - [[self-regulated-learning]] - [[intelligent-tutoring]] - [[formative-assessment]] - [[teacher-role]] - [[llm]] ## Connected Articles - [[cognitive-offloading-llm-synthesis-writing]] - [[cognitive-shift-ai-education]] - [[principled-ai-education]] ## Citation Li, S., & Zheng, J. (2026). [Generativism: Toward a Learning Theory for the Age of Generative Artificial Intelligence](https://arxiv.org/abs/2606.12441). arXiv:2606.12441. --- ## [GIFT-AI: Teaching the Game and Leveling the Field: Peer and AI Review + Reflection in a Business Writing Course](https://edtechdev.github.io/aied/articles/gift-ai-pairr-business-writing-2025/) > **Synthesis:** MacArthur et al. (2025) present the GIFT-AI approach — "teaching the game and leveling the field" — applying the Peer and AI Review + Reflection (PAIRR) model specifically to an upper-division Business Writing course (34 participating students at UC Davis in 2024). The model scaffolds major assignments so students receive peer review, then criteria-based chatbot feedback on the same draft, reflect on and critically assess both, and revise. The article shows how PAIRR builds AI literacy and writerly agency while leveling the playing field for students with differing preparation — particularly multilingual international students — and offers detailed implementation guidance for professional writing courses. ## Key Findings - The PAIRR model guides students in limited, guided use of chatbot feedback on drafts, requiring them to critically assess and reflect on both peer and AI feedback before revising — combining human-centered peer review with AI feedback in a way that protects writerly agency and builds [[ai-literacy]]. - Students valued AI feedback for both higher-order and sentence-level comments but sometimes found it too general or "surface-level"; they appreciated peers' contextual and situational knowledge — e.g., "my peers looked at it from the employer's perspective… ChatGPT did not do that as much." - Students did not simply trust AI's fluent prose: one quarter of coded reflections in the larger study expressed skepticism about or noted inaccuracies in AI feedback, a sign of developing AI literacy. - For multilingual international students, the model addresses challenges of linguistic inadequacy, pressure to conform to standard language norms, and unfamiliarity with U.S. source-use norms — positioning AI as "an additional, supportive perspective" rather than a tool to "correct" deficient writing or a get-out-of-writing-free card. - Three student profiles (Pengxi, high self-efficacy; Haoyu, low self-efficacy who initially over-relied on AI; Thuy, high-achieving first-generation student) illustrate how PAIRR encourages critical evaluation of AI feedback, recognition of peer-AI agreement, and preservation of students' own voices. ## Study Design & Method This is a Brief Research Report presenting the PAIRR model applied to a face-to-face and hybrid upper-division Business Writing course over a 10-week term at a public R1 university in the western U.S. The course (enrollment capped at 25) serves students majoring in management, economics, communications, and related fields. In 2024, 57% of students identified as Asian or Pacific Islander and 69% as multilingual; six were international students, 26% first-generation, 48% on financial aid. Students completed five major assignments (Job/Grad School Application, Internal Memo, Feasibility Study, Proposal, and a 5-minute Proposal Pitch), each requiring a draft, audience analysis, formal peer review by 2–3 peers, and revision. Data and findings draw on the larger PAIRR study (Sperber et al., 2025; N=654) and course-specific reflections, analyzed thematically following Saldaña in MaxQDA, with a focus on the Business Writing course and three multilingual international student profiles. ## Implications for AI in Education The article offers a tested, evidence-based curricular model for integrating AI into [[writing-education|professional writing]] instruction — directly relevant to [[peer-review]] and [[formative-assessment]]. It addresses workforce demands (employers rank written communication and problem-solving highly; AI literacy is now a required "technical skill") by building students' [[ai-literacy]] and communication skills in tandem. For [[student-experience|educational equity]], it argues that underprepared students are less likely to use AI and more likely to misuse it, and that guided AI literacy instruction can "level the playing field" while "teaching the game" of appropriate AI collaboration. It positions AI as a machine tutor (not a teacher replacement), re-centering human-in-the-loop writing instruction and human relationships, consistent with findings on [[over-reliance]] risk, and connects to multilingual writing, linguistic justice, and [[academic-integrity]] in the AI era. ## Limitations As a Brief Research Report focused on one course, the study's findings are largely descriptive and course-specific, and the article draws on the larger PAIRR study for broader claims. The Business Writing course sample is small (34 participating students of 46 enrolled). The authors note AI policies vary substantially across institutions and courses, and the applicability of the model across different writing courses and contexts is discussed as future work rather than empirically demonstrated here. ## Connected Concepts - [[ai-feedback-quality]] - [[writing-education]] - [[ai-literacy]] - [[student-experience]] - [[peer-review]] - [[self-regulated-learning]] - [[metacognition]] - [[academic-integrity]] - [[language-learning]] ## Connected Articles - [[pairr-ai-peer-review-2025]] — Peer and AI Review + Reflection (PAIRR): A Human-Centered Approach - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and Teacher Feedback - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy - [[student-rationalization-ai-writing]] — "It's OK Because...": The Wild West of Student Rationalization - [[learner-centered-feedback-ai]] — Enhancing Learner-Centered Feedback With AI - [[genai-linguistic-diversity-academic-writing]] — Generative AI and Linguistic Diversity in Academic Writing ## Citation MacArthur, M., Minnillo, S., Sperber, L., Whithaus, C., & Stillman, N. (2025). [*GIFT-AI: Teaching the game and leveling the field: Peer and AI Review + Reflection in a business writing course*](https://doi.org/10.3389/fcomm.2025.1615752). --- ## [ProPACT: Pair Programming with AI](https://edtechdev.github.io/aied/articles/golrang-propact-pair-programming-2026/) > **ProPACT** (Proactive AI-Driven Adaptive Collaborative Tutor) is an AI-driven adaptive tutoring system for pair programming that **treats collaboration itself as the object of instruction.** Unlike individual-centric, reactive systems, it models *dyadic* learning states in real time and intervenes *before* collaborative breakdowns occur, using multimodal sensing and predictive forecasting. ## Key Findings 1. **Significant performance gains from proactive feedback.** Dyads receiving ProPACT feedback achieved substantially higher debugging success (*t*[49.96] = −13.51, *p* < .0001) and completed tasks more efficiently (*t*[44.70] = 4.39, *p* < .0001) compared to the no-feedback control condition. 2. **Dyadic sensing enables predictive intervention.** ProPACT constructs a multimodal dyadic learner model from Joint Visual Attention (JVA — cosine similarity of gaze distributions over 30-second windows), Joint Mental Effort (JME — cross-recurrence quantification of pupil-diameter signals), and individual Mental Effort (IPA from pupillary fluctuations). An XGBoost-based forecaster predicts sub-optimal collaboration states up to **30 seconds in advance.** 3. **Five-tier adaptive feedback hierarchy works.** The system escalates through minimally intrusive scaffolds: (A1) do nothing when collaboration is productive; (A2) temporarily enable GitHub Copilot when cognitive strain rises; (A3) show a gaze-awareness tool highlighting the partner's visual focus; (A4) issue unobtrusive dialogue prompts to re-align mental effort; and (A5) provide directive task-based hints only as a last resort. Signals are discretized against a normalized resting baseline using a ±2SD criterion (High / Average / Low). 4. **Post-intervention gains in collaborative regulation.** Beyond task-level improvements, dyads showed sustained increases in JVA and JME after the intervention, indicating that the system fostered durable collaborative skills rather than just providing momentary assistance. ## Implications ProPACT represents a shift from *individual* to *dyadic* learner modeling in [[intelligent-tutoring]]. By treating the pair — not the person — as the unit of analysis, it addresses a long-standing gap in [[collaborative-learning]] support. Traditional ITS architectures focus on individual cognition; ProPACT demonstrates that multimodal signals (gaze, pupil dilation) can be fused to model the health of a collaborative process in real time. The proactive forecasting approach is a departure from reactive feedback paradigms common in [[adaptive-learning]]. By predicting breakdowns 30 seconds ahead, ProPACT avoids the latency inherent in "detect-then-respond" architectures, allowing scaffolds to arrive *before* students experience frustration or disengagement. This has implications for engagement-metrics and real-time classroom orchestration. For [[cs-education]] specifically, ProPACT validates that AI-assisted pair programming can improve both task outcomes and collaborative skill development. The system's integration with [[collaborative-ai-tutoring]] workflows suggests a future where AI tutors monitor not just what students produce (code), but *how* they work together. The gaze-awareness tool (A3) is a particularly novel intervention: rather than providing didactic content, it surfaces the partner's attentional focus as a lightweight nudge toward shared attention. This aligns with [[multimodal-ai-tutoring]] research emphasizing non-verbal channels for learning support. ## Connected Concepts - [[adaptive-learning]] - [[adaptive-learning]] - [[ai-tutoring]] - [[rag]] ## Connected Articles - [[collaborative-ai-tutoring]] — Collaborative AI Tutoring - [[multimodal-ai-tutoring]] — Multimodal AI Tutoring in STEM - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[transfer-of-learning]] — AI Learning Transfer - [[ai-powered-personalized-learning-elementary-fractions-2026]] — Exploring Fraction Comprehension and Interest in Elementary Education Through AI-Powered Personalized Learning - [[ai-stem-bibliometric-trends]] — Why does AI unlock new possibilities in STEM education? A Bibliometric Analysis of Trends and Future Agenda - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[ai-vocational-education-training-review]] — Artificial intelligence in vocational education and training: A systematic review of educational purposes, theoretica... - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[astra-atco-training-simulator]] — ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots - [[automated-grading-linux-bash-examinations-large-language-models]] — Automated Grading of Linux/Bash Examinations Using Large Language Models - [[behaviorally-adaptive-visual-diversion-assessment-2026]] — Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery - [[buggy-genai-code-student-responses]] — When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code - [[chat-debugging-llm-hardware-education-2026]] — WIP: Chat-Debugging: Large Language Model as a Hardware Debugging Assistant - [[clara-collaboration-literacy-dashboard]] — CLARA: An AI-Augmented Analytics Dashboard for Collaboration Literacy ## Citation Golrang, A., Sharma, K., Dehaen, S., & Viberg, O. (2026). [*ProPACT: A Proactive AI-Driven Adaptive Collaborative Tutor for Pair Programming*](https://arxiv.org/abs/2605.02703). arXiv:2605.02703. --- ## [Comparative Validation of GPT-4o-mini and Teacher Mean Scores for Automated Scoring of Music Analysis Responses: Single-Pass Deployment, Repeatability, and Strategy-Specific Bias](https://edtechdev.github.io/aied/articles/gpt4o-mini-music-analysis-scoring/) > **GPT-4o-mini can produce stable rubric-based scores for open-ended music analysis responses, with few-shot chain-of-thought prompting agreeing most strongly with teacher means while RAG systematically over-scores and self-consistency trades individual-level agreement for repeatability.** **Baicheng Lin, Lingxi Jin, Kyung-Seok Min** — arXiv (cs.HC / cs.SD) preprint, 2026 (Sejong University, Ewha Womans University). ## Synthesis 300 university-level music analysis responses scored by teachers on four dimensions (Harmony, Form, Reasoning, Terminology) benchmarked against three GPT-4o-mini prompting strategies. Fs+CoT (few-shot + chain-of-thought) showed the strongest agreement with teacher mean scores in both single-pass and median-aggregated conditions. RAG showed systematic over-scoring; SC (self-consistency over 5 generations) was highly repeatable but weaker at individual-level agreement. Dimension-level analysis: Terminology showed weaker agreement than Reasoning, indicating strategy-specific and dimension-specific scoring profiles. Operational use requires strategy-specific calibration, dimension-level validation, and continued human oversight. ## Connected Concepts - [[formative-assessment]] - [[automated-essay-scoring]] - [[cs-education]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[automated-assessment]] - [[socratic-method]] - [[prompt-engineering]] ## Connected Articles - [[credential-cognitive-stewardship-ai-assessment]] — What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[llm-feedback-programming-classroom]] — LLM-Generated Feedback in Introductory Programming: A Classroom Study - [[aicode-collaborative-feedback-system]] — AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education ## Citation Baicheng Lin, Lingxi Jin, Kyung-Seok Min (2026). [Comparative Validation of GPT-4o-mini and Teacher Mean Scores for Automated Scoring of Music Analysis Responses: Single-Pass Deployment, Repeatability, and Strategy-Specific Bias](https://arxiv.org/abs/2608.01783). arXiv:2608.01783. arXiv (cs.HC / cs.SD) preprint. --- ## [Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education](https://edtechdev.github.io/aied/articles/ground-truth-reliability-aied/) ## Core Argument The AIED community over-relies on **inter-rater reliability (IRR)** — typically a single Cohen's κ coefficient — as a mechanical gatekeeper for "ground truth." This practice is insufficient and potentially misleading for the complex, noisy realities of educational data. The authors propose **four practical shifts** to strengthen the evidence base of labeled AIED datasets. ## The Problem ### Noise vs. Bias in Educational Labeling Human judgment is subject to both **noise** (random variability) and **bias** (systematic directional error). While bias and fairness have received extensive attention, noise is an underexamined obstacle in AIED. Education is inherently noisy — assigning grades, defining engagement, and identifying giftedness all involve subjective interpretation. ### Why κ Alone Fails Educational settings present specific challenges that undermine threshold-based IRR heuristics: - **High-inference constructs**: Complex concepts like "engagement" or "understanding" require deep judgment beyond surface-level agreement - **Skewed label distributions**: Class imbalance inflates or deflates κ in misleading ways - **Temporally segmented multimodal data**: Video, audio, and log data over time create segmentation challenges where raters may agree on the label but disagree on *when* to apply it ### The LLM Annotation Risk The growing use of LLMs as annotators introduces new threats: - **Automation bias**: Over-trusting LLM output without verification - **Circular validation**: Using the same model to evaluate its own output - **Silent degradation**: LLM annotators may produce deceptively consistent but subtly wrong labels ## The Four Shifts ### 1. IRR as Diagnostic, Not Gatekeeper > Stop treating κ > 0.8 as a binary stamp of approval. Instead, use IRR to **localize disagreement** — identify *where* and *why* raters disagree, then refine constructs and codebooks accordingly. Disagreement is information, not failure. ### 2. Transparent Annotation Reporting Require thorough documentation of: - **Rater expertise and training**: Who labeled the data and what qualifies them? - **Codebook development**: How were categories defined and iterated? - **Reconciliation procedures**: How were disagreements resolved — majority vote, discussion, expert arbitration? - **Segmentation rules**: For multimodal/continuous data, how was it divided into labelable units? ### 3. Mitigate LLM Annotation Risks - **Bias audits** on LLM annotators before deployment - **Verification workflows** with human-in-the-loop checks - Guard against automation bias and circular validation through independent evaluation ### 4. Complement Agreement with Validity Evidence Go beyond agreement statistics with: - **Uncertainty-aware labeling**: Assign multiple labels to the same item to capture ambiguity and nuance - **Criterion-related checks**: Predictive tests to verify labels forecast intended educational outcomes - **Close-the-loop evaluations**: Test whether systems trained on these labels actually improve learning beyond a reasonable control condition ## Case Studies The paper illustrates these shifts through case studies of **multimodal tutoring data**, demonstrating how the four-shift framework applies to real AIED annotation challenges. ## Practical Recommendations 1. **Always report multiple IRR metrics** (κ, α, percentage agreement) and discuss their limitations given the data characteristics 2. **Make codebooks and annotation guidelines public** whenever possible 3. **Treat LLM annotations as hypotheses to verify**, not as ground truth 4. **Include at least one validity check** beyond agreement in every labeled dataset paper 5. **Design annotation workflows that surface ambiguity** rather than forcing binary decisions ## Open Questions - What is the "minimum viable validity evidence" for different AIED use cases (high-stakes assessment vs. formative practice)? - How should the four shifts be adapted for fully automated annotation pipelines with no human raters? - Can uncertainty-aware labeling be standardized across the field, or is it inherently task-specific? ## Connected Concepts - [[human-in-the-loop-ai]] - [[ai-ed-evaluation]] - [[automated-essay-scoring]] - [[cs-education]] - [[administrator]] - [[hallucination-risk]] - [[automated-question-generation]] - [[open-source]] ## Connected Articles - [[short-answer-scoring-quality-degradation]] — Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback - [[multimodal-ai-feedback-learning]] — LLM-based Multimodal AI Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback - [[aied-carbon-footprint-reporting]] — The Environmental Cost of LLMs in AIED: Reporting and Practices ## Citation Thomas, D. R., Borchers, C., Vanacore, K. P., Koedinger, K. R., & Kizilcec, R. F. (2026). [Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education](https://arxiv.org/abs/2603.29141). arXiv:2603.29141. Accepted to AIED 2026. --- ## [Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics](https://edtechdev.github.io/aied/articles/guided-llm-scaffolding-independent-learning/) > Experimental study comparing Guided vs. Unrestricted LLM access. Explicit training in reasoning-focused scaffolding (stepwise hints, verification) led to significantly better independent performance and self-assessment calibration compared to uncritical reliance. This work emphasizes that [[ai-literacy]] is a developmental capacity requiring structured [[scaffolding]] and [[prompt-engineering]] discipline. It connects to the need for [[curriculum-design]] that targets [[metacognition]] and [[agentic-ai]] rather than just syntax mastery. ## Key Findings - In a **four-week quasi-experimental summer program** in an undergraduate Probability and Statistics course, students were organized into three balanced conditions: **no LLM access, unrestricted LLM access, and guided LLM access**. - The guided condition used the **same LLM platform** as the unrestricted condition, but students received explicit training and rules intended to promote reasoning-focused help-seeking, stepwise hints, verification, and ethical use. - Guided use was associated with a **clearer learning-oriented interaction pattern** than unrestricted access, especially in prioritizing reasoning over final answers and requesting stepwise support. - Guided-LLM students showed a **promising pattern of stronger no-help quiz performance** during the intervention phase, while unrestricted access appeared more useful for assisted practice completion than for consistently improving independent performance. - All quizzes and the delayed final exam were completed **without LLM or external assistance**, separating AI-supported practice performance from independent learning; available time measures did not support a simple duration-based explanation, and **self-assessment calibration** suggested better alignment between perceived and demonstrated understanding in the guided condition. ## Study Design & Method The design distinguishes between *assigned LLM access* and the *quality of students' actual interaction* with the model. The three balanced conditions isolate the effect of guidance: because the guided and unrestricted groups used the same platform, differences can be attributed to training and usage rules rather than tool availability. The use of LLM-free quizzes and a delayed final exam provides a no-help measure of whether AI-supported practice transferred to independent performance. ## Implications for AI in Education The central conclusion is that **LLM access alone is an incomplete educational intervention**: for [[ai-tutoring]] and [[curriculum-design]], the design challenge is to scaffold how students use LLMs so that these systems function as partners in reasoning rather than answer-getting tools. The findings support investing in [[prompt-engineering]]-style training and help-seeking guidance, and align with concerns about [[over-reliance]] when access is unrestricted — while noting that the study's modest scale and single-course context warrant replication. ## Connected Concepts - [[ai-literacy]] - [[scaffolding]] - [[prompt-engineering]] - [[curriculum-design]] - [[metacognition]] - [[agentic-ai]] ## Connected Articles - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing - [[ase-26-agentic-software-engineering-curriculum]] — ASE-26: A Curriculum for Agentic Software Engineering as a Discipline - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation ## Citation Amanlou, M., Amou-Jafari, Y., Livani, M., Boloukazari, F., Bagheri, F., & Bahrak, B. (2026). [*Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics*](https://arxiv.org/abs/2606.01375). Proceedings of the 34th International Conference on Computers in Education. --- ## [HAIML: A Human-Centered AI Metacognitive Learning Model — A Framework for Human Agency and Reflective Learning in the Age of Artificial Intelligence](https://edtechdev.github.io/aied/articles/haiml-human-centered-ai-metacognitive-model-2026/) > **Synthesis:** HAIML is a human-centered framework for learning in AI-supported environments that preserves human agency, metacognitive awareness, ethical reasoning, and personal responsibility. Grounded in self-efficacy, self-regulated learning, experiential learning, metacognition, and automation-bias research, the model spans three interconnected layers — Experiential AI Use, Metacognitive Reflection, and Ethical Decision-Making — guiding learners from direct engagement with AI to reflective and responsible use. ## Key Findings 1. **A gap in existing frameworks.** AI-literacy, technical-proficiency, responsible-use, and academic-integrity frameworks give little attention to how AI influences the learner's thinking and decision-making processes. 2. **Three-layer model.** HAIML organizes learning around Experiential AI Use, Metacognitive Reflection, and Ethical Decision-Making, moving learners from direct engagement with AI to reflection on how AI shapes their thinking, and finally to informed, responsible decision-making. 3. **Agency-preserving design.** The framework integrates concepts from human agency, self-regulated learning, metacognition, and automation-bias research to keep learners in control of AI-mediated cognition. ## Implications For [[metacognition]] and [[self-regulated-learning]], HAIML offers a structured way to make AI interactions objects of reflection, addressing [[cognitive-offloading]] and [[over-reliance]] risks by keeping learners aware of how AI influences their thinking. It complements [[ai-literacy]] frameworks by adding a metacognitive and ethical layer focused on internal decision-making rather than tool proficiency alone. The model's human-centered orientation connects to [[human-ai-collaboration]] and to [[human-ai-collaboration]] design, and to [[ethics]] in education. As a framework paper, it would benefit from empirical validation of whether the three-layer progression measurably improves agency and reflection in AI-supported learning. ## Connected Concepts - [[metacognition]] - [[self-regulated-learning]] - [[human-ai-collaboration]] - [[ai-literacy]] - [[cognitive-offloading]] - [[over-reliance]] - [[student-modeling]] - [[ethics]] ## Connected Articles - [[metacognitive-learning-scenarios-taxonomy]] — Metacognitive learning scenarios taxonomy - [[absent-cognitive-baseline-2026]] — Absent cognitive baseline - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] — Learning to learn in the age of generative AI - [[trust-reliance-ai-education-2026]] — Trust and reliance in AI education - [[ai-fallibility-warning-help-seeking]] — AI fallibility warnings and help-seeking ## Citation Reardon, C. (2026). [*HAIML: A human-centered AI metacognitive learning model — A framework for human agency and reflective learning in the age of artificial intelligence*](https://osf.io/preprints/edarxiv/qhjgs_v1/). EdArXiv preprint. --- ## [Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot](https://edtechdev.github.io/aied/articles/hashmi-socratic-physics-chatbot-2025/) > **Synthesis:** A custom Socratic AI chatbot deployed in a large-enrollment introductory mechanics course with 150 first-year STEM majors, demonstrating that AI-driven Socratic dialogue can foster expert-like reasoning while generating fine-grained learning analytics for physics education research. ## Key Findings 1. **Socratic dialogue improves question specificity.** Student question specificity rose dramatically from ~10–15% on the first conversational turn to **100% on the final turn**, indicating that sustained Socratic interaction trains students to formulate increasingly precise, expert-like physics questions. 2. **Specificity correlates with academic performance.** Self-reported expected course grade showed a significant positive correlation with question specificity (Pearson *r* = 0.43), suggesting that the ability to formulate precise physics questions — a skill the chatbot explicitly cultivates — is linked to broader course outcomes. 3. **Students rated the chatbot positively on knowledge-building.** Post-interaction surveys yielded a median rating of **4.0/5** for knowledge-based skills and **3.4/5** for overall effectiveness, indicating acceptable student reception for a tool deployed at scale. 4. **Dual-purpose design enables both instruction and research.** The chatbot served simultaneously as a [[socratic-method]] teaching tool and as a data-collection instrument for [[learning-analytics]], with full dialogue transcripts enabling fine-grained analysis of student reasoning patterns. ## Implications This study provides empirical evidence that the [[socratic-method]] — a pedagogical approach with ancient roots — can be effectively operationalized through [[generative-ai]] at scale. Unlike rule-based Socratic tutors that rely on pre-scripted question sequences, an LLM-powered chatbot can adapt its questioning dynamically to each student's reasoning trajectory, making it viable for [[intelligent-tutoring]] in large-enrollment courses where one-on-one Socratic dialogue is otherwise impractical. The specificity trajectory finding is significant for [[socratic-method]] research: it demonstrates that the benefit of Socratic dialogue is not just in the answers students produce, but in the *quality of questions they learn to ask.* This metacognitive dimension aligns with research on [[socratic-method]] as a tool for developing disciplinary ways of thinking rather than merely transmitting content. For [[stem-education]] and [[physics-education]] specifically, the chatbot's deployment in a real course (not a lab study) with 150 students establishes feasibility for production use. The dual-purpose architecture — serving both instruction and research — models how AI tutoring systems can function as instruments for [[educational-measurement]] as well as pedagogical tools. The correlation between question specificity and course grade (*r* = 0.43) hints at a potential mechanism: AI-driven Socratic dialogue may improve outcomes by training the cognitive skill of precise problem formulation, which is foundational to [[physics-education]] and [[computational-thinking]]. ## Connected Concepts - [[computational-thinking]] - [[educational-measurement]] - [[generative-ai]] - [[learning-analytics]] - [[physics-education]] - [[socratic-ai-dialogue]] - [[socratic-method]] - [[stem-education]] - [[intelligent-tutoring]] ## Connected Articles - [[socratic-ai-physics-tutor-taxonomy-2026]] - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[becker-chatgpt-typology-physics-2026]] ## Citation Hashmi, A., et al. (2025). [*Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot*](https://arxiv.org/abs/2508.14778v1). arXiv:2508.14778v1. --- ## [SafeTutors: Pedagogical Safety in AI Tutoring](https://edtechdev.github.io/aied/articles/hazra-safetutors-pedagogical-safety-2026/) > **SafeTutors** is a benchmark that jointly evaluates safety and pedagogy in AI tutoring systems across mathematics, physics, and chemistry. It argues that **tutoring safety is fundamentally different from conventional LLM safety**: the primary risk is not toxic content but the quiet erosion of learning through answer over-disclosure, misconception reinforcement, and the abdication of scaffolding. ## Key Findings 1. **Universal harm across all models.** Every evaluated model — from 3.8B open-weight models to GPT-5-mini — exhibited broad pedagogical harm. No model was consistently safe across all subjects and interaction modes, indicating that tutoring safety is not solved by general capability improvements. 2. **Scale does not reliably improve safety.** Increasing model size did not produce consistent improvements in pedagogical safety, challenging the assumption that larger models are inherently better tutors. This finding parallels broader critiques in llm-evaluation that general benchmarks do not capture domain-specific safety requirements. 3. **Multi-turn dialogue dramatically worsens behavior.** Pedagogical failure rates escalate from **17.7% in single-turn** interactions to **77.8% in multi-turn** conversations. The crescendo-based escalation design reveals that models which appear safe in one-turn evaluations systematically degrade across sustained interaction — single-turn "safe/helpful" results mask systematic tutor failure. 4. **Harms are subject-dependent.** Violation patterns vary significantly across mathematics, physics, and chemistry, indicating that mitigations must be discipline-aware. A tutoring safety strategy that works for math may not transfer to science domains. 5. **An 11-dimension, 48-sub-risk taxonomy grounds the evaluation.** SafeTutors' risk taxonomy spans Cognitive, Epistemic, Metacognitive, Motivational-Affective, Developmental & Equity, Instructional Alignment, Behavioral & Inquiry, Ethical-Epistemic Integrity, Informational-Semantic, Reflective-Critical, and Pedagogical Relationship dimensions — each with multiple sub-risks drawn from learning-science literature. ## Implications SafeTutors fundamentally reframes the conversation around [[pedagogical-safety]] and [[ai-tutor-safety-harms]]. The dominant paradigm has been to evaluate AI tutors on problem-solving accuracy and generic safety (toxicity, refusal), but SafeTutors demonstrates that a tutor can be technically accurate and "safe" by conventional metrics while systematically undermining learning. The benchmark's central insight — that tutoring harm is qualitatively different from content harm — has major implications for [[ai-tutoring]] regulation and deployment. The multi-turn degradation finding is particularly alarming for real-world deployment. Most tutoring interactions extend over multiple turns, yet the evaluation community has largely relied on single-turn benchmarks. SafeTutors provides evidence that this practice is dangerously misleading. Systems like [[eduzone-llm-safety-k12]] and [[vetting-dual-llm-safety-education]] that prioritize multi-turn safety evaluation are essential, not optional. The risk taxonomy itself is a significant contribution, providing a theoretically grounded vocabulary for discussing tutoring harm. It bridges educational-theory and AI safety, enabling researchers to move beyond vague claims about "tutor quality" toward precise identification of specific failure modes. This taxonomy could inform the design of [[pedagogical-safety-rl]] approaches like [[singh-eduqwen-pedagogical-rl-2026]] that train models to avoid specific pedagogical harms. For [[k-12]] contexts, where the stakes of pedagogical harm are highest, SafeTutors provides empirical evidence that current models are not safe enough for unsupervised deployment. The subject-dependence of harms suggests that safety evaluation must be integrated into discipline-specific [[ai-tutor-behavioral-evaluation]] pipelines rather than treated as a one-time gate. ## Connected Concepts - [[ai-tutoring]] - [[k-12]] - [[pedagogical-safety]] - [[llm]] - [[regulation]] - [[scaffolding]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] — The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[eduzone-llm-safety-k12]] — EduZone: A Framework for Evaluating LLM Safety for K-12 Students and Teachers - [[pedagogical-safety-rl]] — Pedagogical Safety in Educational Reinforcement Learning - [[singh-eduqwen-pedagogical-rl-2026]] — EduQwen: Pedagogical RL - [[vetting-dual-llm-safety-education]] — VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning ## Citation Hazra, R., Ghuku, B., Marchenko, I., Tokarieva, Y., Layek, S., Banerjee, S., Stoyanovich, J., & Pechenizkiy, M. (2026). [*SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems*](https://arxiv.org/abs/2603.17373). arXiv:2603.17373. --- ## [Human-Centric Artificial Intelligence Pedagogy (HCAP) framework developed from TPACK through integration of artificial intelligence literacy and competency](https://edtechdev.github.io/aied/articles/hcap-human-centric-ai-pedagogy-framework-2026/) > **Synthesis:** Chiu (2026) proposes the Human-Centric AI Pedagogy (HCAP) framework, an evolution of the Technological Pedagogical Content Knowledge (TPACK) model designed for the generative AI era. Arguing that AI's agentic autonomy, epistemic complexities, and ethical dimensions render the established TPACK framework insufficient, HCAP integrates five knowledge domains: AI-Technological, AI-Content, AI-Pedagogical, Human-AI Collaborative, and Ethical Knowledge. A three-round Delphi study with 30 teachers across diverse subjects produced a consensus on 25 critical knowledge items, providing an empirically grounded model that translates theoretical AI pedagogy into actionable teacher competencies and equips educators to move from merely using AI to strategically orchestrating human-AI collaborative learning. ## Key Findings - **HCAP extends TPACK with five interdependent knowledge domains:** three that transform the original TPACK cores through a sociotechnical lens — AI-Technological Knowledge (I-TK), AI-Content Knowledge (I-CK), and AI-Pedagogical Knowledge (I-PK) — plus two entirely novel domains: **Human-AI Collaboration Knowledge (HAIC-K)** and **Ethical Knowledge (Ethics-K)**. - The framework addresses four critical gaps in TPACK: **tool agency** (understanding AI's operational logic, probabilistic nature, and prompt engineering), **epistemic challenges** (teachers shifting from content deliverers to critical validators), **ethical complexity** (making ethical reasoning a core, explicit domain), and **dynamic collaboration** (conceptualizing human-AI collaboration where cognitive tasks are shared). - A **three-round Delphi study with 30 teachers** (adopting a 75% agreement threshold) refined and reached consensus on **25 critical knowledge items** across the five domains, including: prompt engineering, limitation awareness, system auditing, data fundamentals, tool integration (I-TK); critical validation/source triangulation, bias detection, contextualization, disciplinary methodological critique (I-CK); reflective practice, assessment redesign, scaffolded co-agency design, personalization, ethical integration (I-PK); dynamic role allocation, interaction flow design, critical dialogue, critical interdependence, group dynamics collaboration (HAIC-K); and equity auditing, data privacy, usage protocols, societal impact, well-being advocacy, inclusive design (Ethics-K). - The knowledge domains are anchored in foundational literacies: I-TK in data and computational literacy, I-CK/I-PK in critical/algorithmic/media and scientific literacies, HAIC-K in collaborative and epistemic literacies, and Ethics-K in ethical and well-being literacies. ## Study Design & Method This study used a **three-round Delphi method** with a panel of 30 teachers from diverse subjects to establish expert consensus on the knowledge and skills required within each HCAP domain. An initial list of 20 knowledge items was refined iteratively: in Round One, seven items reached consensus and two new items were proposed; in Round Two, four more reached consensus, one new item (well-being advocacy) was proposed, and some items were split or renamed; in Round Three, nine more reached consensus and one new item (ethical integration) was proposed. The final list comprised 25 knowledge items. The study adopted a 75% agreement threshold (in line with prior Delphi research), and items endorsed for removal by more than half of the teachers were excluded. Teacher modifications to item labels and descriptions were incorporated across rounds. ## Implications for AI in Education The HCAP framework provides a concrete, empirically grounded roadmap for teacher education and [[faculty-development]], translating a theoretical model into actionable competencies for orchestrating human-AI collaborative learning. It positions teachers not merely as tool users but as **strategic orchestrators and conductors of human-AI collaboration** who use AI ethically, critically, and productively. The five-domain structure offers a practical basis for designing future-ready teacher training programs and professional development, connecting to [[ai-literacy]], [[teacher-ai-competency]], [[teacher-role]], and [[instructional-design]]. It responds to the paradigm shift from passive, deterministic digital tools to active, agentic AI systems, and its ethical and collaboration domains address the human-centred concerns central to responsible [[ai-education]] in the generative AI era. ## Limitations The study's consensus was derived from a panel of 30 teachers, and the 75% agreement threshold and consensus criteria are inherently subjective (as the author notes, there are no universal standards for Delphi consensus). The knowledge items reflect a teacher-perspective view and were validated through expert consensus rather than classroom outcome data; the framework's implementation effectiveness requires empirical validation in practice. The Delphi refinement process involved renaming and restructuring items, so the final 25-item list reflects the specific panel's judgements and may not generalize to all educational contexts. ## Connected Concepts - [[ai-literacy]] - [[teacher-ai-competency]] - [[teacher-role]] - [[pedagogical-llm-training]] - [[faculty-development]] - [[ethics]] - [[instructional-design]] ## Connected Articles - [[ai-tpack-preservice-math-teachers]] — AI-TPACK and Pre-Service Math Teachers - [[ai-tpack-teacher-multi-agent-workflow]] — AI-TPACK Teacher Multi-Agent Workflow - [[finkelstein-principled-ai-education-2025]] — Principled AI Education - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] — The Scaffolded AI literacy (SAIL) framework - [[genai-literacy-training-teacher-education-dbr-2026]] — AI Literacy Training for Teachers - [[teacher-education-ai-literacy-sdt-2026]] — Teacher Education for AI Literacy (SDT) ## Citation Chiu, T. K. F. (2026). [*Human-Centric Artificial Intelligence Pedagogy (HCAP) framework developed from TPACK through integration of artificial intelligence literacy and competency*](https://doi.org/10.1080/10494820.2026.2615818). *Interactive Learning Environments*. --- ## [Agentic AI-driven Immersive Simulation: A Knowledge-Aware Virtual Training Platform for High Dose Rate (HDR) Brachytherapy](https://edtechdev.github.io/aied/articles/hdr-brachytherapy-agentic-ai-simulation-2026/) > **Synthesis:** Xu et al. (2026) present an agentic AI-driven immersive simulation for training in **High Dose Rate (HDR) brachytherapy**, integrating VR and mobile computing to create a high-fidelity, risk-free environment for mastering complex procedural skills. A knowledge-aware assistant uses [[rag|Retrieval-Augmented Generation]] to ground agent interactions in authoritative clinical guidelines, providing natural-language interfaces and hands-free, real-time guidance during intricate maneuvers. A prototype on Meta Quest 3 with a local GPU-backed AI backend demonstrated feasible architecture with suitable latency and high context precision, answer completeness, and relevance. The work connects to [[simulation]], [[professional-training]], and [[professional-training]] research. ## Immersive, Risk-Free Clinical Training The convergence of the Metaverse and LLM-based AI agents is catalyzing a shift toward autonomous, immersive, and personalized pedagogical frameworks in medical education. This system targets HDR vaginal cylinder brachytherapy in cancer care, integrating VR and mobile computing to let trainees master complex procedural skills without the facility or safety constraints of physical anatomy or live radioactive sources. ## RAG-Grounded Knowledge Assistance A core contribution is the seamless integration of a knowledge-aware assistant leveraging Retrieval-Augmented Generation to ground agent interactions in authoritative clinical guidelines. This enables an interactive agent to provide natural-language interfaces and hands-free, real-time guidance during intricate medical maneuvers. ## Validation A prototype deployment comprising a Meta Quest 3 interface linked to a local GPU-accelerated AI backend demonstrated a feasible architecture, maintaining suitable end-to-end latency and high context precision, answer completeness, and relevance in the RAG-enhanced pedagogical support. ## Connected Concepts - [[simulation]] - [[professional-training]] - [[professional-training]] - [[rag]] - [[agentic-ai]] - [[multimodal]] - [[personalized-learning]] - [[intelligent-tutoring]] - [[pedagogical-safety]] - [[human-in-the-loop-ai]] ## Connected Articles - [[tibetcpr-ai-training-feedback]] - [[astra-atco-training-simulator]] - [[medgame-llm-medical-education-gamification]] - [[ai-use-critical-thinking-medical-students-2026]] - [[agentschool-multi-agent-simulation-education-2026]] ## Citation Xu, R., Barasa, K., Kumal, M., Liu, X., Zhou, W., & Qian, X. (2026). [*Agentic AI-driven immersive simulation: A knowledge-aware virtual training platform for high dose rate (HDR) brachytherapy*](https://arxiv.org/abs/2608.08163). arXiv:2608.08163. --- ## [Collaborative AI Literacy Framework](https://edtechdev.github.io/aied/articles/hingle-collaborative-ai-literacy-2025/) > **Collaborative AI Literacy Framework** — SEFI 2025. A systematic review of 9 studies (2015–2023) examining how collaborative learning (CL) approaches can be harnessed to build AI literacy across diverse educational contexts. Using the ICAP framework (Interactive–Constructive–Active–Passive) as an analytical lens, the review demonstrates that CL effectively increases AI literacy across activities, settings, and learner groups. Most studies were conducted in classroom settings, with some broadening participation through educators, families, or AI agents supporting teamwork. Instructional activities spanned all four ICAP modes, revealing a rich design space for collaborative AI literacy interventions. ## Key Findings This systematic review is among the first to map the intersection of [[collaborative-learning]] and [[ai-literacy]], two fields that have largely developed in parallel despite their natural synergies. The review identified 9 studies (2015–2023) that met inclusion criteria for collaborative learning interventions targeting AI literacy outcomes. **The ICAP lens.** The authors applied the ICAP framework — which classifies cognitive engagement as Interactive, Constructive, Active, or Passive — to analyze the instructional activities in each study. This revealed that all four modes of cognitive engagement were represented across the corpus, suggesting that AI literacy instruction benefits from a [[instructional-design]] approach that sequences activities through multiple engagement levels rather than relying on any single mode. This aligns with broader findings about the effectiveness of [[active-learning]] strategies in STEM contexts. **Effectiveness across contexts.** Collaborative learning was found to increase AI literacy outcomes across a wide range of instructional activities, educational settings, and learner populations. The studies spanned both formal classroom environments and informal learning contexts, with some interventions deliberately broadening participation — for example, by involving parents and families alongside students, or by incorporating [[ai-learning-companions-framework]] AI agents as collaborative partners in the learning process. **Classroom-centric but expanding.** Most studies took place in classroom settings, but the review identified emerging patterns of collaboration that extend beyond traditional boundaries: educator-family partnerships, peer-to-peer learning communities, and AI-agent-supported teamwork. These expanding participation models connect to the [[community-centered-ai-education-adults]] paradigm and suggest pathways toward more inclusive [[ai-literacy-equity-programming-policy]]. **Limited but growing evidence base.** The small sample size (9 studies) reflects the nascency of empirical research at this intersection. The authors note the need for more rigorous study designs, larger sample sizes, and longitudinal tracking of AI literacy development — concerns echoed in broader calls for [[stanford-evidence-base-ai-k12-2026]] research. ## Implications The review's findings have direct implications for [[instructional-design]] practice in AI literacy education. The presence of all four ICAP modes across successful interventions suggests that effective AI literacy curricula should intentionally sequence students through passive exposure (e.g., lectures on AI concepts), active manipulation (e.g., hands-on tool use), constructive generation (e.g., creating AI artifacts), and interactive dialogue (e.g., collaborative problem-solving with peers and AI). This multi-modal approach resonates with the [[ai-literacy-continuum-higher-education]] framework. For the [[collaborative-learning]] research community, this review provides a structured taxonomy for designing and evaluating AI literacy interventions. The ICAP framework offers a common vocabulary for comparing approaches and identifying which engagement modes are most impactful for different AI literacy competencies — an important step toward building the cumulative science called for in the [[ai-literacy-assessment-misalignment]] literature. Practically, the findings support the integration of collaborative AI literacy activities into existing [[k-12-ai-education]] curricula and [[faculty-development]] programs. The success of interventions that involve families and community members suggests that AI literacy is not solely a school-based competency but a societal one — a perspective aligned with [[ai-education-global-capacity]] and [[ai-lifelong-learning-policy]]. Looking forward, the review highlights the potential for [[icap-cognitive-engagement-llm-agents]] research to further enrich collaborative AI literacy instruction by designing LLM-based agents that can serve as interactive learning partners within ICAP-structured activities, extending the social dimension of learning beyond human peers. ## Connected Concepts - [[ai-literacy]] - [[ai-literacy-assessment-misalignment]] - [[k-12-ai-education]] - [[higher-ed]] - [[llm]] ## Connected Articles - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[ai-lifelong-learning-policy]] — Artificial Intelligence in Lifelong Learning: Opportunities and Challenges in Adult Education Policy - [[ai-literacy-continuum-higher-education]] — Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education - [[ai-literacy-equity-programming-policy]] — Programming Language Policy as an AI Literacy Equity Problem: A 15-Nation Comparative Analysis - [[community-centered-ai-education-adults]] — Co-Designing Community-Centered AI Education for Adults: A Midwestern Case Study - [[icap-cognitive-engagement-llm-agents]] — Measuring Cognitive Engagement in Collaborative Discourse with an Extended ICAP Framework: Comparing Human Annotation... - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers ## Citation Hingle, A., & Johri, A. (2025). [Collaborative AI Literacy Framework](https://arxiv.org/abs/2508.15111v1). arXiv:2508.15111. --- ## [Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues](https://edtechdev.github.io/aied/articles/history-aware-student-simulation/) A key part of developing large language model (LLM)-powered, automated tutoring tools is student simulation, i.e., using LLMs to role-play as students, which can facilitate tutor model evaluation and training. Existing work mostly focuses on within-dialogue simulation, which lacks context on student knowledge and behavior, partly due to not grounding in past student question-answering or dialogue interactions. In this work, we introduce the task of history-conditioned student simulation, where the goal is to accurately predict student dialogue turns by leveraging information in the student's learning history. We propose a two-component framework in which a profile generator summarizes a student's history and a simulator predicts student turns conditioned on the resulting profile. We train both components with reinforcement learning (RL), yielding profiles optimized for faithful student simulation. - **Beyond within-dialogue simulation:** Duan et al. introduce history-conditioned student simulation, a task where LLM-based student simulators are grounded in past question-answering and dialogue interactions. This moves beyond prior work that simulated students only within a single dialogue context. - **Two-component RL framework:** A profile generator summarizes a student's learning history into a compact profile, and a simulator predicts student dialogue turns conditioned on that profile. Both components are jointly trained via [[reinforcement-learning]] (if that page exists) to optimize for faithful simulation. - **Real-world math tutoring dataset:** The paper collects and evaluates on the first-of-its-kind dataset of real student dialogues and question responses from a math learning platform, grounding the work in authentic [[student-experience]] data rather than synthetic proxies. - **Significant implications for tutor evaluation:** History-aware student simulation enables more realistic evaluation of [[intelligent-tutoring]] systems, allowing developers to test tutor responses against diverse student profiles without requiring live students. This connects to [[kt4eqg-personalized-question-generation]] which also leverages student modeling for personalized learning. - **RL-optimized profiles** ensure that the compressed student representation captures information most relevant to dialogue prediction, rather than using generic summary metrics. This approach could generalize to other [[learning-analytics]] tasks where student state compression is needed. ## Connected Concepts - [[reinforcement-learning]] - [[student-experience]] - [[intelligent-tutoring]] - [[learning-analytics]] ## Connected Articles - [[kt4eqg-personalized-question-generation]] ## Citation Zhangqi Duan, Shuyan Huang, Alexander Scarlatos, Jaewook Lee, Simon Woodhead, & Andrew Lan (2026). [Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues](https://arxiv.org/abs/2605.30051). arXiv:2605.30051. arXiv preprint. --- ## [Interpretable Knowledge Tracing](https://edtechdev.github.io/aied/articles/huang-interpretable-knowledge-tracing-2026/) > **Interpretable Knowledge Tracing** — A novel framework for dialogue-based Knowledge Tracing that explicitly models both student ability and tutor-turn difficulty using Item Response Theory, producing interpretable cognitive quantities from LLM output logits. Addresses two critical gaps in prior work: ignored question difficulty and opaque latent representations that undermine tutor trust. **Shuyan Huang, Alexander Scarlatos, Jaewook Lee, Andrew Lan** (UMass Amherst) — arXiv:2605.01097, May 2026. ## Key Findings Existing dialogue-based Knowledge Tracing (KT) systems suffer from two fundamental limitations. First, they ignore **question difficulty** — prior work relies solely on student knowledge states, leading to inaccurate predictions when tutors scaffold or increase task difficulty across turns. Second, they produce **opaque latent representations** — high-dimensional LLM embeddings lack interpretability, making it impossible for human tutors to understand *why* the system made a particular prediction, which erodes trust. Huang et al. propose an **interpretable difficulty-aware conversational KT framework** built on Llama-3.1-8B-Instruct. The architecture has three modules: 1. **Knowledge Estimator** — Extracts next-token logits for vocabulary tokens "GOOD" and "BAD" from the LLM's output layer after processing the dialogue history. Student ability θ = z^GOOD − z^BAD, producing a scalar that represents the model's latent assessment of the student's current knowledge state. 2. **Difficulty Estimator** — Similarly extracts logits for "HARD" and "EASY" to compute tutor-turn difficulty d = z^HARD − z^EASY. This captures how challenging the current tutor utterance is, independent of the student's ability. 3. **IRT-based Predictor** — Combines θ and d using a 1PL (Rasch) model with a learnable scalar α: p(correct) = 1 / (1 + exp(−α(θ − d))). This mirrors classical [[item-response-theory]] formulations where the probability of a correct response depends on the difference between ability and difficulty. The framework was evaluated on two datasets: **QATD2k** (real-world dialogues from the Eedi math tutoring platform, 1,573 train / 393 test) and **MathDial** (simulated dialogues with GPT-3.5 students and crowd-sourced tutors, 2,235 train / 588 test). It was compared against five deep learning baselines (DKT, DKVMN, SAINT, AKT, simpleKT) and one LLM-based baseline (LLMKT). Results show that LLM-based methods substantially outperform deep learning models, and the proposed framework achieves state-of-the-art performance: **64.29% accuracy** and **65.25 AUC** on QATD2k, **68.82% accuracy** and **76.59 AUC** on MathDial — modestly but consistently outperforming LLMKT across both datasets and metrics. The key insight is that both student knowledge and task difficulty are explicitly represented as **meaningful cognitive quantities** rather than uninterpretable latent vectors. A human tutor can inspect θ and d at any dialogue turn to understand the model's assessment: "the student seems confused (low θ) and this question is particularly hard (high d)." This transparency is essential for building [[ai-tutoring]] systems that tutors can trust and act upon. ## Implications for AI in Education This work bridges a critical gap between the predictive power of LLMs and the interpretability demands of real educational settings. While [[knowledge-tracing]] has traditionally used deep learning models like DKT and SAINT that trade interpretability for accuracy, Huang et al. demonstrate that LLMs can produce both — competitive or superior accuracy while yielding transparent, inspectable intermediate representations. The difficulty-aware component addresses a well-known blind spot in [[student-modeling]]: student performance is a joint function of knowledge and task demands. By disentangling these, the framework avoids falsely attributing poor performance to low knowledge when a tutor has simply asked a harder question, or conversely, mistaking scaffolded success for mastery. This connects to broader work on [[knowledge-tracing-irt]] that uses IRT to add structure to neural KT models. The logit-extraction approach is notable for its simplicity — rather than training separate classifiers or using complex prompting strategies, it repurposes the LLM's own vocabulary probabilities as cognitive signals. This technique relates to emerging work on using LLM internal representations for educational assessment, including [[cold-start-knowledge-tracing-safeinsights]] and [[neural-symbolic-knowledge-tracing]]. For practical deployment, the framework's interpretability enables several downstream applications: tutors can receive real-time alerts when student ability drops below a threshold; difficulty estimates can guide adaptive scaffolding decisions; and longitudinal θ trajectories can track learning over multiple sessions. The explicit difficulty model also supports better [[ai-tutor-authoring-promptdecipher]] by helping content creators understand which tutor utterances are most effective at different ability levels. Future work should extend the framework to multi-skill settings (where students may have different abilities across different knowledge components), explore fine-tuning strategies that further improve the knowledge and difficulty estimators, and validate the approach in live tutoring deployments with real tutor feedback on interpretability quality. ## Connected Concepts - [[ai-tutoring]] - [[cold-start-knowledge-tracing-safeinsights]] - [[item-response-theory]] - [[knowledge-tracing]] - [[rag]] ## Connected Articles - [[ai-tutor-authoring-promptdecipher]] — PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions - [[knowledge-tracing-irt]] — Interpretable Knowledge Tracing via IRT - [[llm-student-modeling-memory]] — LLM Student Modeling and Long-Term Memory Architecture - [[mbp-kt-meta-behavioral-knowledge-tracing]] — MBP-KT: Learning Global Collaborative Information from Meta-Behavioral Pattern for Enhanced Knowledge Tracing - [[neural-symbolic-knowledge-tracing]] — Neural-Symbolic Knowledge Tracing - [[at-risk-students-ml-prediction]] — Analysis and Prediction of At-Risk Students Using Machine Learning Algorithms - [[calibrating-trustworthiness-llm-education-2026]] — Calibrating Trustworthiness: Co-Designing Metrics and Visualizations for Evaluating LLMs in Education - [[code-gen]] — CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation - [[correct-answer-trap-misconceptions]] — The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions - [[cross-subject-validity-delayed-start]] — Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[difficulty-aware-dialogue-kt]] — Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues - [[educlaw-bench-pedagogical-llm-agents-2026]] — EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners - [[explainable-probabilistic-kt]] — Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[nie-personavlm-long-term-personalization-2026]] — PersonaVLM: Long-Term Personalization for AI Tutors - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components - [[precision-education-student-digital-twins-2026]] — From Precision Medicine to Precision Education: A Vision for AI-Powered Student Digital Twins, Preventive Student Suc... - [[quantum-education-its]] — Quantum Education Intelligent Tutoring - [[reinforcement-learning-measurement-model-assessment]] — Reinforcement Learning Measurement Model - [[reliable-programming-kt]] — Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental... - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[temporal-smoothness-debiased-kt]] — Temporal Smoothness Doubly Robust Learning for Debiased Knowledge Tracing - [[zhang-tutormoments-2026]] — When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle ## Citation Amherst), A.S.H.A.S.J.L.A.L., gaps:, E.D.K.T.S.F.T., simpleKT, D.L.D.D.S.A., & LLMKT, L. (2026). [*Interpretable Knowledge Tracing*](https://arxiv.org/abs/2605.01097) --- ## [Human-AI Co-Mentorship in Project-Based Learning: A Case Study in Financial Forecasting](https://edtechdev.github.io/aied/articles/human-ai-co-mentorship/) > A pedagogical model where human mentors and AI tools jointly support student learning in project-based contexts. Human mentors provide conceptual guidance, debugging, and problem formulation support; AI tools accelerate execution, code generation, and rapid iteration. Demonstrated by Chawla et al. (2026) in a financial forecasting project with high-school students. # Human-AI Co-Mentorship ## Definition A pedagogical model where human mentors and AI tools jointly support student learning in project-based contexts. Human mentors provide conceptual guidance, debugging, and problem formulation support; AI tools accelerate execution, code generation, and rapid iteration. Demonstrated by Chawla et al. (2026) in a financial forecasting project with high-school students. ## Key Insight: Workflow Design over Traditional Instruction Instead of "learn theory first, apply later," students identified the sequence of steps needed to solve their problem and used AI tools to execute each step. This just-in-time approach enabled students with limited backgrounds in AI and finance to build meaningful predictive models. AI tools served as scalable accelerators, not replacements for human mentorship. ## Connections to Agentic Workflows This model bridges [[agentic-workflows-education]] and practical classroom implementation. Where agentic workflows describe AI agent paradigms (reflection, planning, tool use, multi-agent collaboration), co-mentorship shows how those paradigms manifest in human-AI educational partnerships. The daily stand-up structure echoes the reflection loop in [[agentic-education-coding]]. ## Connected Concepts - [[math-education]] - [[pedagogical-agent]] - [[reinforcement-learning]] - [[intelligent-tutoring]] - [[socratic-ai-dialogue]] - [[active-learning]] - [[teacher-role]] - [[lifelong-learning]] ## Connected Articles - [[agentic-workflows-education]] - [[agentic-education-coding]] ## Citation Khvatskii, A.F.C.A.C.R.S.J.G.G. (2026). [*Human-AI Co-Mentorship in Project-Based Learning: A Case Study in Financial Forecasting*](https://arxiv.org/abs/2605.05144) --- ## [What do you mean by human-AI collaboration: Prerequisite functions and the affordances needed to achieve it](https://edtechdev.github.io/aied/articles/human-ai-collaboration-prerequisite-functions/) > Asks what is gained and lost when 'collaboration' is applied freely to human-AI interaction. Argues true collaboration requires symmetric/negotiated relationship, shared goals, low and shifting division of labor, interactive exchange, and mutual modeling. Introduces a 5-level diagnostic taxonomy: Transactional, Situational, Operational, Praxical, and Synergistic. Only Synergistic satisfies full collaborative conditions. Most current human-AI interaction is consultation, governance, delegation, or instruction. - Introduces a 5-level taxonomy of human-AI teaming; most current AI interaction is consultation/delegation, not true collaboration. ## Key Findings - The chapter reconstructs the requirements that a situation, an interaction, and a set of cognitive processes have historically had to meet before being called collaborative, returning to longstanding accounts of collaborative learning. - **True human-AI collaboration** requires a partly symmetric and negotiated relationship, shared and negotiable goals, a low and shifting division of labour, interactive and synchronous exchange, and mutual modelling, grounding, and socially shared regulation. - Reviewing process-sensitive empirical studies of writing and problem solving, the author shows that **most current human-AI interaction is better described as consultation, governance, delegation, or instruction** rather than collaboration. - The chapter introduces a **five-level diagnostic taxonomy of human-AI teaming — transactional, situational, operational, praxical, and synergistic** — defined by the affordances an AI system exhibits, with only the highest level beginning to satisfy the conditions the tradition places on collaboration. - The author argues that most of the **prerequisite functions** an AI must possess for collaboration are present-day engineering choices rather than capabilities to be awaited. ## The Five-Level Taxonomy The taxonomy runs from **transactional** interactions (discrete, affordance-poor exchanges) up through **situational**, **operational**, and **praxical** teaming to **synergistic** collaboration, where evidence of collaboration should be sought in the trajectory of the human-AI unit rather than in output quality or user satisfaction alone. The high bar is not intended as a stick with which to beat existing systems but as a target that lets the field name what AI systems already do well without pretending it is the collaboration they do not yet achieve. ## Implications for AI in Education For [[human-ai-collaboration]] research and [[teacher-role]] practice, the chapter warns against inflating a precise construct into a slogan: calling all useful human-AI interaction "collaboration" obscures what would actually be required. Because most required functions are engineering choices, designers of [[ai-tutoring]] and [[collaborative-learning]] environments can treat the taxonomy as a specification of design targets — and researchers can measure collaboration in the trajectory of the human-AI unit rather than in user satisfaction alone, keeping the stronger meaning of the word worth preserving. ## Connected Concepts - [[ai-tutoring]] - [[collaborative-learning]] - [[affective-tutoring]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[help-seeking]] - [[pedagogical-agent]] - [[pedagogical-llm-training]] ## Connected Articles - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[modular-educational-llm-agency]] — Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistance - [[teacher-student-agency-orchestration]] — Balancing Teacher and Student Agency: Co-Orchestration Tool Design Supporting Real-Time Dynamic Pairing - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation ## Citation Mutlu Cukurova (2026). [What do you mean by human-AI collaboration: Prerequisite functions and the affordances needed to achieve it](https://arxiv.org/abs/2606.15509). arXiv:2606.15509. Handbook of AI and the Future of Education (forthcoming). --- ## [Human-AI collaboration in higher education: Exploring the impact of technology expectations and distrust](https://edtechdev.github.io/aied/articles/human-ai-collaboration-trust-expectations/) > **Synthesis:** Human-AI collaboration in higher education: Exploring the impact of technology expectations and distrust ## Key Findings - Drawing on **expectation confirmation theory (ECT)**, the study models how students' expectations about generative AI and their confirmation of those expectations shape intentions to collaborate with GenAI. - An **online survey of 245 higher education students** (59.2% female, 68.2% aged 20–25, including international students at a Danish university) who already use GenAI found positive and significant relationships between **effort expectation** and **performance expectation** (β = 0.530, p < .001), and from performance expectation to confirmation (β = 0.571, p < .001), as ECT predicts. - **Positive confirmation of GenAI collaboration** significantly influences students' intentions to collaborate (β = 0.587, p < .001), which in turn positively impacts their behavior (β = 0.513, p < .001); the model explains 50.2% of the variance in performance expectation, 55.3% in confirmation, 47.2% in collaboration intention, and 31.2% in collaboration behavior. - **GenAI distrust negatively moderates** two key links: (1) the relationship between effort expectation and performance expectation (β = −0.169, p < .01), and (2) the relationship between expectation confirmation and intentions to collaborate with GenAI (β = −0.099, p < .05); it did not significantly moderate the other three hypothesized paths. - Perceived drawbacks center on the **risk of plagiarism (72.2% agree or strongly agree)**, **inaccuracy of responses (70.6%)**, and **flawed referencing (58.0%)**, while **time saving (89.0%)**, user-friendliness (79.2%), and enhanced productivity (76.7%) dominate perceived benefits — evidence that benefit and drawback perceptions coexist rather than exclude each other. - The work **extends the theoretical boundaries of ECT** into the human-AI collaboration context and clarifies the distinct role of distrust in educational technology adoption. ## Study Design & Method The authors administered an online survey to higher education students who use GenAI at a Danish university, measuring the core ECT constructs — effort expectation, performance expectation, confirmation, and collaboration intention — along with a measure of distrust toward GenAI, using items drawn from prior expectation-confirmation and information-systems adoption research. The hypothesized model was estimated with partial least squares structural equation modeling (PLS-SEM) in SmartPLS 4 with 5000 bootstrap samples, with all constructs meeting reliability and validity thresholds and good model fit (SRMR = 0.070; NFI = 0.884). The moderation analysis tests whether distrust weakens the expectation-driven pathway to collaboration intention, and an importance-performance map analysis (IPMA) identified confirmation as the most important predictor of both intentions and actual collaboration while effort expectation showed the highest performance. ChatGPT was the most used tool, and reported collaboration concentrated on higher-level cognitive tasks — explaining and evaluating theories and concepts, editing and proofreading text, summarizing literature, and generating research ideas — suggesting augmentation of learning more than automation. This design positions the study as one of the first to bring ECT's confirmatory framework to bear on [[human-ai-collaboration]] in [[higher-ed]] rather than on conventional information systems. ## Implications for AI in Education For educators and GenAI practitioners, the findings suggest that **managing students' initial expectations and addressing distrust** are as important as the tool's raw capability. If distrust dampens the effect of confirmation on collaboration intention, then onboarding, transparency about limitations, and trust-building interventions should be part of any responsible deployment of [[generative-ai]] in the classroom. The results also support designing feedback and scaffolding that help students form accurate expectations, so that positive experiences translate into sustained collaboration — a practical complement to [[ai-literacy]] efforts that teach students how to evaluate AI outputs critically. Notably, distrust did not distort students' evaluations of GenAI's benefits — students acknowledged the tool's usefulness while remaining skeptical of its outputs — reinforcing that expectation management and transparency are complements to, not substitutes for, capability. For [[student-experience]] design, the coexistence of strong benefit and drawback perceptions suggests responsible-use guidance should address plagiarism risk, inaccuracy, and flawed referencing directly rather than assuming skepticism will resolve itself. ## Limitations The authors note that the rapid evolution of GenAI makes it difficult to separate initial expectations from early experiences, since both occur simultaneously, and that this study could not measure them separately. Findings derive from a single institutional context — 245 students at one Danish university — and cannot be assumed to generalize to the global student population; multi-institutional and cross-national samples are needed. The distrust construct was measured with only two items capturing skepticism toward the technology and lack of trust in its outputs, and two hypothesized moderation paths (H6c and H6e) showed positive but non-significant coefficients that warrant further examination through qualitative approaches, larger samples, or experimental designs. ## Connected Concepts - [[higher-ed]] - [[student-experience]] - [[ai-literacy]] - [[math-education]] - [[prompt-engineering]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[plagiarism-detection]] ## Connected Articles - [[student-mental-models-genai]] — Uncovering Students' Mental Models of Generative Artificial Intelligence - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis ## Citation Razmerita, L., Zheng, X., & Allen, J. P. (2026). [*Human-AI collaboration in higher education: Exploring the impact of technology expectations and distrust*](https://doi.org/10.1016/j.caeai.2026.100625). --- ## [Human Autonomy and Sense of Agency in Human-Robot Interaction: A Systematic Literature Review](https://edtechdev.github.io/aied/articles/human-autonomy-agency-hri-review-2025/) > **Synthesis:** Glawe, Schmeckel, Brauner, and Ziefle (2025) systematically review empirical studies on human autonomy and sense of agency in human-robot interaction (HRI), aiming to bridge the gap between design frameworks and regulatory demands (e.g., the EU AI Act, IEEE Ethically Aligned Design) on one hand and available empirical evidence on the other. Using the PRISMA workflow, they queried five databases and identified 22 empirical studies published between 2011 and early 2024. Across both concepts, separate methodological strands emerged: self-determination-theory-based measures of autonomy via psychometric scales, and neuroscientific measures of agency (primarily intentional binding). ## Key Findings - **Human autonomy and sense of agency are core human values** foundational to well-being, intertwined in definition and central to current ethical and governance discourses on AI and cyber-physical systems (EU AI Act, IEEE Ethically Aligned Design). - A **PRISMA-guided systematic review** of five databases identified **22 empirical studies** published 2011–early 2024 on autonomy and sense of agency in HRI. - **Two distinct methodological strands emerged:** self-determination-theory-based psychometric measures of autonomy, and neuroscientific measures of agency (primarily intentional binding). - The review bridges design frameworks and regulatory demands with empirical evidence, highlighting gaps and directions for HRI design and governance. ## Study Design & Method This is a **systematic literature review** following the PRISMA workflow. The authors queried five databases and screened the literature to identify 22 empirical studies published between 2011 and early 2024 examining human autonomy and sense of agency in human-robot interaction. They analyzed the conceptualization and measurement of autonomy and agency across the studies, identifying distinct methodological strands (psychometric/SDT-based measures of autonomy; neuroscientific/intentional-binding measures of agency) and synthesizing implications for design frameworks and regulatory compliance. ## Implications for AI in Education The review provides a foundational evidence base for understanding how [[human-robot-interaction]] and [[social-robots]] affect [[agency]] and autonomy in learning contexts. As robots are increasingly used in education, preserving learners' sense of agency is a core ethical and design concern, connecting to [[ethics]], [[well-being]], and the governance of AI in education. The review's synthesis of measurement approaches (SDT-based autonomy scales, intentional-binding measures of agency) offers researchers tools for evaluating how educational robots support or undermine learner agency, and informs responsible design of embodied AI in classrooms. ## Limitations The review is limited to 22 studies, reflecting a relatively young and methodologically fragmented literature; the two concepts (autonomy and agency) are measured with very different instruments, making cross-study synthesis challenging. The temporal scope (2011–early 2024) predates the widespread use of LLM-powered social robots, so recent developments may be underrepresented. The review focuses on HRI broadly rather than education specifically. ## Connected Concepts - [[human-robot-interaction]] - [[social-robots]] - [[agency]] - [[ethics]] - [[well-being]] - [[self-determination-theory]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[task-context-trust-educational-hri-2026]] — Task Context Shapes Trust in Educational HRI - [[knowledge-based-design-generative-social-robots-2026]] — Knowledge-Based Design for Generative Social Robots ## Citation Glawe, F., Schmeckel, T., Brauner, P., & Ziefle, M. (2025). *Human autonomy and sense of agency in human-robot interaction: A systematic literature review*. arXiv:2509.22271. --- ## [Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use](https://edtechdev.github.io/aied/articles/human-llm-collaborative-coding-k12-educator-ai/) > **Alex Liu, Min Sun, Lief Esbenshade, Michael Xiao, Victor Tian, Zachary Zhang, Kevin He** — arXiv preprint (2026). ## Synthesis A multi-phase human-LLM collaborative pipeline adapted open, axial, and selective coding to build a hierarchical codebook from 45,000 messages exchanged between K-12 educators and a generative AI platform — an instance of LLMs as analytic assistants at a scale manual coding cannot match. LLMs generated candidate labels and structured annotations at scale across three phases, while human researchers retained conceptual authority over category definitions, merging decisions, and interpretive frameworks. The resulting 72-item codebook was validated through systematic human coding of an independent 2,560-message sample, with reliability established via set-valued agreement measures; human coders extended the instrument with five codes the LLM-assisted phases had not surfaced. The article provides a procedural account (what LLMs can do, in which phases, under what safeguards) directly relevant to AIED research methodology and to conceptualizing how educators actually use generative AI platforms. ## Connected Concepts - [[formative-assessment]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[k-12-ai-education]] - [[ai-education]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[automated-essay-scoring]] ## Connected Articles - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth - [[connected-ai-lesson-planning-vietnam]] — ConnectED: A Curriculum-Aligned AI System for Vietnamese Instructional Lesson Planning and Student Learning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[chat-debugging-human-ai-collaboration-circuits]] — Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits - [[post-covid-ict-career-aspirations]] — Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills? ## Citation Liu, A., Sun, M., Esbenshade, L., Xiao, M., Tian, V., Zhang, Z., & He, K. (2026). [Human-LLM collaborative inductive coding for conceptualizing K-12 educator AI use](https://arxiv.org/abs/2607.28889). arXiv:2607.28889. --- ## [Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis](https://edtechdev.github.io/aied/articles/human-vs-llm-ordered-coding/) > **Authors:** Kamila Misiejuk, Sonsoles López-Pernas, Eduardo Araujo Oliveira, Brendan Eagan, Mohammed Saqr **Source:** Computers and Education: AI, Vol 11 — Open Access (CC BY 4.0) **Source:** Computers and Education: AI, Vol 11 — Open Access (CC BY 4.0) ## Key Findings - Automating qualitative coding of learner text has been a long-standing goal of **learning analytics** because it is an essential step toward timely, scalable feedback — yet it is especially hard for **ordered coding schemes** required by temporal analytical methods, where a single utterance can carry more than one code and the assignment order matters. - The problem goes beyond multi-class and multi-label classification, which means it **cannot be easily tackled with classic language models such as BERT**; the study instead evaluates modern generative LLMs — here, gemini-2.5-flash, prompted with a sliding context window of up to five preceding messages from the same conversation. - The empirical base was **6,013 Discord messages** from small-group collaboration in three master's-level courses at the University of Eastern Finland, coded by two researchers using a hybrid eight-code scheme covering cognitive and regulatory processes, and in parallel by the LLM. - The paper makes two main contributions: it presents **two evaluation approaches** for assessing the quality of ordered data coding and the usability of LLMs in automatically coding ordered processes, and it demonstrates an **LLM prompting method that leverages a consistent context window**. - Results reveal **systematic and statistically significant differences between LLM and human coding** across structural, transitional, and code-level metrics, for both binary and ordered tasks (overall code-frequency association χ2(7) = 1427, p < 0.001). - Frequency discrepancies were large: the LLM over-assigned **Reaction (2,107 vs. 823 human codes)** and **Discussion (1,092 vs. 756)**, while humans produced far more **Monitoring (1,185 vs. 283)** and **CoRegulation (1,352 vs. 770)**; only Coordination (2,591 vs. 2,439) and Socializing (1,329 vs. 1,182) aligned within chance. - Binary agreement was modest: overall accuracy ranged **0.693–0.888**, recall was very low for **Feedback (0.131)** and **Monitoring (0.100)**, and Cohen's kappa ranged **0.090–0.539** with most categories below 0.400. - The LLM detected multi-step transition patterns far more often than humans (e.g., **Coordination→Socializing: 46 vs. 378 occurrences**; **Discussion→Coordination: 19 vs. 268**), and its detection was position-dependent — Discussion and Monitoring found early in messages, socio-emotional codes late — whereas human coding was more evenly distributed across message positions. - Because classification errors can propagate through automated feedback systems, **relying on LLM outputs risks amplifying inaccuracies** and producing misleading interpretations of learning processes. ## Study Design & Method The researchers treat coding quality not as a single accuracy number but along multiple dimensions: **structural** properties of the coded sequence, **transitional** patterns between consecutive codes, and **code-level** agreement for binary and ordered assignments. LLM outputs were produced with a prompting strategy that maintains a consistent context window, giving the model access to surrounding textual context needed for accurate interpretation (up to five preceding messages per utterance, with position slots T1–T8 inside messages). Each LLM-coded output was then compared against human coding using the two proposed evaluation approaches, including Transition Network Analysis (TNA) with permutation tests and centrality comparisons — e.g., significant betweenness and in-strength differences for Discussion, Feedback, Socializing, CoRegulation, and Consolidation. ## Implications for AI in Education The findings are a caution for the automated-feedback pipeline: LLMs can produce plausible-looking coding of temporal learner data, but systematic deviations from human judgment can cascade into downstream temporal analyses. The LLM's tendency to foreground surface-level social exchange (Socializing, Reaction) while under-representing regulatory and collaborative processes (Coordination, CoRegulation, Monitoring) risks painting a more rigid, socially-driven picture of group learning than humans would code. Practitioners should treat LLM coding as a **human-in-the-loop** proposition, validating outputs with the proposed evaluation metrics before feeding them into automated feedback systems — a relevant consideration for [[automated-assessment]] and [[educational-nlp]] work in [[learning-analytics]], and for any analysis of [[self-regulated-learning]] processes built on LLM-coded transcripts. ## Limitations The study evaluated a single dataset drawn from three courses at one institution where Discord was the collaboration medium, so generalizability to other learning processes, discourse types, and educational levels is untested. It relied on a single LLM (gemini-2.5-flash), a single prompting strategy, and a single codebook — even slight prompt changes can alter results, and a simpler codebook might have performed better. The context window was fixed at up to five preceding messages, with no systematic sensitivity analysis of window size. Finally, human coding is treated as "ground truth" even though qualitative coding is inherently interpretive; in ordered coding this is compounded because disagreement can stem not only from code presence but from how text is segmented into codes. ## Connected Concepts - [[automated-assessment]] - [[self-regulated-learning]] - [[educational-nlp]] - [[automated-essay-scoring]] - [[learning-analytics]] - [[open-source]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] ## Connected Articles - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[socratic-tests-conversational-assessment]] — The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations - [[automatic-short-answer-grading]] — Automatic Short Answer Grading with LLMs - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming ## Citation Misiejuk, K., López-Pernas, S., Oliveira, E. A., Eagan, B., & Saqr, M. (2026). [*Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis*](https://doi.org/10.1016/j.caeai.2026.100649). --- ## [It Felt a Bit Eerie": Exploring Humanlike Interactions During Collaborative Writing with an Artificial Agent](https://edtechdev.github.io/aied/articles/humanlike-ai-collaborative-writing/) # Humanlike AI in Collaborative Writing This comparative user study (n=48) examines how the temporal and visual dimensions of AI collaboration shape the experience of [[writing-education|writing tasks]], revealing that humanlike design features in AI agents create both positive social expectations and unexpected social costs. ## The Humanlike-to-Machinelike Spectrum Three AI-assisted text editor variants were tested along two dimensions: - **Temporal**: synchronous (humanlike) vs. asynchronous (machinelike) suggestions - **Visual**: presence (humanlike) vs. absence (machinelike) of a cursor Synchronous suggestions increased efficiency but led to contextual misalignment — the AI's suggestions didn't fit the writer's intent. A visual cursor improved perceived intent understanding but evoked feelings of surveillance that participants described as "eerie." This maps onto emerging concerns in [[ai-peer-feedback-systems]] about the social dynamics of AI interaction in educational contexts. ## Implications for AI Writing Tools These findings have direct relevance for [[mindcopilot-llm-co-writing|LLM co-writing tools]] and the broader ecosystem of [[generative-ai]] writing assistants. The uncanny valley effect in AI collaboration suggests that anthropomorphic design choices must be paired with careful expectation management — without the mutual alignment natural in human-human collaboration, humanlike features can backfire. The study also connects to [[socially-fluent-ai-identity-detection|research on AI social fluency]]: users' inability to detect AI teammates above chance takes on new significance when AI interfaces are designed to feel more human. If students can't reliably distinguish human from AI collaborators, the design choices described here could amplify both the benefits and costs of AI in [[writing-education]]. ## Connected Concepts - [[writing-education]] - [[generative-ai]] ## Connected Articles - [[ai-peer-feedback-systems]] - [[mindcopilot-llm-co-writing]] - [[socially-fluent-ai-identity-detection]] ## Citation Yin, M., Chiang, A., Cox, S. R., & Xiao, R. (2026). ["It felt a bit eerie": Exploring humanlike interactions during collaborative writing with an artificial agent](https://arxiv.org/abs/2605.24729). arXiv:2605.24729. cs.HC. --- ## [Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations](https://edtechdev.github.io/aied/articles/hybrid-e-assessment-semi-automated-grading/) **Hartwig Grabowski, Michael Canz** — cs.AI, cs.CV, cs.CY This paper identifies the didactic narrowing caused by fully digital e-assessment (overuse of closed question formats) and proposes a hybrid approach that retains paper-based, problem-oriented examination tasks while enabling semi-automated grading. The core technical innovation is applying vision-capable LLMs to recognize handwritten characters in structured answer fields under realistic exam conditions, combined with a two-pass validation principle and comparison against a solution key to reduce misclassifications. The approach addresses organizational, technical, and legal constraints that become relevant in large student cohorts. This work connects [[automated-grading]] research to real-world assessment practice by acknowledging that fully digital assessment often degrades assessment quality, even as institutions push for scalability. The hybrid approach could inform [[formative-assessment]] redesign in large-enrollment STEM courses. ## Connected Concepts - [[automated-grading]] - [[formative-assessment]] ## Connected Articles - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions ## Citation Grabowski, H., & Canz, M. (2026). [*Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations*](https://arxiv.org/abs/2606.08855). arXiv:2606.08855. --- ## [Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs](https://edtechdev.github.io/aied/articles/hybrid-human-ai-tutoring-differentiated/) ## Key Findings In a large-scale quasi-experiment with 635 students (grades 5-8), hybrid human-AI tutoring produced substantial gains over AI-only tutoring: +25% time on task, +36% skill proficiency, and +61% standardized academic growth. The study's core innovation was a **differentiated tutoring policy**: students below the grade median received proactive human-initiated support, while those above received reactive, on-demand support. Proactive tutoring showed marginally higher growth (+75%, p = .065) and was particularly beneficial for students farthest below the cutoff, helping narrow achievement gaps. ## Design Implications This work provides evidence that differentiated human-AI instruction is a practical, cost-effective strategy for scaling hybrid tutoring. Rather than providing equal human support to all students, systems should allocate human tutor attention where it yields the greatest marginal benefit — to struggling learners. The findings align with prior work on [[ai-tutor-effectiveness-review]] showing that human-AI combinations outperform AI-only approaches. ## Connection to Broader AIED Research This study extends the taxonomy in [[tutoring-specific-vs-general-ai]] by showing that the human component remains differentiable and valuable even alongside AI. It also connects to [[personalized-learning]] frameworks and speaks to [[equity-in-ai-education]] concerns about achievement gaps. ## Connected Concepts - [[personalized-learning]] - [[equity-in-ai-education]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[tutoring-specific-vs-general-ai]] ## Citation Gurung, A., Gao, G., Gutterman, J., Thomas, D. R., Gupta, S., Branstetter, L., Brunskill, E., Aleven, V., & Koedinger, K. R. (2026). [Improving hybrid human-AI tutoring by differentiating human tutor roles based on student needs](https://arxiv.org/abs/2605.11155). Proceedings of the 19th International Conference on Educational Data Mining (EDM'26). arXiv:2605.11155. --- ## [Hybrid intelligence feedback systems in design thinking development: Stage-specific insights on pedagogical effects and characteristics of generative AI and instructors](https://edtechdev.github.io/aied/articles/hybrid-intelligence-feedback-systems-in-design-thinking-development-stage-specif/) > **Synthesis:** This study compares the pedagogical effects on students' design thinking and students' perceptions of feedback systems by GenAI and human instructors. A within-class randomized experimental design with 80 undergraduates revealed no significant overall difference in learning gains but identified respective stage-specific strengths: GenAI proved more effective during the empathise stage (superior timeliness and privacy), while human instructors excelled in the prototype stage (contextual anchoring). Comparable effects yet complementary functions were observed in other stages. ## Key Findings This study compares the pedagogical effects on students' design thinking and students' perceptions of feedback systems by GenAI and human instructors. A within-class randomized experimental design with 80 undergraduates revealed no significant overall difference in learning gains but identified respective stage-specific strengths: GenAI proved more effective during the empathise stage (superior timeliness and privacy), while human instructors excelled in the prototype stage (contextual anchoring). Comparable effects yet complementary functions were observed in other stages. ## Connected Concepts - [[generative-ai]] - [[design-thinking]] ## Connected Articles - [[sequenced-ai-feedback-learning]] - [[multimodal-ai-feedback-learning]] - [[liang-genai-systematic-review-human-ai-2026]] - [[learning-by-chatting-genai-impact]] ## Citation Fei, X. (2026). [*Hybrid intelligence feedback systems in design thinking development: Stage-specific insights on pedagogical effects and characteristics of generative AI and instructors*](https://doi.org/10.1111/bjet.70080). *British Journal of Educational Technology.* --- ## [Hypergamigication Through Integrating Game Engines and Learning Management Systems: Ender's Game](https://edtechdev.github.io/aied/articles/hypergamification-game-engine-lms/) > **Araz Yusubov, Michael Bechtel, Tangiz Alizada** — arXiv preprint (2026). ## Synthesis The paper proposes 'hypergamification': using a comprehensive game environment generated from LMS content rather than bolting isolated game design elements onto a course. The key architectural idea is bidirectional integration — the game world is built from the LMS's actual learning content, and player activity flows back into the LMS. A working pilot implementation is demonstrated: an importable Unity package for Blackboard integration plus a demo game ('Ender's Game') built with the package, showing that course content can drive a playable environment. The approach differs from typical gamified LMS features (badges, points, leaderboards) by making the game the primary interface, with the potential to raise engagement but at the cost of substantial development effort and platform coupling. The authors discuss limitations (integration complexity, content-authoring burden, assessment alignment) and outline future work, positioning hypergamification as a design direction for fully game-based course delivery rather than an evaluated intervention. ## Connected Concepts - [[active-learning]] - [[affective-computing]] - [[math-education]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[administrator]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] ## Connected Articles - [[physiological-signals-exam-outcomes-ml]] — Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning - [[savvy-student-attention-video-learning]] — SAVVY: Student Attention Visualization for Video-based Learning Analysis - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[wordstream-glass-learning-analytics]] — Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use ## Citation Yusubov, A., Bechtel, M., & Alizada, T. (2026). [Hypergamigication through integrating game engines and learning management systems: Ender's Game](https://arxiv.org/abs/2607.29300). arXiv:2607.29300. --- ## [Measuring Cognitive Engagement in Collaborative Discourse with an Extended ICAP Framework: Comparing Human Annotation, In-Context Learning, and Reflective LLM Agents](https://edtechdev.github.io/aied/articles/icap-cognitive-engagement-llm-agents/) > **Lan Anh Do, Hanling Jiang, Shuchin Aeron, Ayanna K. Thomas** — CogSci 2026 (accepted full paper). **Lan Anh Do, Hanling Jiang, Shuchin Aeron, Ayanna K. Thomas** — CogSci 2026 (accepted full paper). ## Synthesis This study applies an extended 7-point ICAP framework (Interactive, Constructive, Active, Passive) to characterize cognitive engagement in collaborative dialogue, comparing trained human annotators with LLM-based labeling: in-context learning (ICL), zero-shot prompting, and self-reflective agents. Human interrater reliability was robust across framework refinement stages (kappa = 0.906–0.998), far exceeding ICL-based annotation (kappa = 0.541–0.609) — a large gap between human and LLM labeling of engagement. The human-refined framework improved human agreement (Δkappa = 0.10) but gave only modest gains to ICL LLMs (Δkappa < 0.04); agent-refined frameworks improved cross-model agreement but stayed below human-refined performance. Findings highlight the promise of reflective-agent approaches for scaling engagement measurement while underscoring that LLM annotation of learning processes still trails trained humans — relevant for learning analytics pipelines that rely on automated discourse coding. ## Connected Concepts - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[knowledge-tracing]] - [[administrator]] - [[help-seeking]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[rag]] ## Connected Articles - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[llm-facilitation-timing-online-discussions]] — To Facilitate or not to Facilitate: Human and LLM Facilitator Tendencies in Online Discussions - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[hingle-collaborative-ai-literacy-2025]] — Collaborative AI Literacy Framework - [[spritz-ai-disciplinary-mediation-student-teams-2026]] — Exploring AI-Supported Disciplinary Mediation in Student Project Teams' Text-Based Communication - [[prober-ai-inquiry-writing]] — Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing ## Citation Do, L. A., Jiang, H., Aeron, S., & Thomas, A. K. (2026). [Measuring cognitive engagement in collaborative discourse with an extended ICAP framework. CogSci 2026](https://arxiv.org/abs/2607.28651). arXiv:2607.28651. --- ## [ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring](https://edtechdev.github.io/aied/articles/icle-plus-plus-essay-scoring/) Introduces ICLE++, a new annotated corpus of persuasive student essays that addresses critical limitations of the dominant ASAP benchmark in [[automated-essay-scoring]] research. Unlike ASAP — used by virtually all recent AES models but limited to U.S. grade 7–10 native-English essays — ICLE++ provides both holistic scores and fine-grained trait-specific annotations, enabling evaluation of cross-corpus generalization, multi-trait scoring, and cross-prompt scoring. The authors demonstrate that models trained on trait-specific annotations transfer better across corpora than those trained on holistic scores alone, making ICLE++ a foundational resource for the next generation of AES research. ## Key Contributions - **New annotated corpus:** ICLE++ provides persuasive student essays annotated with holistic scores and multiple trait-specific scores, filling a gap left by the field's over-reliance on ASAP - **Cross-corpus generalization:** Models trained on ASAP often fail to generalize to other corpora (e.g., TOEFL essays by English learners, essays written without time constraints) — ICLE++ enables systematic evaluation of this transfer - **Trait-level scoring advances:** Fine-grained trait annotations support multi-trait scoring and cross-prompt scoring, moving AES beyond single holistic score prediction - **Addresses ASAP limitations:** ASAP's confounding variables — essay length as a proxy for quality in timed settings, native-speaker-only population — are well-documented; ICLE++ provides a complementary benchmark ## Implications for AI in Education ICLE++ strengthens the [[automated-essay-scoring]] research ecosystem by providing a second major annotated corpus that can test whether AES advances generalize beyond ASAP. This connects directly to [[assessment-validity]] concerns: if models trained on one corpus fail on another, their scores are not valid measures of writing quality. The trait-specific annotations also enable more diagnostically useful AES — rather than a single score, systems can report strengths and weaknesses across specific writing dimensions, supporting [[formative-assessment]] and connecting to psychometric frameworks like [[item-response-theory]]. The work complements [[psyscore-essay-scoring-zpd-feedback]]'s psychometrically-aware trait scoring and [[choi-anchor-aes-prompting-2025]]'s finding that prompting-based AES with anchor papers approaches human reliability. While Choi et al. show what prompting can achieve holistically, ICLE++ provides the annotated data needed to evaluate whether trait-level scoring generalizes — a question also explored by [[self-referential-l2-writing-llm-assessment]] for L2 learners. The cross-corpus generalizability question likewise connects to [[ai-scoring-language-bias-physics]]'s finding that AI scoring systematically underestimates linguistically diverse students, and to [[aiawe-automated-writing-evaluation]]'s demonstration that open-source models can achieve strong AES performance on specific datasets. ## Connected Concepts - [[automated-essay-scoring]] - [[automated-grading]] - [[writing-education]] - [[benchmark]] - [[formative-assessment]] - [[educational-measurement]] - [[assessment-validity]] - [[item-response-theory]] - [[higher-ed]] ## Connected Articles - [[psyscore-essay-scoring-zpd-feedback]] - [[choi-anchor-aes-prompting-2025]] - [[ai-scoring-language-bias-physics]] - [[self-referential-l2-writing-llm-assessment]] - [[aiawe-automated-writing-evaluation]] ## Citation Shengjie Li, Vincent Ng (2026). [ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring](https://arxiv.org/abs/2607.27671). arXiv:2607.27671. cs.CL. --- ## [Would You Let a Humanoid Play Storytelling With Your Child? A Usability Study on LLM-Powered Narrative Human-Robot Interaction](https://edtechdev.github.io/aied/articles/icub-humanoid-storytelling-llm-hri-2025/) > **Synthesis:** Lombardi et al. (2025) present a framework for enhancing the attention and social capability of the iCub humanoid robot by integrating advanced perceptual abilities that recognize social cues, understand surroundings through generative models such as ChatGPT, and respond with contextually appropriate social behaviour. They implement an interaction task using a narrative (storytelling) protocol in which a human and the robot co-create a short imaginary story, exchanging cubes with creative images. Experiments quantify the usability and quality of experience perceived by participants interacting with the LLM-powered narrative human-robot interaction system. ## Key Findings - A key challenge in human-robot interaction is developing systems that can **perceive and interpret social cues** to enable natural, adaptive interactions. - The framework integrates **generative models (e.g., ChatGPT)** so the iCub humanoid can understand its surroundings and respond with contextually appropriate social behaviour. - The **storytelling task** has human and robot co-create an imaginary story by exchanging image-cubes, supporting collaborative narrative interaction. - The study **validates the protocol and framework** through usability and quality-of-experience measurement with participants. ## Study Design & Method This is a **usability/quality-of-experience study** of an LLM-powered narrative human-robot interaction system built on the iCub humanoid platform. The researchers integrated perceptual capabilities for social-cue recognition with generative-model understanding (ChatGPT) and implemented a storytelling interaction protocol where the human and robot jointly create a story by exchanging cubes with creative images. Participants interacted with the system, and the degree of usability and quality of experience was quantified to validate the framework and protocol. ## Implications for AI in Education The work demonstrates how **LLM-powered social robots** can support collaborative, engaging learning interactions such as storytelling, relevant to [[human-robot-interaction]], [[social-robots]], and [[educational-robotics]]. Integrating generative models enables robots to respond adaptively and contextually to children, supporting naturalistic educational interaction. This connects to [[llm|large language models]] in education and to child-focused learning activities such as storytelling, with implications for how embodied AI agents can participate in co-creation and narrative learning. ## Limitations The study focuses on usability and perceived quality of experience rather than measured learning outcomes; sample sizes and context are not specified in the abstract. The iCub platform is research hardware with limited classroom availability, and the generalizability of the LLM-integrated interaction framework to other robot platforms and age groups warrants further study. ## Connected Concepts - [[human-robot-interaction]] - [[social-robots]] - [[educational-robotics]] - [[llm]] - [[student-engagement]] ## Connected Articles - [[robobuddy-llm-social-robots-classroom-2025]] — RoboBuddy: LLM-Powered Social Robots for Storytelling - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Robot-LLM Integration and Embodiment in Creative Writing - [[social-robot-study-companions]] — Social Robots as Study Companions ## Citation Lombardi, M., Calabrese, C., Ghiglino, D., Foglino, C., De Tommaso, D., Da Lisca, G., Natale, L., & Wykowska, A. (2025). *Would you let a humanoid play storytelling with your child? A usability study on LLM-powered narrative human-robot interaction*. arXiv:2508.02505. --- ## [Generative AI and the Productivity Divide: Human-AI Complementarities in Education](https://edtechdev.github.io/aied/articles/idan-anand-genai-productivity-divide-2026/) > **Synthesis:** Idan & Anand (2026) conduct an RCT showing that GenAI access significantly increases task performance on average — but the gains are highly uneven, NOT predicted by GPA or prior knowledge, but by AI Interaction Competence (AIC): the ability to elicit, filter, and verify model outputs. High-AIC participants realized outsized gains while low-AIC saw limited or negative returns. A scaffolding intervention (conceptual maps) reduced outcome variance, showing that standardized workflows can mitigate the new "AI productivity divide." This randomized controlled experiment assigned participants — analogs of early-career knowledge workers — to self-study a technical domain using either traditional resources or LLM assistance. On average, GenAI significantly increased task performance, but the distribution was highly skewed: the top quartile of users captured most of the gains while the bottom quartile saw negligible or negative returns. Critically, performance was not predicted by GPA or prior domain knowledge, but by **AI Interaction Competence (AIC)** — the ability to elicit, filter, and verify model outputs. A conceptual-map scaffolding intervention reduced outcome variance, suggesting that organizations can mitigate AI-mediated inequality through structured workflows. - GenAI increased mean task performance, but gains concentrated among high-AIC users - GPA and prior knowledge did NOT predict GenAI-augmented performance — AIC did - Low-AIC participants saw limited or even negative marginal returns from GenAI - Conceptual map scaffolding intervention reduced outcome variance - Introduces the concept of a new "productivity divide" driven by AI interaction skills rather than domain expertise ## Connected Concepts - [[prompt-engineering]] - [[affective-tutoring]] - [[student-experience]] - [[administrator]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[help-seeking]] - [[bias-mitigation]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[structured-llm-feedback-programming]] — The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study ## Citation Idan, L., & Anand, B. (2026). [Generative AI and the Productivity Divide: Human-AI Complementarities in Education](https://arxiv.org/abs/2605.18143). --- ## [IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems](https://edtechdev.github.io/aied/articles/iks-instruct-dataset-indian-knowledge/) Presents a 24,795-example multilingual instruction dataset for teaching LLMs to deliver educational content grounded in Indian Knowledge Systems. Spans seven languages and bridges a gap in non-Western pedagogical content for instruction tuning. Demonstrates that domain-specific educational datasets improve LLM performance on culturally grounded knowledge tasks. ## Key Findings - **IKS-Instruct** contains **24,795 instruction-response pairs** for teaching language models to deliver educational content grounded in Indian Knowledge Systems (IKS), spanning **seven languages** (English, Hindi, Sanskrit, Tamil, Telugu, Kannada, Malayalam). - The dataset covers **41 pedagogical techniques** from Vedic oral and mathematical traditions and is aligned with the **Central Board of Secondary Education (CBSE) curriculum for classes 6 through 12**. - Pairs derive from **six source types**: classical text corpora (Bhagavad Gita, Thirukkural, Sangam literature, Vedic texts), curriculum-aligned pedagogical templates, Vedic mathematical sutra demonstrations, bilingual instruction pairs, technique-grounded multi-turn dialogues, and cross-tradition comparative analyses. - Quality is assessed through a **multi-judge evaluation framework** in which independent language models score responses on **12 dimensions**, including technique fidelity, pedagogical quality, factual accuracy, and IKS cultural depth. - Under a uniform five-judge external panel (median aggregation over 1,201 stratified items), the strongest IKS-Instruct fine-tune of a compact **7B model reaches a median judge score of 6.39**, within 0.15 of a strong general-purpose reference model (Nemotron-Nano at 6.54) at a fraction of its deployment cost. - The base model without IKS fine-tuning scores **near zero on IKS-specific dimensions**, and model quality does **not increase monotonically with data curation** — a result the authors report alongside data-quality gains. ## Dataset Design & Method IKS-Instruct targets a gap in instruction tuning: existing datasets such as Alpaca, FLAN, and Dolly are dominated by English-language general-knowledge tasks and lack coverage of specialized pedagogical domains. The six source types are engineered so that each pair carries structured metadata — **source provenance, technique classification, language, subject, class level, and quality scores** — released in JSONL format. This design lets the dataset support both instruction-following and culturally grounded pedagogy rather than generic fact recall. ## Implications for AI in Education The evaluation shows that a compact, domain-tuned model can approach the IKS-specific quality of a far larger general-purpose model at a fraction of deployment cost, which matters for resource-constrained educational deployments and for [[language-learning]] and [[culturally-relevant-pedagogy]] in multilingual classrooms. The non-monotonic relationship between curation and quality also warns against assuming "more data is always better," reinforcing the value of technique-fidelity-aware evaluation when adapting LLMs to non-Western knowledge traditions. ## Connected Concepts - [[language-learning]] - [[culturally-relevant-pedagogy]] - [[adaptive-learning]] - [[knowledge-tracing]] - [[pedagogical-agent]] - [[open-source]] - [[affective-computing]] - [[reinforcement-learning]] ## Connected Articles - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[llm-children-reading-story-generation]] — Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety - [[book-level-synthetic-textbook-organization]] — Beyond Rephrasing: Book-Level Organization Improves Synthetic Textbook Data for Mid-Training ## Citation Shwetha Singaravelu, Gayathri Muruganantham, Lakshmi Rajendran, Santhosh Sivasubramani (2026). [IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems](https://arxiv.org/abs/2607.23322). arXiv:2607.23322. cs.CL, cs.CY. --- ## [Informal Learning Emerges in Everyday Human-LLM Interaction](https://edtechdev.github.io/aied/articles/informal-learning-everyday-human-llm-interaction/) As LLMs take over task execution, a central worry is that everyday AI use becomes cognitive offloading that erodes people's own capability development. This study analyses 128,569 naturalistic human-LLM conversations, translating learning-science constructs into turn-level behavioural signatures to test whether informal learning actually emerges in routine use. The authors find that users do engage in learning-supporting behaviours — cognitive engagement, self-explanation, and elaboration — within ordinary [[llm]] interactions, tempering the pure offloading concern. The analysis contributes to debates on [[ai-literacy]] and the risks of [[over-reliance]], and reframes [[student-experience]] as a site where incidental learning can occur. It also informs [[self-regulated-learning]] and [[metacognition]] research by quantifying how conversational patterns either preserve or displace opportunities to think. ## Connected Concepts - [[llm]] - [[ai-literacy]] - [[over-reliance]] - [[student-experience]] - [[self-regulated-learning]] - [[metacognition]] ## Connected Articles - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning ## Citation Zixin Chen, Haotian Li, Ziang Xiao, Huamin Qu, et al. (2026). [Informal Learning Emerges in Everyday Human-LLM Interaction](https://arxiv.org/abs/2607.17643). arXiv:2607.17643. arXiv:2607.17643 [cs.HC]. --- ## [INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators](https://edtechdev.github.io/aied/articles/inside-llm-student-simulator-reasoning-2026/) > **Synthesis:** Niousha, Kang, & Norouzi (2026) introduce **INTERNAL STUDENT DIALOGUE (INSIDE)**, a student modeling framework that fine-tunes LLMs to both *act* like students and *think* like them. Two students may submit identical work for entirely different reasons, so INSIDE generates internal dialogue grounded in Bloom's Taxonomy across cognitive, affective, and action dimensions, fine-tuning on paired think-traces and actions. Evaluated against prompting baselines, INSIDE improves action fidelity (matching real students' code generation) and reasoning alignment (up to 57.9% across models). The work advances [[student-modeling]] and [[simulating-students]] research toward cognitively faithful student simulation. ## The Reasoning Gap in Student Simulation LLM-based simulators often reproduce observable actions but fail to capture the underlying reasoning behind them. In education, where student simulation is used to evaluate tutoring systems, this gap is pronounced: two students may submit identical submissions for entirely different reasons. ## The INSIDE Framework INSIDE fine-tunes LLMs not only to act like students but to think like them. It generates internal dialogue grounded in Bloom's Taxonomy across cognitive, affective, and action dimensions, and is trained on paired think traces and actions. ## Evaluation INSIDE is baselined against different prompting frameworks and evaluated on two axes: **fidelity of simulated actions** and **quality of generated internal dialogue**. It improves simulation fidelity in action fidelity (matching real students' code generation) and achieves the highest reasoning alignment across models up to 57.9%. Accepted at the Conference on Language Modeling (COLM) 2026. ## Connected Concepts - [[student-modeling]] - [[simulating-students]] - [[metacognition]] - [[intelligent-tutoring]] - [[personalized-learning]] - [[llm]] - [[learning-analytics]] - [[cognitive-diagnosis]] - [[higher-ed]] - [[scaffolding]] ## Connected Articles - [[history-aware-student-simulation]] - [[valid-student-simulation-llm-2026]] - [[simulating-students-diverse-cognitive-levels-2025]] - [[simulating-students-llm-review-2026]] - [[llm-student-simulation-misconception-faithfulness]] - [[agentschool-multi-agent-simulation-education-2026]] - [[llm-student-simulation-teacher-insights]] - [[llm-student-modeling-memory]] ## Citation Niousha, R., Kang, M., & Norouzi, N. (2026). [*INSIDE the student's mind: Jointly modeling latent reasoning and action in LLM student simulators*](https://arxiv.org/abs/2608.10492). arXiv:2608.10492. --- ## [A Framework for Institutional Change in the Age of AI](https://edtechdev.github.io/aied/articles/institutional-change-framework-ai/) > Perl-Nussbaum & Finkelstein (2026) adapt institutional-change models to generative AI as an **arrival technology** — one that entered classrooms before pedagogical evidence existed — yielding a six-dimension framework and design implications for leading change under uncertainty ([[faculty-development]], [[higher-ed]], [[educational-policy-ai]]). # Institutional Change Framework for AI ## Core Contribution Perl-Nussbaum & Finkelstein (2026) propose a **six-dimension framework** for adapting institutional change models in STEM higher education to the realities of generative AI. Unlike prior reforms based on **adoption technologies** (stable, evidence-based practices like Peer Instruction or PhET simulations), generative AI is an **arrival technology** — it entered classrooms before pedagogical evidence could form. The framework identifies where existing change models break down and derives actionable design implications for leading change under genuine uncertainty. Central insight: "Institutions cannot wait for best practices, but cannot responsibly scale unjustified innovations. Neither banning nor uncritical embrace is tenable." ## Six Dimensions of Reconsideration ### Tools Dimension **1. Evidence Base:** Prior models assumed reform begins with evidence-based tools; AI arrived without pedagogical evidence. Design implication: privilege **humble, local inquiries** — document what faculty are trying, share cautiously, avoid overclaiming generalizability. **2. Rate of Change:** Prior models assumed stable tools; AI evolves on timescales of months. Design implication: organize reform around **pedagogical approaches** rather than specific AI tools, which may be obsolete before evidence accumulates. **3. Scope:** Prior models targeted bounded interventions; AI's impact is broad and systemic. Design implication: adopt a **systemic view** — coordinate across courses and departments rather than responding piecemeal. ### People Dimensions **4. Faculty:** Prior models positioned faculty as adopters of proven practices; in the AI era, faculty must become **inquiry-driven experimenters** navigating genuine uncertainty. This connects to the [[faculty-development]] finding that faculty are "pragmatic realists" navigating competing priorities with scarce resources. **5. Change Agents:** Prior models cast change agents as disseminators of best practices; when best practices don't exist, they must become **facilitators of collective inquiry**. This reframes Centers for Teaching and Learning from trainers to community organizers — a transformation anticipated by [[universities-ai-era-rethinking]]. **6. Students:** Prior models treated students as recipients; in the AI era, students are often **ahead of faculty** in AI use and must be engaged as **partners in reform**. This aligns with broader calls in [[student-experience]] and [[ai-literacy]] for student agency in AI-era education. ## Design Implications 1. **Privilege humble and local inquiries** — document and share local experiments rather than scaling unproven practices 2. **Organize around pedagogical approaches** — anchor reform in durable pedagogical principles, not ephemeral tools 3. **Reposition change agents as inquiry facilitators** — build faculty learning communities around shared questions 4. **Engage students as partners** — leverage their AI experience in co-creating institutional responses 5. **Build capacity for continuous adaptation** — design for ongoing change, not one-time adoption ## Application: Physics Department Workshop Series The framework was piloted through a faculty workshop series in a university physics department. Rather than prescribing AI tools, workshops facilitated collective inquiry around pedagogical goals, engaged participants as co-investigators, and organized around teaching approaches rather than specific technologies. This connects to the departmental-level work described in [[stem-education]] and the workshop-based models in [[faculty-development]]. ## Connections to Wiki - Extends [[universities-ai-era-rethinking]] by providing a concrete framework for institutional adaptation, not just rethinking - Complements [[faculty-development]] by adding the change-agent and student-partner dimensions missing from playbook-centered approaches - Connects to [[principled-ai-education]] — the framework operationalizes Finkelstein's goals-models-technologies lens at the institutional level - Relates to [[higher-ed]] and [[teacher-role]] — redefining instructor and institutional roles in AI-era education - Aligns with [[ai-literacy]] calls for faculty AI literacy as a prerequisite for meaningful institutional change - Contrasts with [[scaffolding]] and [[pedagogy-ai-mistakes]] — while those address classroom-level AI integration, this framework addresses the institutional conditions needed for such integration to succeed - Shares the systemic perspective of [[ai-education-global-capacity]] — institutional capacity is a bottleneck for AI in education globally ## Open Questions - How does this framework apply outside STEM — in humanities, social sciences, professional programs? - What metrics track institutional adaptation progress under genuine uncertainty? - How do different institutional types (research universities, community colleges, liberal arts) shape framework application? - Can the framework be operationalized into assessment tools for institutional readiness? ## Connected Concepts - [[faculty-development]] - [[higher-ed]] - [[educational-policy-ai]] - [[student-experience]] - [[ai-literacy]] - [[stem-education]] - [[teacher-role]] - [[scaffolding]] ## Connected Articles - [[universities-ai-era-rethinking]] - [[principled-ai-education]] - [[pedagogy-ai-mistakes]] - [[ai-education-global-capacity]] ## Citation Perl-Nussbaum, D., & Finkelstein, N. D. (2026). [*A Framework for Institutional Change in the Age of AI*](https://arxiv.org/abs/2605.12757). arXiv:2605.12757. --- ## [Policy Fragmentation or Institutional Alignment? Institutional Governance of AI in Universities and Business Schools](https://edtechdev.github.io/aied/articles/institutional-governance-ai-universities/) > **Synthesis:** This study analyzes AI policies across higher education institutions in 34 U.S. states, using NLP to uncover a clear divergence: university-level policies emphasize data security and risk mitigation, while school-level policies (when they exist) focus on pedagogical applications and tool usage. Relatively few business schools maintain distinct AI policies, creating misalignment with discipline-specific learning objectives. The findings highlight the challenges of institutional AI governance and the tension between centralized [[regulation|compliance-oriented policies]] and the need for discipline-specific pedagogical guidance. ## Research Approach The study applied natural language processing to analyze AI policies from institutions across 34 states: - **University-wide policies:** Broad governance frameworks addressing AI use campus-wide - **School/department-level policies:** Unit-specific guidelines, primarily in business schools - **Comparative analysis:** NLP identified thematic differences between governance levels ## Key Findings **Policy Divergence:** | Policy Level | Primary Focus | Gaps | |-------------|---------------|------| | University-wide | Data security, risk mitigation, legal compliance | Limited pedagogical guidance | | School-level (when present) | Pedagogical applications, tool usage guidelines | Rarely exist; inconsistent when present | | Business school-specific | Discipline-integrated AI use | Few maintain distinct policies | - **Risk vs. pedagogy tension:** University policies prioritize security and compliance; pedagogical guidance is scarce at the top level - **Missing middle:** Most institutions lack school/department-level policies, leaving faculty without discipline-specific AI guidance - **Accreditation implications:** The gap between university risk-management framing and discipline-specific learning objectives creates challenges for [[higher-ed|accreditation bodies]] - **Faculty and student impact:** Without clear, aligned policies, both faculty and students navigate AI use inconsistently ## Implications for AI Governance in Education The study reveals a fundamental structural challenge in [[ai-education|AI governance]]: risk-averse centralized policies that fail to address pedagogical needs at the discipline level. The authors recommend that guidelines be aligned with broader institutional policies while explicitly addressing discipline-specific learning objectives and evolving workforce demands. This aligns with emerging [[ai-literacy]] frameworks that emphasize both technical and pedagogical dimensions of AI readiness. ## Connected Concepts - [[regulation]] - [[higher-ed]] - [[ai-education]] - [[ai-literacy]] ## Connected Articles - [[generative-ai-reduced-study-time-math]] — Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[will-skill-not-tool-chinese-university-students-acceptance-of-generative-ai-for-]] — Will, Skill, Not Tool: Chinese university students' acceptance of generative AI for academic writing in informal English medium instruction settings - [[ai-higher-ed-workforce-survey]] — The Impact of AI on Work in Higher Education - [[ai-higher-ed-bridge-gap]] — Higher Education Must Bridge the AI Gap - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well ## Citation Manikonda, L., & Outlaw, D. (2026). [*Policy Fragmentation or Institutional Alignment? Institutional Governance of AI in Universities and Business Schools*](https://arxiv.org/abs/2608.03584v1). arXiv:2608.03584v1. --- ## [Instructional Agents: Reducing Teaching Faculty Workload through Multi-Agent Instructional Design](https://edtechdev.github.io/aied/articles/instructional-agents-multi-agent-course-gen/) > **Synthesis:** Instructional Agents is a multi-agent LLM framework that automates end-to-end course material generation by simulating role-based collaboration among Teaching Faculty, Instructional Designer, Teaching Assistant, Course Coordinator, and Program Chair agents, all structured around the ADDIE instructional design framework. Evaluated across 5 university courses, the system supports four modes (Autonomous, Catalog-Guided, Feedback-Guided, Full Co-Pilot) balancing automation and human oversight. ## System Design The framework maps the first three phases of the **ADDIE** framework (Analyze, Design, Develop) onto a multi-agent workflow: - **Analyze**: Teaching Faculty + Course Coordinator produce an Instructional Foundation Report (objectives, learner profiles, constraints) - **Design**: Teaching Faculty + Instructional Designer structure syllabi, slide outlines, and assessment plans - **Develop**: Teaching Assistant generates concrete materials (slides, scripts, assessments); Program Chair reviews from a program-level perspective; Test Student agent supplies simulated learner feedback The Teaching Faculty agent serves as the primary authority throughout, maintaining continuous oversight. ## Four Interaction Modes 1. **Autonomous** — fully automated generation with no human input 2. **Catalog-Guided** — human provides a course catalog description as seed input 3. **Feedback-Guided** — human reviews and provides iterative feedback between phases 4. **Full Co-Pilot** — tight human-AI collaboration throughout all phases ## Evaluation Evaluated across 5 university-level courses using both human and automated reviewers. Key findings: - Produces high-quality materials that are reviewed and refined by teaching faculty prior to classroom use - Significantly reduces time required to prepare classroom-ready content - Multi-agent collaboration preserves pedagogical coherence better than single-model approaches - Trade-offs exist between automation level and output quality — Feedback-Guided and Full Co-Pilot modes produce higher-quality outputs at the cost of more human involvement ## Significance - **Scalable instructional design** — supports institutions with limited instructional design capacity - **Democratizing access** — reduces barriers to high-quality course material creation, especially in underserved settings - **Role-based coherence** — simulates real-world instructional collaboration rather than treating generation as an isolated task - **Source code available** at the project website ## Connected Concepts - [[higher-ed]] - [[llm]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation Yao, H., Xu, W., Turnau, J., Kellam, N., & Wei, H. (2026). [*Instructional Agents: Reducing Teaching Faculty Workload through Multi-Agent Instructional Design*](https://aclanthology.org/2026.eacl-long.191). In *Proceedings of EACL 2026*. --- ## [Role of Instructional Guidance in Generative AI-Assisted Learning](https://edtechdev.github.io/aied/articles/instructional-guidance-genai-learning/) Investigates how instructional guidance shapes student-AI interaction in [[higher-ed|construction engineering education]]. Introduces a **five-step prompting framework** grounded in Generative Learning Theory (GLT) to guide learner interaction during review activities. Three conditions tested in a controlled experiment: slide-based learning, unprompted AI-supported learning, and prompted AI-supported learning. **Key findings:** Performance differences concentrated on tasks requiring explanation and reasoning (higher-order cognitive outcomes). The prompted condition achieved significantly higher open-ended scores (≈2–3 points on 18-point scale, p<0.01). No significant differences in multiple-choice recall across conditions. Unprompted AI use did not outperform traditional slide-based review. Demonstrates that the effectiveness of AI-supported learning depends critically on how [[scaffolding|interaction is structured]] — a simple prompting framework grounded in learning science can significantly improve [[active-learning|higher-order cognitive outcomes]]. Provides a basis for integrating [[personalized-learning|learning science principles]] into GenAI systems for education. ## Connected Concepts - [[higher-ed]] - [[scaffolding]] - [[active-learning]] - [[personalized-learning]] ## Connected Articles - [[flowcode-ai-creative-coding]] — Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education - [[llm-tts-dialogue-lesson-generation]] — A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large Language Models: An Exploratory Study of Educational Potential - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[q-learning-lab-rl-teaching]] — Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis - [[courseblueprint-adaptive-video-generation]] — CourseBlueprint: A Structured Pipeline for Adaptive Pedagogical Video Generation Grounded in Course Corpora - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Hou, X., Xiao, B., Liu, H., & Mueller, S. (2026). The Role of Instructional Guidance in Generative AI-Assisted Learning: Empirical Evidence from Construction Engineering Education. [arXiv:2606.05509](https://arxiv.org/abs/2606.05509). --- ## [Interactive learning dashboards: rethinking learning visualisations as engagement tools](https://edtechdev.github.io/aied/articles/interactive-learning-dashboards-engagement/) > **Synthesis:** Graf et al. (2026) transformed a conventional Learning Analytics Dashboard (LAD) into an interactive ILAD by adding an LLM-powered pedagogical agent and a Judgement of Learning (JoL) self-assessment feature. In a 5-week case study with 30 CS students across three conditions (no agent, "telling" agent, "eliciting" agent), the elicit condition produced more reflection and more accurate mastery calibration. The study bridges [[learning-analytics]] dashboard design with [[intelligent-tutoring]] principles and [[metacognition]] research, showing that how learners interact with their data matters more than simply seeing it. ## Study Design - **30 CS students** in a university programming course, paid to regularly use the ILAD over **5 weeks** - **3 conditions** randomized: no agent, "tell" agent (provides info about learner data), "elicit" agent (asks questions about learner data) - ILAD extended a conventional LAD with two features: 1. **LLM-powered pedagogical agent** with access to learning analytics and course context 2. **Interactive Judgement of Learning (JoL)** — self-assessment required before viewing system metrics ## Key Findings - Students in the **elicit condition** engaged in more reflection - Elicit-condition students more **accurately judged their own mastery** (better JoL calibration) - The "tell" agent (providing information) did not produce the same benefits — highlighting that pedagogical strategy matters, not just AI presence - The study demonstrates how interactivity can shift LADs from static visualizations to engagement tools that promote [[metacognition|metacognitive]] processes ## Design Implications The paper challenges the dominant LAD paradigm of "show data → hope for insight." Drawing on [[intelligent-tutoring]] research (Chi's ICAP framework), the authors argue that: 1. **Interactivity ≠ navigation** — clicking filters is not cognitive engagement; two-way dialogue is 2. **Elicitation beats telling** — asking learners to self-assess before revealing metrics drives calibration better than showing data upfront 3. **LLMs enable scalable interactivity** — the pedagogical agent can ask personalized questions at scale, bringing ITS-style dialogue to dashboard contexts These findings connect to broader [[self-regulated-learning]] theory and the [[metacognition]] literature on judgement calibration. ## Connected Concepts - [[higher-ed]] - [[learning-analytics]] - [[llm]] - [[metacognition]] - [[pedagogical-agent]] - [[self-regulated-learning]] - visualization ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams ## Citation Graf, L., Bassner, P., Anzinger, M., Dietrich, F., Krusche, S., & Poquet, O. (2026). [Interactive learning dashboards: rethinking learning visualisations as engagement tools](https://doi.org/10.1007/s10639-026-14082-1). *Education and Information Technologies*. --- ## [Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation](https://edtechdev.github.io/aied/articles/international-students-conversational-ai-adaptation/) > Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation **Nourian et al. (2026)** — Multiple institutions. arXiv cs.HC. Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation ## Summary This mixed-methods study (survey n=60, interviews n=14) investigates how international students in the U.S. adopt and perceive [[conversational-ai-tutors-framework]] tools like ChatGPT and Google Gemini for cross-cultural adaptation. The current support ecosystem — university systems and informal social networks — remains fragmented, echoing the integration gaps identified in [[ai-higher-ed-bridge-gap]]. **Key findings:** International students currently treat AI as a **"first-aid tool"** for immediate challenges: language barriers, navigating bureaucracy, and cultural adjustment. However, there is a **clear desire to transform AI** from short-term help into a **long-term support companion** — a finding that connects to [[ai-learning-companions-framework]] and the broader [[student-experience]] literature. The study distinguishes between domains where AI can provide sustained support (language practice, administrative navigation, cultural information) and where it falls short (emotional connection, deep cultural understanding, replacing human community). This maps onto the [[human-in-the-loop-ai]] paradigm, suggesting AI should augment rather than replace human support systems. The findings have implications for designing [[equity-in-ai-education]] interventions for international student populations and align with [[culturally-relevant-pedagogy]] principles. The study contributes to [[higher-ed]] understanding of how diverse student populations interact with [[generative-ai]] tools beyond academic contexts. ## Connected Concepts - [[student-experience]] - [[human-in-the-loop-ai]] - [[equity-in-ai-education]] - [[culturally-relevant-pedagogy]] - [[higher-ed]] - [[generative-ai]] ## Connected Articles - [[conversational-ai-tutors-framework]] - [[ai-higher-ed-bridge-gap]] - [[ai-learning-companions-framework]] ## Citation Nourian, L., Callis, A., Patterson, S., Miao, J., Heard, J., & Tigwell, G. W. (2026). [Understanding how international students in the U.S. are using conversational AI to support cross-cultural adaptation](https://arxiv.org/abs/2605.15127). arXiv:2605.15127. --- ## [ISD Agent Benchmark](https://edtechdev.github.io/aied/articles/jeon-isd-agent-bench-2026/) > **ISD-Agent-Bench** is a comprehensive benchmark for evaluating LLM-based instructional design agents, comprising **25,795 scenarios** generated via a Context Matrix framework that combines 51 contextual variables with 33 ISD sub-steps from the ADDIE model. It employs a multi-judge evaluation protocol to mitigate LLM-as-judge bias. ## Key Findings 1. **Hybrid agents outperform both pure theory and pure technique.** The best-performing approach integrates classical ISD frameworks (ADDIE, Dick & Carey, Rapid Prototyping ISD) with modern ReAct-style reasoning. The performance hierarchy is: Hybrid (theory + technique) > pure theory-based > technique-only. This demonstrates that grounding LLM agents in established educational design theory provides a structural advantage that raw prompting cannot replicate. 2. **The Context Matrix framework enables systematic scenario generation.** Rather than ad-hoc benchmark construction, ISD-Agent-Bench uses a Context Matrix that combinatorially varies **51 contextual variables** across 5 categories with **33 ISD sub-steps** derived from ADDIE, producing 25,795 total scenarios. This systematic coverage ensures agents are tested across diverse instructional design situations rather than narrow task types. 3. **Theoretical quality strongly correlates with benchmark performance.** Agents grounded in classical ISD theories showed significant advantages in problem-centered design and objective-assessment alignment — two areas where theory-free agents consistently struggled. This provides empirical validation for the role of [[instructional-design]] theory in guiding AI behavior. 4. **Multi-judge protocol addresses a critical evaluation challenge.** Recognizing that single-LLM evaluation introduces systematic bias, the benchmark employs diverse LLMs from different providers as judges, achieving high inter-judge reliability. This protocol-level innovation is as important as the benchmark itself for the validity of [[agentic-ai]] evaluation. ## Implications ISD-Agent-Bench fills a significant gap in the evaluation landscape. While benchmark-driven progress has propelled general LLM capabilities, instructional design agents have lacked standardized, theory-grounded evaluation. This benchmark enables rigorous comparison of [[agentic-ai-education-scoping-review]] approaches and provides a foundation for future research on [[multi-agent-instructional-design]] systems. The finding that classical ISD theory improves agent performance has practical implications for system builders: rather than treating instructional design as a generic prompting task, agents benefit from structured theoretical grounding. This resonates with broader work on [[educational-llm-alignment]], which argues that pedagogical goals require more than general capability — they require specific structural priors. The 51-variable Context Matrix is itself a contribution, formalizing what makes instructional design contexts vary (learner characteristics, content domain, delivery mode, constraints, outcomes). This taxonomy could inform future work on [[agentic-workflows-education]] and context-aware llm-evaluation. For the [[ai-ed-evaluation]] community, the multi-judge protocol represents a methodological advance that may generalize beyond instructional design to other educational AI evaluation tasks where LLM-as-judge bias is a concern. ## Connected Concepts - [[agentic-ai]] - [[ai-ed-evaluation]] - [[agentic-ai]] - [[ai-education]] - [[llm]] - [[rag]] ## Connected Articles - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-workflows-education]] — Agentic Workflows in Education - [[educational-llm-alignment]] — Educational LLM Alignment - [[multi-agent-instructional-design]] — Multi-Agent Systems for Instructional Design - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[ai-in-the-wild-college]] — AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI ## Citation Jeon, Y., Kim, S., Son, H., Lee, S., Jeong, Y., & Lee, U. (2026). [*ISD-Agent-Bench: A Comprehensive Benchmark for Evaluating LLM-based Instructional Design Agents*](https://arxiv.org/abs/2602.10620). arXiv:2602.10620. --- ## [MathBuddy: Affective Math Tutoring](https://edtechdev.github.io/aied/articles/kar-mathbuddy-affective-math-tutoring-2025/) > **MathBuddy: Affective Math Tutoring** — EMNLP 2025 Demo. An emotionally aware LLM-powered mathematics tutor that dynamically models student emotions from both conversational text and facial expressions, aggregating multimodal affective signals to shape pedagogically appropriate LLM responses. Maps detected affective states to relevant pedagogical strategies, achieving a +23-point win rate advantage and +3-point improvement on the DAMR (Dimensions of Affective Math Response) score at the overall level. Evaluated across eight pedagogical dimensions in user studies, MathBuddy demonstrates that affective awareness substantially improves the quality of AI math tutoring. ## Key Findings MathBuddy addresses a critical gap in [[ai-tutoring]]: most AI math tutors respond to the *content* of student answers but ignore the *emotional state* of the learner. This is problematic because mathematics learning is deeply affective — frustration, confusion, anxiety, and confidence all shape how students engage with mathematical problems and whether they persist through difficulty. **Multimodal emotion modeling.** MathBuddy captures student affect through two complementary channels: (1) conversational text analysis, which detects emotional signals in the linguistic content of student responses (e.g., expressions of frustration, confusion, or confidence), and (2) facial expression analysis, which captures moment-to-moment emotional reactions through computer vision. These modalities are aggregated to produce a unified affective state assessment that is more robust than either modality alone — a design principle consistent with the broader [[multimodal-ai-tutoring]] literature. **Affect-to-pedagogy mapping.** The system maps detected affective states to specific [[instructional-design]] pedagogical strategies. For example, detected frustration may trigger scaffolded hint sequences or encouraging messages; detected confidence may prompt the tutor to increase problem difficulty or reduce scaffolding; detected confusion may trigger a shift from procedural guidance to conceptual explanation. This dynamic mapping connects MathBuddy to the [[affective-tutoring]] paradigm and the [[student-modeling]] tradition in [[intelligent-tutoring]]. **LLM-powered response generation.** At its core, MathBuddy uses an LLM prompted with the aggregated affective state, the mathematical context, and the tutoring history to generate emotionally appropriate pedagogical responses. The LLM is not merely an answer engine but an affective-pedagogical reasoning system that must balance mathematical correctness with emotional attunement. **Rigorous evaluation.** The system was evaluated along eight pedagogical dimensions — including scaffolding quality, emotional attunement, mathematical accuracy, and learner engagement — through controlled user studies. The +23-point win rate and +3-point DAMR improvement establish clear evidence that affective awareness adds measurable value beyond content-only tutoring approaches. This connects to the engagement-metrics literature and the growing recognition that [[engagement-intensity-learner-modeling]] is essential for effective AI tutoring. ## Implications For the [[intelligent-tutoring]] field, MathBuddy strengthens the case that effective tutoring is fundamentally relational, not merely informational. The substantial performance gains from adding affective awareness suggest that [[affective-computing]] should be considered a core component of next-generation tutoring systems, not an optional enhancement. This aligns with findings from the [[multimodal-affective-its-presentation]] literature showing that affect-aware tutors outperform their affect-blind counterparts. For [[k-12]] mathematics education specifically, MathBuddy's multimodal approach to emotion detection offers a pathway toward AI tutors that can respond to the full range of student experiences — including math anxiety, which is a well-documented barrier to [[math-education]] achievement. The ability to detect and respond to negative affective states in real time could make AI tutoring particularly valuable for students who might otherwise disengage. The dual-modality aggregation strategy (text + facial expression) also has implications for [[multimodal-learning-genai]] system design more broadly. MathBuddy demonstrates that combining complementary signal types produces more reliable affective assessments than either modality alone, a finding that generalizes beyond math tutoring to any educational context where learner emotions matter. Methodologically, the DAMR evaluation framework and the eight-dimension pedagogical assessment provide a template for rigorous evaluation of affect-aware tutoring systems — addressing concerns about [[ai-tutor-behavioral-evaluation]] and the need for multi-dimensional quality metrics beyond simple accuracy measures. Finally, the open-source release of the codebase (github.com/ITU-NLP/MathBuddy) contributes to the [[open-source]] educational AI ecosystem and enables further research into affect-aware [[pedagogical-llm-training]]. ## Connected Concepts - [[math-education]] - [[affective-computing]] - [[affective-tutoring]] - [[k-12]] - [[open-source]] - [[pedagogical-llm-training]] - [[intelligent-tutoring]] - [[ai-tutoring]] - [[student-modeling]] - [[rag]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] - [[engagement-intensity-learner-modeling]] - [[multimodal-affective-its-presentation]] - [[multimodal-ai-tutoring]] - [[multimodal-learning-genai]] - [[zhang-tutormoments-2026]] - [[lak2026-hint-button-unproductive-use]] ## Citation Kar, D., Böss, L., Braca, D., Dennerlein, S. M., Hubig, N. C., Wintersberger, P., et al. (2025). [MathBuddy: Affective Math Tutoring](https://arxiv.org/abs/2508.19993v2). arXiv:2508.19993. --- ## [LLM Fallacy Misattribution in Education](https://edtechdev.github.io/aied/articles/kim-llm-fallacy-misattribution-2026/) > **The LLM Fallacy** is a cognitive attribution error in which individuals misinterpret LLM-assisted outputs as evidence of their own independent competence — producing a systematic gap (∆C) between perceived and actual capability. This divergence persists regardless of whether the LLM output is correct or erroneous, and it is driven by three system-level properties (opacity, fluency, and interactional immediacy) mediated by attribution ambiguity and cognitive outsourcing. **Kim, Yu & Yi (ddai Inc., 2026)** · arXiv: 2604.14807v2 ## Key Findings 1. **The LLM Fallacy is distinct from related cognitive phenomena.** It is not hallucination (a system output failure), automation bias (over-reliance on system outputs), or cognitive offloading (delegating mental effort). Each of those focuses on the output or the delegation decision; the LLM Fallacy concerns how outputs are *integrated into self-perception*. When a student uses an LLM to write an essay and then believes they are a strong writer — regardless of the essay's actual quality or the extent of AI contribution — that is the LLM Fallacy in action. This distinction matters because interventions that address [[cognitive-offloading-llm-synthesis-writing]] (teaching students when to delegate) do not necessarily address the misattribution of competence that follows. 2. **Three system-level properties create the conditions for misattribution.** *Opacity* hides the retrieval, pattern-matching, and synthesis mechanisms so the division of labor between human and machine is invisible. *Fluency* acts as a metacognitive cue — grammatically coherent, well-structured outputs are processed as signals of understanding, even when the user contributed only a prompt. *Interactional Immediacy* (sub-second response cycles) biases cognition toward fast, intuitive judgments that skip the self-assessment step. Together, these properties make it difficult for users to calibrate what they *can do independently* against what the *system did on their behalf* — the same mechanism explored in [[efficiency-gain-illusion-ai-overreliance]]. 3. **The typology spans six domains, all relevant to education.** The authors identify *Computational* (producing code without understanding architecture), *Linguistic* (fluent text in languages the user doesn't command), *Analytical* (structured explanations mistaken for personal reasoning skill), *Creative* (narratives misattributed as personal creativity), *Epistemic* (summaries leading users to equate access to information with conceptual mastery — the illusion of explanatory depth), and *Professional Signaling* (resumes, interviews, and portfolios that reflect AI-output production skill rather than independent expertise). Each of these has direct analogues in educational settings, from [[over-reliance]] effects in programming courses to the erosion of [[assessment-validity]] when submitted work reflects LLM scaffolding rather than learning. 4. **The capability divergence (∆C) is independent of output correctness.** This is perhaps the most troubling finding for education. Even when LLM output is accurate and well-reasoned, the user may still misattribute that quality to themselves — inflating self-perceived competence without corresponding growth in independent ability. Conversely, even when LLM output is erroneous, the user may still incorporate the *process* of using the tool into their self-concept as a competent user. The divergence is about attribution, not accuracy, which means that simply improving LLM reliability does not solve the problem. The [[cognitive-offloading-speedup-illusion]] literature documents a parallel phenomenon: users feel faster and more capable even when objective task performance does not improve. 5. **The framework identifies necessary conditions and institutional implications.** The LLM Fallacy arises when (a) a task involves LLM-mediated output generation requiring domain expertise, (b) the interaction is sufficiently seamless that the human/system boundary is not salient, and (c) the output exhibits fluency associated with skilled human performance. In education, these conditions are almost definitionally met by AI-assisted writing, coding, and problem-solving. The result is that AI can improve short-term performance (assignment grades) while weakening the relationship between performance and competence — a direct threat to the validity of [[authentic-assessment]] and the signaling function of educational credentials. ## Implications for AI in Education The LLM Fallacy framework challenges the dominant narrative that AI integration in education is primarily about access and efficiency. Even when AI tools improve assignment completion rates and grades, they may simultaneously erode students' ability to accurately self-assess their own competence. This is not a call to ban LLMs — it is a call to build [[ai-literacy]] curricula that explicitly address *attribution calibration*: teaching students to distinguish what the tool contributed from what they contributed, and to use that distinction to guide further learning. The framework also has consequences for assessment design. If the LLM Fallacy systematically inflates perceived competence, then assessment formats that cannot disentangle AI-mediated performance from independent capability — most current essay assignments, take-home coding projects, and open-book exams — are measuring something closer to *prompting skill plus LLM capability* than to student learning. This aligns with concerns raised in [[constructing-epistemic-ai-literacy-student-ai-co-programming]] about the blurred line between co-production and independent demonstration of knowledge. For AI literacy efforts, the LLM Fallacy suggests a new dimension beyond the typical "how to prompt" and "fact-check outputs" curriculum. Students need [[agentic-literacy-debt]] skills — the ability to track and account for the division of cognitive labor between themselves and the AI — and metacognitive practices that maintain accurate self-assessment even in the presence of fluent, confident AI outputs. Without this, we risk graduating students whose [[ai-partner-science-epistemic-vigilance]] is underdeveloped — who trust themselves too much and the AI too little in exactly the wrong proportions. The long-term institutional risk is credential inflation: if degrees and certificates increasingly signal AI-output-production ability rather than independently grounded competence, the signaling value of educational credentials erodes. This is not a theoretical concern — it is the direct implication of the LLM Fallacy playing out at scale across cohorts of students using AI tools without attribution-aware pedagogy. ## Connected Concepts - [[over-reliance]] - [[ai-literacy]] - [[assessment-validity]] - [[cognitive-load-theory]] - [[cognitive-offloading-llm-synthesis-writing]] - [[llm]] ## Connected Articles - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[ai-partner-science-epistemic-vigilance]] — AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation - [[authentic-assessment]] — Authentic Assessment - [[cognitive-offloading-speedup-illusion]] — Cognitive offloading and the speedup illusion in human-AI interaction - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[efficiency-gain-illusion-ai-overreliance]] — The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation Kim, H., Yu, H., & Yi, H. (2026). [*The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows*](https://arxiv.org/abs/2604.14807). arXiv:2604.14807. --- ## [Knowledge-Based Design Requirements for Generative Social Robots in Higher Education](https://edtechdev.github.io/aied/articles/knowledge-based-design-generative-social-robots-2026/) > **Synthesis:** Vonschallen, Oberle, Schmiedel, and Eyssel (2026) adopt a knowledge-based design perspective to investigate what information tutoring-oriented generative social robots (GSRs) require to function responsibly and effectively in higher education. Recognizing that GSRs powered by large language models enable adaptive, conversational tutoring but introduce risks such as misinformation, overreliance, and privacy violations, they conducted twelve semi-structured interviews with university students and lecturers, identifying twelve design requirements across three knowledge types: self-knowledge, user-knowledge, and (a third type concerning the domain/context). ## Key Findings - **Generative social robots (GSRs) powered by LLMs enable adaptive, conversational tutoring** but introduce risks such as **misinformation, overreliance, and privacy violations**. - Existing frameworks for educational technologies and responsible AI define desired behaviors but **rarely specify the knowledge prerequisites** that enable generative agents to express those behaviors reliably. - Based on **twelve semi-structured interviews** with university students and lecturers, the study identified **twelve design requirements across three knowledge types** (self-knowledge, user-knowledge, and domain/context knowledge). - The findings offer a **knowledge-based design perspective** for building responsible, effective tutoring GSRs in higher education. ## Study Design & Method This is a **qualitative interview study**. The researchers conducted twelve semi-structured interviews with university students and lecturers to identify the knowledge prerequisites that tutoring-oriented generative social robots need to function responsibly and effectively in higher education. Using a knowledge-based design perspective, they analyzed the interviews to derive twelve design requirements organized across three knowledge types (self-knowledge, user-knowledge, and domain/context knowledge), addressing the gap between responsible-AI behavior frameworks and the informational requirements generative agents need. ## Implications for AI in Education The study provides a design foundation for building [[social-robots|generative social robots]] as tutors in [[higher-ed]], addressing the risks of misinformation, overreliance, and privacy violations. It connects to [[generative-ai]], [[llm|large language models]], [[ai-tutoring|tutoring]], and [[ethics|responsible AI]] design. For designers and educators, it specifies the knowledge a tutoring robot must hold (about itself, the user, and the domain) to behave responsibly and effectively, informing the development of trustworthy AI tutoring agents. ## Limitations The findings derive from twelve interviews in a specific higher-education context, so they may not generalize across disciplines, institutions, or learner populations. The design requirements are identified but not yet fully validated through implementation and evaluation. The focus is on the knowledge prerequisites of tutoring robots rather than on broader social-robot design or measured learning outcomes. ## Connected Concepts - [[social-robots]] - [[generative-ai]] - [[llm]] - [[higher-ed]] - [[ai-tutoring]] - [[ethics]] ## Connected Articles - [[teachy-mini-generative-social-robot-higher-ed-2026]] — Teachy Mini: A Knowledge-Based Generative Social Robot - [[task-context-trust-educational-hri-2026]] — Task Context and Trust in Educational HRI - [[human-autonomy-agency-hri-review-2025]] — Human Autonomy and Agency in HRI ## Citation Vonschallen, S., Oberle, D., Schmiedel, T., & Eyssel, F. (2026). *Knowledge-based design requirements for generative social robots in higher education*. arXiv:2602.12873. --- ## [Knowledge Distillation for Automated AI Tutor Evaluation](https://edtechdev.github.io/aied/articles/knowledge-distillation-ai-tutor-evaluation/) Addresses the lag between LLM integration into K-12/higher education and reliable methods for evaluating pedagogical quality. The authors introduce a knowledge-distillation approach to automate AI-tutor evaluation, distilling expert judgments of pedagogical quality into a scalable evaluator. Directly advances [[intelligent-tutoring]] evaluation and [[automated-grading]] of tutor behavior across [[k-12]] and [[higher-ed]], building on [[llm]]-based assessment. It complements [[ai-tutor-behavioral-evaluation]] and the [[ai-tutor-effectiveness-review]], offering a practical route to scalable, expert-aligned tutor quality measurement. ## Key Findings - Introduces **FATE (FLC AI Tutor Evaluator)**, a specialized **8B-parameter language model** designed to evaluate AI tutors, aligned with the four core evaluation tracks of the **BEA 2025 Shared Task**: Mistake Identification, Mistake Location, Guidance, and Actionability. - Because pedagogical evaluation is a specialized task with limited labeled data, the authors use **knowledge distillation from a frontier LLM** to generate additional supervision, yielding **absolute performance gains up to 22.63 percentage points**. - FATE is demonstrated as an automated evaluator by benchmarking instructional responses from popular commercial models: **Gemini 2.5 Flash performed best on average (82.88%)**, followed by ChatGPT 5.5 Instant (80.75%), DeepSeek V4 Flash (80.13%), and Claude Sonnet 4.6 (74.00%). - The work responds to the rapid integration of LLMs into K-12 and higher education, which has outpaced the development of reliable methods for evaluating their pedagogical quality. ## Method: Distillation of Pedagogical Judgment The key idea is that pedagogical evaluation — judging whether a tutor correctly identifies a mistake, locates it, guides the learner, and offers actionable next steps — is itself a specialized NLP task with scarce expert-labeled data. FATE closes that gap by **distilling supervision from a frontier LLM**, treating the stronger model's judgments as soft targets for the smaller 8B evaluator. The resulting model can then score tutor responses against the four-track rubric at scale, making continuous, expert-aligned quality measurement of AI tutors practical for real deployments. ## Implications for AI in Education Automated tutor evaluation of this kind is a prerequisite for accountability in AI tutoring: without reliable measures of pedagogical ability, institutions cannot compare vendors, monitor quality over time, or certify that tutors teach rather than merely answer. The benchmark results also illustrate meaningful quality differences among commercial models on pedagogical dimensions, informing procurement and design choices for [[ai-tutoring]] systems. ## Connected Concepts - [[intelligent-tutoring]] - [[automated-grading]] - [[k-12]] - [[higher-ed]] - [[llm]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] - [[ai-tutor-effectiveness-review]] ## Citation Tahmid Al Hannan, Diego Garcia, Alex Njoroge, Suha Al Juboori, Tarek Sakakini (2026). [Knowledge Distillation for Automated AI Tutor Evaluation](https://arxiv.org/abs/2607.10647). arXiv:2607.10647. arXiv preprint. --- ## [Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs](https://edtechdev.github.io/aied/articles/knowledge-gap-detection-ai-tas/) This paper introduces a pipeline that maps student questions directed at a conversational AI teaching assistant to curriculum topics using a few-shot text classifier, grounded in a GPT-4-extracted prerequisite knowledge graph. Evaluated on 1,340 question events from 164 graduate students in an AI course, the classifier achieved 80.0% accuracy across 43 labels (42 topics + abstention). Topic-level question volume correlated significantly with student self-reported difficulty (Spearman's ρ = 0.491, p = 0.008), demonstrating that conversational AI interaction logs carry actionable signals about topic-level knowledge gaps. The work bridges [[student-modeling]] and [[learning-analytics]] by repurposing existing AI TA logs as diagnostic tools for instructors.}, Curricula that deploy AI teaching assistants generate a byproduct — student interaction logs — that can be mined for curriculum-level insights without additional assessment burden. This approach is complementary to [[knowledge-tracing-irt]] models because it captures which topics students find difficult (via question volume) rather than which skills they have mastered. The GPT-4-extracted prerequisite graph provides an interpretable curriculum structure that instructors can inspect and validate. ## Connected Concepts - [[student-modeling]] - [[learning-analytics]] ## Connected Articles - [[knowledge-tracing-irt]] ## Citation Medhat, Y., Park, J., Thajchayapong, P., & Goel, A. K. (2026). [*Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs*](https://arxiv.org/abs/2606.10736). arXiv:2606.10736. --- ## [Interpretable Knowledge Tracing via IRT](https://edtechdev.github.io/aied/articles/knowledge-tracing-irt/) > Two critical gaps in dialogue-based Knowledge Tracing (KT): > Most LLM-based dialogue tutoring systems produce opaque predictions. Huang et al. map raw LLM logits into **student ability (θ)** and **task difficulty (d)** parameters via Item Response Theory, yielding predictions that are both more accurate and interpretable in cognitive terms.^[[huang-interpretable-knowledge-tracing-2026]] ## The Problem with Opaque KT Two critical gaps in dialogue-based Knowledge Tracing (KT): 1. **Ignored question difficulty:** Existing methods rely only on student knowledge states. When a tutor scaffolds up or increases task difficulty, predictions become inaccurate. 2. **Opaque representations:** High-dimensional LLM embeddings lack interpretability, making it hard to align predictions with learning theory or gain tutor trust. ## The Framework Built on Llama-3.1-8B-Instruct with three modules: ### 1. Knowledge Estimator Extracts next-token logits for vocabulary tokens **"GOOD"** and **"BAD"** from dialogue history + original question: `θ = z^GOOD − z^BAD` ### 2. Difficulty Estimator Extracts logits for **"HARD"** and **"EASY"** from dialogue history + next tutor-posed task: `d = z^HARD − z^EASY` ### 3. IRT Predictor Maps ability and difficulty into correctness probability via a 1PL (Rasch) model with learnable scalar α: `p(correct) = 1 / (1 + exp(−α(θ − d)))` > *"This formulation provides interpretability since both student knowledge and task difficulty are explicitly represented as meaningful cognitive quantities."*^[[huang-interpretable-knowledge-tracing-2026]] ## Results Evaluated on QATD2k (real) and MathDial (simulated) tutor-student dialogue datasets: | Model | QATD2k AUC | MathDial AUC | |-------|------------|--------------| | DKT | 54.73 | 63.14 | | DKVMN | 51.79 | 60.64 | | LLMKT | 64.89 | 75.99 | | **Ours (IRT)** | **65.25** | **76.59** | LLM-based methods substantially outperform traditional deep learning models (∼10-15 AUC points), and the explicit IRT parameterization slightly improves over raw LLMKT.^[[huang-interpretable-knowledge-tracing-2026]] ## Relationship to Tutoring-Specific Design This framework is an enabler for [[tutoring-specific-vs-general-ai|tutoring-specific AI]]: by explicitly modeling difficulty and student readiness at each turn, the system can dynamically calibrate scaffolds rather than defaulting to full assistance. It operationalizes the insight that general-purpose LLMs can be *re-framed* as psychometric instruments through prompt design.^[[huang-interpretable-knowledge-tracing-2026]] ## Limitations - **Simulated data:** MathDial uses GPT-3.5 as the "student," which may not reflect real learner behavior - **Fine-tuning required:** LoRA on Llama-3.1-8B; not zero-shot - **Binary correctness only:** Does not model partial understanding or open-ended reasoning ## Connected Concepts - [[affective-computing]] - [[knowledge-tracing]] - [[reinforcement-learning]] - [[lifelong-learning]] - [[personalized-learning]] - [[adaptive-learning]] - [[affective-tutoring]] - [[pedagogical-agent]] ## Connected Articles - [[huang-interpretable-knowledge-tracing-2026]] - [[tutoring-specific-vs-general-ai]] ## Citation Amherst), A.S.H.A.S.J.L.A.L., gaps:, E.D.K.T.S.F.T., simpleKT, D.L.D.D.S.A., & LLMKT, L. (2026). [*Interpretable Knowledge Tracing via IRT*](https://arxiv.org/abs/2605.01097) --- ## [From Confusion to Consolidation: A Staged Conversational Workflow for Post-Lecture Review](https://edtechdev.github.io/aied/articles/knowloop-confusion-to-consolidation-2026/) > **Synthesis:** KnowLoop, a dual-agent conversational system for post-lecture review, structures learning around three stages—Recognize (mark in-situ confusion during lectures), Resolve (Teaching Assistant provides context-grounded clarification), and Consolidate (Peer scaffolds reflective teach-back). A 22-participant study shows confusion points serve as personalized review anchors, lecture-grounded clarification enables more targeted support than general-purpose AI, and teach-back prompts learners to reveal conceptual gaps and connect ideas across the lecture. ## Study Design Fang and Reidsma designed KnowLoop, a dual-agent conversational system for post-lecture review organized around learners' in-situ confusion. The system implements three stages: 1. **Recognize** — During lecture viewing, learners press a capture button at moments of confusion. The system records timestamps, aligns them with the lecture transcript, and expands to include surrounding instructional context. 2. **Resolve** — A Teaching Assistant agent (GPT-4o) provides clarification grounded in the marked confusion point and associated lecture transcript. Interaction is constrained to one confusion point at a time to maintain focus. 3. **Consolidate** — A Peer agent (GPT-4o) scaffolds reflective teach-back, revisiting confusion points in order and asking learners to explain their understanding. Progression moves from confusion-point-level articulation to lecture-level summarization. The study involved 22 participants (STEM backgrounds, AI familiarity M=4.00/5, low prior knowledge of lecture content) watching an 18-minute introductory reinforcement learning lecture. ## Key Findings ### Confusion Points as Personalized Anchors Confusion density varied substantially across participants, with no single lecture segment consistently eliciting confusion. This highlights the highly personalized nature of in-lecture breakdowns. Participants also repurposed markers to flag important concepts, not just confusion: *"It wasn't confusion—I clicked because it was an important formula to review later."* ### Lecture-Grounded Clarification The Teaching Assistant provided targeted, course-aligned explanations by grounding responses in the lecture transcript: *"The biggest difference is that GPT has no idea what happened in the lecture… but this system knows the entire lecture."* The agent also actively managed instructional relevance, noting when a question was not a key point and suggesting learners move on. ### Teach-Back Surfaces Gaps The Consolidate stage consistently exposed gaps between what learners believed they understood and what they could articulate: *"Understanding something in your head is one thing; saying it out loud is another."* The Peer's follow-up questions surfaced blind spots that clarification alone had not revealed. ### Tensions - The Teaching Assistant sometimes redirected questions back to lecture content when learners wanted broader exploration - Responses could be too long and text-heavy - The transition into teach-back felt abrupt and anxiety-inducing - Participants wanted the ability to switch between Resolve and Consolidate fluidly ## Design Implications 1. **Anchor AI support in learners' own difficulties** — confusion points, whether marked manually or inferred, provide personalized entry points for review 2. **Distinguish clarification from consolidation through separate agent roles** — these require qualitatively different conversational relationships 3. **Support fluid transitions** between stages — learners experience Resolve and Consolidate as interleaved, not strictly sequential 4. **Context-grounded AI outperforms general-purpose AI for lecture review** — knowing the lecture content enables more targeted, efficient support ## Connected Concepts - [[ai-tutoring]] - [[higher-ed]] - learning-by-teaching - [[personalized-learning]] - [[scaffolding]] - [[student-experience]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation Fang, M., & Reidsma, D. (2026). [*From Confusion to Consolidation: A Staged Conversational Workflow for Post-Lecture Review*](https://doi.org/10.1145/3816046.3816288). In *ACM Conversational User Interfaces 2026 (CUI '26)*, Bremen, Germany. --- ## [KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing](https://edtechdev.github.io/aied/articles/kt4eqg-personalized-question-generation/) **KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing** bridges two key AI-in-education paradigms: [[personalized-learning]] through question generation and [[learning-analytics]] through knowledge tracing. Rather than generating generic practice questions, KT4EQG uses a Knowledge Tracing model to first identify the knowledge concept that would maximize a student's potential improvement in overall mastery, then trains an [[llm]]-based generator to produce a question faithfully grounded in that concept. This two-stage architecture — KT for concept selection, LLM for faithful question generation — outperforms less personalized baselines on XES3G5M and MOOCRadar datasets. The approach represents a significant advance in [[adaptive-learning]] system design, connecting to [[knowledge-tracing-irt]] research on modeling student knowledge states and [[automated-question-generation]] work on producing high-quality educational content. Unlike earlier systems such as [[slidesqaqa-pedagogical-question-generation]] that generate questions from static content, KT4EQG personalizes based on dynamic student models, aligning with [[intelligent-tutoring]] goals of providing the right question at the right time for each learner. ## Connected Concepts - [[personalized-learning]] - [[learning-analytics]] - [[llm]] - [[adaptive-learning]] - [[automated-question-generation]] - [[intelligent-tutoring]] ## Connected Articles - [[knowledge-tracing-irt]] - [[slidesqaqa-pedagogical-question-generation]] ## Citation Xinyi Gao, Qiucheng Wu, Lu Ding, Q. Vera Liao, Kaizhi Qian, Ying Xu, Shiyu Chang, Yang Zhang (2026). [KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing](https://arxiv.org/abs/2605.23933). arXiv:2605.23933. arXiv preprint. --- ## [Kutti AI: A Voice-First, Offline-Capable Learning Companion with Real-Time Struggle Detection for Visually-Impaired Children](https://edtechdev.github.io/aied/articles/kutti-ai-voice-first-learning-companion/) Kutti AI addresses a persistent equity gap in educational technology: nearly all edtech assumes a visual interface, excluding an estimated 1.4 million blind children worldwide. The system inverts this assumption entirely, making spoken conversation the primary and sufficient learning modality — children hear curriculum content, answer aloud, and receive spoken feedback with no visual dependency, positioning it within the [[special-education]] and accessibility strand of [[adaptive-learning]] research. Three engineering contributions make this practical on commodity mobile hardware. First, a multi-signal struggle-detection engine fuses response latency, wrong-attempt counts, and keyword-based hesitation cues to decide in real time when to offer hints or simplify questions — a lightweight alternative to the learner-modeling machinery of full [[intelligent-tutoring]]. Second, a cross-language answer-matching pipeline (translation/transliteration, Levenshtein fuzzy matching, text normalization) ensures children are not penalized for code-switching or pronunciation variation, an important fairness property for multilingual learners and a concrete instance of [[equity]]-aware design. Third, an offline-first on-device ASR pipeline removes the connectivity requirement, extending [[personalized-learning]] to low-resource settings where cloud-dependent tutors fail. The paper is a systems contribution rather than an efficacy study — no learning-gains evaluation is reported — so claims about pedagogical impact should be treated as design hypotheses pending classroom trials. Nonetheless it is a rare example of [[student-experience]] research that centers disabled learners from the outset rather than retrofitting accessibility. ## Connected Concepts - [[special-education]] - [[adaptive-learning]] - [[intelligent-tutoring]] - [[equity]] - [[personalized-learning]] - [[student-experience]] ## Connected Articles - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components - [[ai-powered-personalized-learning-elementary-fractions-2026]] — Exploring Fraction Comprehension and Interest in Elementary Education Through AI-Powered Personalized Learning - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing ## Citation Kadharmoideen Fadurudeen (2026). [Kutti AI: A Voice-First, Offline-Capable Learning Companion with Real-Time Struggle Detection for Visually-Impaired Children](https://arxiv.org/abs/2607.22377). arXiv:2607.22377. arXiv preprint. --- ## [Revisiting the Hint Button: Consistent Negative Associations Between Unproductive Hint Use and Learning Outcomes in Intelligent Tutoring Systems](https://edtechdev.github.io/aied/articles/lak2026-hint-button-unproductive-use/) > **Synthesis:** A three-semester, 999-student analysis of hint usage in a K-12 mathematics ITS finds that two simple, interpretable indicators—premature hint requests and superficial hint reading—are consistently associated with reduced learning gains, even after controlling for prior knowledge. The work argues from an affordance perspective that the persistent "hint button" design common across ITSs can inadvertently enable bypass strategies, transforming scaffolds into shortcuts. ## Study Design An et al. conducted a multi-semester observational study of 999 K–12 students (3 cohorts: Spring 2021, Fall 2021, Fall 2022) using *Decimal Point*, a digital learning game with an underlying [[intelligent-tutoring]] system built with CTAT (Cognitive Tutor Authoring Tools). The game covers decimal concepts and operations across 24 mini-games with multi-level on-demand hints. Students completed pretest, immediate posttest, and one-week delayed posttest assessments. ## Two Unproductive Hint-Use Indicators The paper identifies and validates two computationally straightforward behavioral indicators from fine-grained interaction logs: 1. **Premature hint requests** — requesting hints before making any solution attempt. Even when students are uncertain, attempting a solution first before seeking help is more beneficial for learning. 2. **Superficial hint reading** — advancing through hints too rapidly to reasonably read them (flagged using a 4 words/second reading-speed benchmark), often skipping directly to the bottom-out hint that reveals the answer. ## Key Findings - **Pre-post learning gains were significant across all semesters** (η²_p = 0.059–0.259, all p < .001), confirming ITS effectiveness. - **Both unproductive behaviors were strongly negatively correlated with pretest scores** (ρ = −0.57 to −0.74), meaning lower-prior-knowledge students engaged in them more. - **After controlling for pretest scores via OLS regression**, premature hint requests still significantly predicted lower posttest (β = −0.14 to −0.28) and delayed posttest scores (β = −0.17 to −0.37). Superficial reading showed similar patterns (β = −0.06 to −0.11 posttest; β = −0.08 to −0.14 delayed). - **These associations replicated across all three semesters** with remarkable consistency (999 students total). - **The negative associations were strongest for delayed posttests**, suggesting unproductive hint use particularly harms knowledge retention. ## Theoretical Framing The authors interpret findings through two lenses: - **KLI Framework** (knowledge-learning-instruction): Mathematical skills in *Decimal Point* engage induction and refinement processes requiring active schema construction. Rapidly accessing bottom-out hints circumvents these cognitive activities. - **Affordance Perspective**: The persistent, salient "hint button" signals to learners that help is always available. For some students, this creates an unintended affordance where the interface effectively collapses into one that reveals the answer immediately, transforming the task into a copying exercise. ## Design Implications Rather than removing hints, the paper proposes re-engineering delivery mechanisms: 1. **Delayed hint availability** — requiring minimum engagement time or solution attempts before hints are accessible, particularly for the bottom-out hint. 2. **Beyond behavioral compliance** — prior work shows that interventions regulating help-seeking can improve observed behaviors without producing corresponding learning gains. The challenge extends beyond timing to ensuring meaningful [[active-learning]]. 3. **Shift from whether to how** — the central design question should move from whether to provide hints to how to structure hint delivery mechanisms aligned with productive-struggle principles. ## Practical Significance Unlike prior "gaming the system" detectors requiring complex machine-learned models with 24–40 features per action, these two indicators are simple, interpretable, and computable from standard ITS logs — making them practical for [[learning-analytics]] dashboards and automated real-time interventions across diverse educational settings. ## Limitations - Correlational (observational data), not causal - "Superficial reading" flagged via estimated reading speed (4 wps) — some instances may reflect faster-but-meaningful reading - Domain-specific to K–12 mathematics; generalizability to other domains or older learners remains an open question - Some high-performing students may strategically use bottom-out hints as worked examples ## Connected Concepts - [[math-education]] - [[learning-analytics]] - [[help-seeking]] - [[k-12]] - [[learning-analytics]] - [[student-experience]] - [[intelligent-tutoring]] - [[active-learning]] ## Connected Articles - [[zhang-tutormoments-2026]] - [[kar-mathbuddy-affective-math-tutoring-2025]] ## Citation McLaren, A.M.A.M.M.J.S.B.M., Associations, R.T.H.B.C.N., in, B.U.H.U.A.L.O., Systems, I.T., An, M., Mehrvarz, M., Mellon, C., & PA, P. (2026). [*Revisiting the Hint Button: Consistent Negative Associations Between Unproductive Hint Use and Learning Outcomes in Intelligent Tutoring Systems*](https://doi.org/10.1145/3785022.3785040). LAK26: 16th International Learning Analytics and Knowledge Conference (LAK 2026), April 27–May 01 --- ## [LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework](https://edtechdev.github.io/aied/articles/lata-ferpa-compliant-local-llm-autograder/) > LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework **Rodríguez (2026)** — Oregon State University. Submitted to Computers & Education. LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework ## Summary LaTA (LaTeX Teaching Assistant) is a **privacy-preserving, drop-in autograder** that addresses the core tension in educational LLM deployment: most [[automated-grading]] systems send student work to third-party APIs, violating FERPA and exposing institutions to data risk. **System architecture:** - **Four-stage pipeline:** Ingest → Segment → Grade → Report - **Grading engine:** gpt-oss:120b running **entirely on local commodity hardware** (single Mac Studio) - **Rubric format:** YAML with binary per-item scoring, comparing student work against instructor-authored reference solutions - **Cost:** $0 marginal cost per assignment, 1-3 minutes per submission **Real-world deployment (Winter 2026):** - **Course:** ME 373 at Oregon State University (~200 students) - **Grading error rate:** 0.02-0.04% per rubric line item — near-perfect accuracy - **Operational benefit:** Enabled regrading of corrected assignments and expanded TA office hours **Learning outcomes (vs. traditional cohort):** - **Midterm exam:** +11% performance gain - **Final exam:** +8% performance gain - **Confidence gains:** Δ ≥ +1.49 Likert points on every learning objective (N=159, p < 10⁻²⁷) These results provide strong evidence for the [[feedback-loop]] hypothesis: faster, more consistent feedback drives both learning and confidence. The deployment demonstrates that [[generative-ai]] grading can be both FERPA-compliant and pedagogically effective, addressing concerns raised in [[assessment-validity]] and [[formative-assessment]] discussions. LaTA's success connects to the broader [[stem-education]] and [[higher-ed]] landscape, showing that [[llm]]-based grading can move beyond [[short-answer-scoring-quality-degradation]] concerns when properly designed with instructor-authored rubrics and reference solutions. The open-source, zero-marginal-cost model aligns with [[principled-ai-education]] principles. ## Connected Concepts - [[automated-grading]] - [[feedback-loop]] - [[generative-ai]] - [[assessment-validity]] - [[formative-assessment]] - [[stem-education]] - [[higher-ed]] - [[llm]] ## Connected Articles - [[short-answer-scoring-quality-degradation]] - [[principled-ai-education]] ## Citation Rodríguez, J. A. (2026). [LaTA: A drop-in, FERPA-compliant local-LLM autograder for upper-division STEM coursework](https://arxiv.org/abs/2605.05410). arXiv:2605.05410. Submitted to Computers & Education. --- ## [Beyond Compliance: A Proposed Framework for Ethical Governance of Student Data in Learning Analytics](https://edtechdev.github.io/aied/articles/league-ethical-governance-student-data-2026/) > **Beyond Compliance: A Proposed Framework for Ethical Governance of Student Data in Learning Analytics** — Proposes LEAGUE framework (Lawfulness, Equity, Agency, Governance, Utility, Ethics by Design) for ethical governance of student data in learning analytics. Synthesizes scholarship across LA, educational data mining, data ethics, educational policy, v... [[learning-analytics]] [[privacy]] [[equity]] [[ethics]] [[regulation]] [[higher-ed]] Proposes LEAGUE framework (Lawfulness, Equity, Agency, Governance, Utility, Ethics by Design) for ethical governance of student data in learning analytics. Synthesizes scholarship across LA, educational data mining, data ethics, educational policy, value-sensitive design, and capability-oriented approaches to educational justice. Demonstrates practical value through an illustrative early-alert case study showing how institutions can review LA practices in a more transparent and educationally meaningful way. ## Abstract The rapid growth of learning analytics (LA) in higher education has expanded institutional capacity to monitor engagement, predict academic difficulty, and target support using student data. While these practices offer important educational benefits, governance has often remained compliance-first, centered on meeting baseline legal requirements such as FERPA and GDPR. This paper proposes the LEAGUE framework, a six-pillar model for ethical governance of student data in LA: Lawfulness, Equity, Agency, Governance, Utility, and Ethics by Design. ## Connected Concepts - [[learning-analytics]] - [[privacy]] - [[equity]] - [[ethics]] - [[regulation]] - [[higher-ed]] ## Connected Articles - [[ai-uk-higher-education-policy-2026]] — Artificial Intelligence in UK Higher Educational Policy and Institutional Decision Making - [[ai-lifelong-learning-policy]] — Artificial Intelligence in Lifelong Learning: Opportunities and Challenges in Adult Education Policy - [[white-wu-robotics-ai-education-2026]] — Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions - [[xai-education-framework]] — Explainable Artificial Intelligence in Education (XAI-ED) - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Sahana Varadaraju, & Bharathwaj Vijayakumar (2026). [Beyond Compliance: A Proposed Framework for Ethical Governance of Student Data in Learning Analytics](https://arxiv.org/abs/2608.03968). arXiv:2608.03968. EDULEARN26 Proceedings (IATED, 2026). --- ## [Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course](https://edtechdev.github.io/aied/articles/learner-ai-interaction-patterns-oop/) Examines how different forms of learner-AI interaction relate to learning outcomes in object-oriented programming courses. Identifies distinct patterns of GenAI use among students and correlates them with academic performance, finding that certain interaction patterns (seeking explanation rather than code generation) are associated with stronger learning outcomes. ## Key Findings - Survey data were collected from **210 undergraduate students** in a first-year object-oriented programming (OOP) course where **GenAI use was permitted for coursework but prohibited in assessments**. - Students used GenAI **significantly more often for explanation seeking and debugging than for code generation**. - **Cluster analysis identified five distinct learner-AI interaction profiles**, including a "smart" high-usage pattern characterized by **low reliance on code generation** and high use for conceptual support and debugging. - Usage patterns were associated with differences in **perceived assignment difficulty, self-assessed understanding, trust in AI-generated code, and norm-related attitudes**. - Critically, **no significant differences in assessment performance were found across clusters** — self-directed GenAI use alone did not lead to measurable learning gains. ## Study Design & Method This full research paper investigates how students integrate GenAI tools when learning OOP and how different patterns of use relate to learning experiences and outcomes. The study surveyed 210 first-year undergraduates about their self-directed GenAI use, academic performance, perceived difficulty, understanding, and trust. Cluster analysis was used to derive learner-AI interaction profiles, which were then compared on the self-report and performance measures. ## Implications for AI in Education The absence of performance differences across usage clusters underscores the need for **pedagogically guided and process-aware AI support** in programming education: letting students self-direct their GenAI use, even in sophisticated patterns, does not by itself produce learning gains. The "smart" profile — high conceptual support and debugging with low code generation — offers a concrete target for course design, suggesting educators should steer students toward explanation seeking and debugging rather than answer generation in [[cs-education]] settings. ## Connected Concepts - [[cs-education]] - [[administrator]] - [[math-education]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[socratic-ai-dialogue]] - [[rct]] - [[physics-education]] ## Connected Articles - [[ai-team-teaching-talk-analytics]] — AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[cold-start-knowledge-tracing-safeinsights]] — Reexamining the Cold-Start Problem in Knowledge Tracing Models and Implications for SafeInsights - [[generative-ai-reduced-study-time-math]] — Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[ai-generated-instructional-videos-computing-ed]] — Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education ## Citation Marina Lepp (2026). [Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course](https://arxiv.org/abs/2607.24755). arXiv:2607.24755. cs.HC, cs.AI, cs.CY. --- ## [Enhancing learner-centered feedback with AI: teachers'' practices and perceptions](https://edtechdev.github.io/aied/articles/learner-centered-feedback-ai/) # Enhancing learner-centered feedback with AI: teachers' practices and perceptions ## Summary An empirical study of **21 higher-education teachers** using **PolyFeed**, an AI-powered feedback tool combining (1) a **BERT-based ML model** (from Aldino et al. 2024) that detects which learner-centered feedback components are missing from teacher-written feedback and suggests them, and (2) **ChatGPT-4o mini** to rephrase/enhance the teacher's draft. Teachers gave feedback on a simulated student presentation, then used the tool, then were interviewed. The study answers two questions: *how teachers interact with* AI feedback tools (RQ1) and *how they perceive* them (RQ2). Framework: Ryan et al.'s (2023) learner-centered feedback dimensions — **Future Impact, Sensemaking, Agency**. ## RQ1 — How teachers interacted with the tools ### ML suggestion acceptance (detection → adoption) - The ML model most often flagged missing **Meeting Learning Objective** (20 of 21 teachers omitted it; **16 accepted** the suggestion) and **Student–Teacher Relationship** (14 omitted; **12 accepted**). - **Affirmation and Encouragement** was missing in 6 cases but accepted by **all** 6 teachers (100%). - Teachers used professional judgement to accept or reject suggestions — the ML functioned as a diagnostic assistant, not an autopilot. ### Revision of ChatGPT-enhanced feedback - **12 of 21 teachers** made further sentence-level revisions; the rest left the text unchanged. - Most common actions: **editing (f = 32)** and **removing (f = 27)**; adding was rare (f = 8). - The dominant pattern was **calibrating tone**: Edit–Praise (f = 11, e.g. "First of all, I want to congratulate you on completing your assignment!" → "Well done on completing your assignment"), Remove–Suggestion (f = 9), Remove–Encouragement (f = 8), Remove–Praise (f = 6), Edit–Correction (f = 9). Teachers systematically **moderated exaggerated praise and generic suggestions** to protect authenticity and professional voice. - Revisions clustered most strongly in the **Agency dimension** — especially Student–Teacher Relationship (f = 24; encouragement f = 11) — confirming that the relational/affective dimension of feedback resists AI delegation. ## RQ2 — Teacher perceptions - **Benefits:** promotes reflection (n = 14), improves language and structure (n = 11), identifies missing components (n = 10), saves time (n = 2). - **Challenges:** need for human editing (n = 9), inconsistent tone (n = 7), potential misinformation (n = 5), trust issues (n = 5). - **Experience gap:** teachers with **>5 years of experience reported more challenges** (tone, editing burden, trust, misinformation); **less-experienced teachers valued scaffolding benefits** (identifying missing components, saving time). The authors flag a developmental risk: novice teachers who defer to AI suggestions may build less independent feedback judgement. ## Interpretation - The tools were most valuable as **reflective prompts that surface overlooked aspects** of feedback — "making invisible gaps visible" — scaffolding teachers' evaluative judgement, rather than as providers of finalised text (an **"assist but verify"** pattern; cf. [[human-in-the-loop-ai]]). - **Design implications:** adjustable tone parameters, discipline-specific templates, and transparency controls; without them AI adoption may *increase* rather than reduce workload and can erode teacher authority if positioned as autonomous providers. - The authors call for future research on **student perspectives** of AI-assisted teacher feedback and longitudinal study of whether reliance strengthens or displaces evaluative judgement. ## Connected Concepts - [[faculty-development]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[scaffolding]] - [[teacher-role]] - [[generative-ai]] - [[rag]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[chatgpt-feedback-engagement-genai]] — Students' engagement with ChatGPT feedback: implications for student feedback literacy in the context of generative a... - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and teacher feedback: student perceptions of usefulness and trustworthiness - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams ## Citation Aldino, A. A., Maheshi, B., Li, Y., Zhou, Y., Tsai, Y.-S., Gašević, D., & Chen, G. (2026). Enhancing learner-centered feedback with AI: Teachers' practices and perceptions. [*Assessment & Evaluation in Higher Education*](https://doi.org/10.1080/02602938.2026.2638920), 51(5), 892–910 --- ## [Learning behavior accounts for background-related advantage in AI-assisted education](https://edtechdev.github.io/aied/articles/learning-behavior-background-advantage-ai-ed/) Investigates why AI-for-education shows inconsistent average effects, arguing that learning behavior explains background-related advantage: students from advantaged backgrounds engage with AI tools in ways that compound gains, while others do not. Prior ed-tech research shows average effects mask heterogeneity; this paper quantifies the behavioral mechanism. Links [[generative-ai]] use to learning-gains, [[personalized-learning]], and [[student-experience]], with strong [[equity]] implications: AI assistance may widen gaps unless designed to shift behavior. Connects to [[ai-assisted-learning-modes-eeg]] and the [[over-reliance]] literature on differential benefit. ## Connected Concepts - [[generative-ai]] - [[personalized-learning]] - [[student-experience]] - [[equity]] - [[over-reliance]] ## Connected Articles - [[ai-assisted-learning-modes-eeg]] ## Citation Jingwei Yi, Yueqi Xie, Jiyan He, Rui Ye, Junming Huang, Bin Zhu, Sean Rintel, Yu Xie, Xing Xie, Fangzhao Wu (2026). [Learning behavior accounts for background-related advantage in AI-assisted education](https://arxiv.org/abs/2607.10101). arXiv:2607.10101. arXiv preprint. --- ## [Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning](https://edtechdev.github.io/aied/articles/learning-by-chatting-genai-impact/) > **Shravika Mittal, Su Lin Blodgett, Q. Vera Liao** **Shravika Mittal, Su Lin Blodgett, Q. Vera Liao** ## Summary In an 8-day field experiment comparing ChatGPT vs. Google Search for informal learning, participants using ChatGPT experienced diminished agency, offloaded information selection to AI, and suffered greater meta-cognitive load — resulting in worse learning outcomes, particularly for higher-order critical thinking. The study identifies two key distortions in ChatGPT-mediated information access: output bias toward solution-oriented artifacts over principled knowledge, and a conversational interaction paradigm that reduces exploration of the broader knowledge space. ## Key Findings - **8-day field experiment** with between-subjects design (ChatGPT vs. Google Search) using daily diary protocols for in-situ data collection. - **Diminished agency:** ChatGPT participants offloaded information selection to the AI, reducing their sense of control over the learning process. - **Higher meta-cognitive load:** The reduced sense of control paradoxically increased cognitive burden, as participants had to monitor and evaluate AI-curated outputs. - **Two sources of distortion:** 1. **Output bias** — ChatGPT favours providing solution-oriented artifacts (answers, code, summaries) over principled knowledge (explanations, theory, context). 2. **Behavioral shift** — The conversational, socially-oriented interaction paradigm reduces exploration of the broader knowledge space. - **Worse learning outcomes:** ChatGPT group performed worse overall, especially on higher-order critical learning tasks. - **Core tension:** Offloading information seeking to AI for efficiency inherently conflicts with the depth of processing required for meaningful learning. ## Implications for AIED ### For AI Tutor Design - The finding that ChatGPT's output bias favours "solution-oriented artifacts over principled knowledge" directly parallels the challenge identified in [[tutoring-effectiveness-index|tutoring effectiveness]] — that AI tutors must be designed to elicit reasoning, not provide answers. - The PeteChat/Tutor Not Solver design principles directly address this tension through homework guardrails and SRL support. ### For Metacognition & Self-Regulated Learning - The study provides empirical evidence for the theoretical concern raised in [[llm-fallacy-misattribution]]: that learners misattribute AI-generated outputs to their own understanding, short-circuiting metacognitive monitoring. - The increased meta-cognitive load observed when agency is diminished echoes findings in [[self-regulated-learning]] about the importance of learner control. ### For Technology-Enhanced Learning - Results run counter to the assumption that easier information access automatically improves learning — consistent with the [[over-reliance]] literature showing that AI tools can reduce actual learning while maintaining (or inflating) perceived learning. - The finding that ChatGPT reduces exploration aligns with [[efficiency-gain-illusion-ai-overreliance]]: learners overestimate the benefits of AI assistance on simple tasks. - Supports the case for [[scaffolding]] that preserves learner agency rather than replacing cognitive work. ### For AI Literacy - The study highlights the need for [[ai-literacy]] curricula that teach learners when and how to use AI tools productively, and when to rely on traditional search and self-directed exploration. - Educators should be aware that conversational AI interfaces may inadvertently narrow learning behaviours even when the content seems helpful. ## Connected Concepts - [[self-regulated-learning]] - [[over-reliance]] - [[scaffolding]] - [[ai-literacy]] ## Connected Articles - [[tutoring-effectiveness-index]] - [[llm-fallacy-misattribution]] - [[efficiency-gain-illusion-ai-overreliance]] ## Citation Mittal, S., Blodgett, S. L., & Liao, Q. V. (2026). [Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning](https://arxiv.org/abs/2606.11669). arXiv:2606.11669. --- ## [Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System](https://edtechdev.github.io/aied/articles/learning-engagement-assistant-lea/) LEA (Learning Engagement Assistant) is an **agentic AI tutoring system** that couples course-specific retrieval-augmented generation (RAG) with structured [[knowledge-tracing]] / Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz modes. This paper reports the first real-student classroom deployment of LEA (n = 8, STEM course CMP511) and the first empirical test of its cross-course scalability, extending a prior simulation-only validation that used synthetic learner agents. The study exposes a key gap between simulated evaluation and [[student-experience|real classroom deployment]]: synthetic learners predicted engagement patterns that diverged from observed behaviour, arguing that simulation alone cannot anticipate all aspects of live use. A RAGAS-based scalability evaluation across 660 questions found Answer Relevancy (0.88-0.94) and Context Precision (0.88-0.90) stable across courses, while Faithfulness declined with curriculum distance from LEA's original course (0.69 to 0.50) — a preliminary signal that downstream components, not the orchestration layer, constrain course-agnostic tutoring. The work sits within the broader literature on [[intelligent-tutoring]] and [[agentic-workflows-education]], with implications for [[higher-ed]] and [[stem-education]] deployment, and connects to debates on [[formative-assessment]] quality and the limits of automated [[feedback-loop]] in [[personalized-learning]]. ## Connected Concepts - [[knowledge-tracing]] - [[student-experience]] - [[intelligent-tutoring]] - [[higher-ed]] - [[stem-education]] - [[formative-assessment]] - [[feedback-loop]] - [[personalized-learning]] ## Connected Articles - [[agentic-workflows-education]] ## Citation Teri Rumble, Javad Zarrin, P. George Lovell, Ruth Falconer (2026). [Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System](https://arxiv.org/abs/2607.13370). arXiv:2607.13370. --- ## [Learning-to-learn in the age of generative AI: A scoping review and conceptual framework](https://edtechdev.github.io/aied/articles/learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr/) > **Synthesis:** This paper presents a scoping review of learning-to-learn (L2L) definitions within pedagogical and psychological literature, identifying 21 relevant publications via PRISMA-ScR. It proposes a novel three-layered framework organized by conceptual broadness: Dimensions (cognitive and metacognitive skills), Processes (self-regulation), and Tools (retrieval practice). The framework maps L2L components to GenAI application use cases in higher education, positioning L2L as key to reducing GenAI overreliance and fostering learner agency. ## Key Findings This paper presents a scoping review of learning-to-learn (L2L) definitions within pedagogical and psychological literature, identifying 21 relevant publications via PRISMA-ScR. It proposes a novel three-layered framework organized by conceptual broadness: Dimensions (cognitive and metacognitive skills), Processes (self-regulation), and Tools (retrieval practice). The framework maps L2L components to GenAI application use cases in higher education, positioning L2L as key to reducing GenAI overreliance and fostering learner agency. ## Connected Concepts - [[generative-ai]] ## Connected Articles - [[liang-genai-systematic-review-human-ai-2026]] - [[learning-by-chatting-genai-impact]] - [[generative-refusal-ai-tools-for-thought]] - [[genai-runaway-object-math-higher-ed]] ## Citation## Citation Schorr, I., Bardach, L., Bühler, B., & Kasneci, E. (2026). [*Learning-to-learn in the age of generative AI: A scoping review and conceptual framework*](https://doi.org/10.1016/j.caeai.2026.100575). *Computers and Education: Artificial Intelligence*. --- ## [Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring](https://edtechdev.github.io/aied/articles/learning-to-prompt-adaptive-tutoring/) > **Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer** (2026). arXiv cs.AI preprint **Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer** (2026). arXiv cs.AI preprint ## Overview Adaptive LLM tutoring with subject-aware prompt routing based on 14 pedagogical features. A/B test on 656 conversations from 359 Dutch high-school students showed sim-to-real transfer. A stochastic router achieved 28.1% exercise conversion rate vs 19.6% baseline. Reduces interaction turns by ~3 (p=0.007) while maintaining pedagogical quality. ## Key Contributions ### Paper 1: Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations - **Problem:** Students often ignore well-designed program visualizations; existing cognitive design principles don't explain learner engagement/disengagement. - **Method:** Within-subjects study (N=19 undergraduates, post-CS1/CS2) using think-aloud, interviews, and webcam gaze tracking with a multi-representational probe. - **Gaze finding:** ~47% of time on code despite visual scaffolds; students without prior experience anchored more in code and ignored metaphor views. - **Three engagement themes:** Agency (control over cognitive effort), Representational Fit (wide individual variation), Legitimacy (metaphors seen as childish at university level). - **Implication:** multi-representational-tools need attention to affective and social factors, not just cognitive design. ### Paper 2: Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - **Problem:** Static-prompt LLM tutoring systems fail to adapt across diverse academic disciplines. - **Solution:** Subject-aware prompt routing using 14 pedagogical features extracted from transcripts; contextual bandit formulation with 20 pedagogical prompts. - **Simulation:** Router achieves 0.694 vs 0.647/0.64 static baselines (p<0.001). - **Real-world A/B test:** N=656 conversations, 359 Dutch high-school students. Stochastic router achieves 28.1% exercise conversion rate vs 19.6% baseline. - **Implication:** [[adaptive-prompt-routing]] with stochastic sampling improves both efficiency and engagement in real-world tutoring. ### Paper 3: Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - **Problem:** Automated scoring of student-drawn scientific models lacks reliability indicators, leaving teachers unable to decide when to trust scores. - **Method:** Vision Transformer (ViT) with LoRA + confidence-aware framework using test-time perturbations. - **Dataset:** Six NGSS-aligned middle school assessment items (477-816 responses each, scored Beginning/Developing/Proficient). - **Key innovation:** Response-level confidence enables selective automation — high-confidence auto-scored, uncertain cases deferred for human review. - **Implication:** [[confidence-aware-ai-assessment]] enables practical triage between automation and human oversight in educational assessment. ### Paper 4: PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - **Problem:** AES systems treat scoring and feedback as separate; neural scoring lacks interpretability; LLM feedback is ability-agnostic. - **Solution:** Unified psychometric latent space (θ) via Neural GPCM Trait-Adaptive Scorer. - **Scoring performance:** QWK 0.747 (besting prior SOTA 0.722); 1st in 6/8 ASAP++ prompts and 10/11 trait dimensions. - **Feedback quality:** ZPD-based strategy mapping (Explicit Correction for θ<-1, Scaffolding for -1<θ<1, Socratic for θ>1). Multi-agent fusion from Llama-4-Scout, Qwen3-235B, GPT-4o. - **Evaluation:** >90% win rate in Actionability vs GPT-4o, Llama-4-Scout, Qwen3. - **Implication:** psychometrically-aware-ai can unify assessment and instruction, transforming automated essay scoring from summative to formative. ## Connected Concepts - [[adaptive-prompt-routing]] - [[confidence-aware-ai-assessment]] ## Connected Articles - [[code-anchor-multi-view-visualization]] — Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[psyscore-essay-scoring-zpd-feedback]] — PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - [[llm-judged-helpfulness-pedagogy-signal]] — Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[special-r1-rl-special-education]] — Special-R1: Reinforcement Learning for Special Education — Aligning LLM Tutors to Diverse Learners through Disability-Adaptive Training ## Citation Po-Chin Chang, Nicholas Hogan, Aske Plaat, Michiel T. van der Meer (2026). [Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring](https://arxiv.org/abs/2606.20138). arXiv:2606.20138. arXiv cs.AI preprint. --- ## [Learning with machines: Toward a theory of epistemic co-agency](https://edtechdev.github.io/aied/articles/learning-with-machines-toward-a-theory-of-epistemic-co-agency/) > **Synthesis:** Samuel (2026) introduces the **Epistemic Entanglement Framework**, a theory-informed model for understanding how learners engage with generative AI (GenAI) systems. Arguing that existing learning theories (constructivism, sociocultural theory, connectivism) presume human-centered epistemic agency and cannot account for the ways GenAI simulates reasoning, reframes arguments, and co-constructs meaning, the paper proposes **epistemic co-agency** — a reflexive stance in which learners engage AI outputs dialectically, challenging assumptions, surfacing contradictions, and asserting epistemic sovereignty. The central claim is that the real challenge of AI in education is not technological fluency but cultivating learners who can reason *with, through, and against* generative systems. ## Key Findings - The paper identifies three **epistemic configurations** of human-AI interaction: **Epistemic Augmentation (EA)** — learners treat AI as a content generator, adopting a low-interrogation posture that improves surface productivity but leaves knowledge construction passive; **Epistemic Integration (EI)** — learners strategically prompt, refine, and coordinate AI outputs into evolving reasoning, managing knowledge production without interrogating the epistemic status of claims; and **Epistemic Co-Agency (EC)** — learners treat AI outputs as epistemically contestable contributions, engaging them dialectically while maintaining epistemic sovereignty and accountability for warranting knowledge. - These configurations are **not stages of maturity** but patterned epistemic relationships that vary by task, design, and learner posture; the task of learning design is to help learners recognize which configuration they are in, understand its limits, and develop agility to shift postures as needed. - **Epistemic entanglement** refers to the condition where knowledge construction emerges through reciprocal interaction between human reasoning and AI-generated contributions, such that the boundaries of authorship, authority, and justification become hard to disentangle. - **Epistemic co-agency** explicitly rejects "AI partner"/"co-author" metaphors (which imply mutuality and shared intention); it focuses instead on the learner's capacity to maintain epistemic sovereignty while navigating an entangled system of influence and response. - The framework warns against current AI-in-education tools that prioritize efficiency and risk reinforcing **passive epistemic postures** and overtrust, calling for design that embeds "epistemic friction" (prompt templates requiring justification, multiple-output displays, confidence visualizations, critique scaffolds like "What's missing?" and "Whose perspective is absent?"). ## Study Design & Method This is a conceptual/theoretical paper proposing a framework, not an empirical study. Drawing on distributed cognition, sociomaterialism, posthumanist theory, and the concept of relational agency, the author develops the Epistemic Entanglement Framework and articulates the three configurations (EA, EI, EC) in terms of distinct human postures, cognitive demands, and design implications. The configurations are also mapped onto analytic dimensions from Thürmel's (2015) graduated agency framework (activity level, adaptivity, interaction potential, personification, joint agency). The paper concludes with pedagogical and assessment implications for scaffolding learners toward co-agency, and acknowledges the author used ChatGPT as a supplementary writing/ideation tool during manuscript development. ## Implications for AI in Education The paper reframes the learner-AI relationship as a **developmental epistemic process** rather than static tool-use or naive collaboration. For [[instructional-design|instructional design]], it argues current tools (SchoolAI, Curipod, ChatGPT) prioritize efficiency in ways that risk reinforcing passive epistemic postures, and calls for embedding epistemic friction and critique scaffolds. For [[formative-assessment|assessment]], it argues traditional assessments of correctness/coherence are poorly suited to capture epistemic shifts; instead, assessments should measure the reasoning trajectory (how the learner moved from question to insight and how AI figured in that arc), with reflective prompts, process explanations, and evidence of revision — while protecting against superficial "AI detection" that reduces engagement to surveillance. It positions GenAI not as an "assistant" but as an object of inquiry and a new epistemic actor, connecting to [[ai-literacy]], [[critical-thinking]], [[metacognition]], and [[cognitive-offloading]], and cautioning against [[over-reliance]]. ## Limitations As a conceptual paper, the framework has not been empirically tested; the author explicitly calls for future research to test and refine the model. The framework's configurations are theoretical constructs without operationalized measures. The paper does not provide concrete curriculum designs, only illustrative examples. The context is higher education, and the author notes disciplinary norms vary in how knowledge is validated, which may affect how the framework applies across fields. ## Connected Concepts - [[ai-literacy]] - [[critical-thinking]] - [[metacognition]] - [[over-reliance]] - [[cognitive-offloading]] - [[formative-assessment]] - [[scaffolding]] - [[higher-ed]] ## Connected Articles - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] — Metacognitive AI Literacy: Beyond the Skills Gap - [[genai-over-reliance-learning-2026]] — Generative AI and Over-Reliance - [[ai-metacognition-stem-review]] — AI Tools Scaffolding Metacognition in STEM - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] — Learning-to-Learn in the Age of Generative AI ## Citation Samuel, A. (2026). [*Learning with machines: Toward a theory of epistemic co-agency*](https://doi.org/10.1016/j.caeai.2026.100573). *Computers and Education: Artificial Intelligence*. --- ## [Rethinking Higher Education: From Fixed Curricula to Learnity Graphs](https://edtechdev.github.io/aied/articles/learnity-graphs-lifelong-learning-framework-2026/) > **Synthesis:** Szekely, Gal-Ezer & Harel (2026) argue that AI-mediated knowledge access warrants rethinking fixed higher-education curricula, proposing "learnity graphs" — structured representations of learning as interconnected units of knowledge, skills, experience, and artifacts — as a lifelong-learning framework that integrates academic, professional, and personal learning. ## Key Findings 1. **The value proposition of universities persists.** The authors emphasize that universities remain essential for foundational knowledge, theoretical depth, and conceptual grounding; the goal is not to replace academic study but to extend it. 2. **AI mediation changes the calculus.** In an era where knowledge is increasingly AI-mediated and accessible, the fixed-sequential-curriculum model is open to reconsideration, shifting emphasis toward creativity, interdisciplinary integration, hands-on experience, and long-term development. 3. **Learnity graphs as the core construct.** The paper introduces *learnity graphs*, a structured representation of learning as interconnected units of knowledge, skills, experience, and actual artifacts, together with a method for presenting and leveraging the representation. 4. **Lifelong integration across domains.** The framework integrates academic, professional, and personal learning, positioning learning as a continuous, graph-structured activity rather than a bounded curricular sequence. ## Implications This conceptual contribution speaks directly to debates about [[curriculum-design]] in an era of [[generative-ai]]. By framing learning as a graph of interconnected units rather than a fixed sequence, it aligns with [[personalized-learning]] and [[student-modeling]] traditions while proposing a concrete representational mechanism. The "learnity graph" resonates with [[knowledge-graph]] approaches and with the broader movement toward [[lifelong-learning]] in response to AI-driven workforce change. For [[higher-ed]] institutions, the framework is a provocation to move beyond static degree pathways toward adaptive, cross-domain learning environments. It connects conceptually to [[instructional-design]] discussions about [[transfer-of-learning]] and [[self-regulated-learning]], since learners must actively navigate and maintain their own learnity graphs. The proposal also has an implicit [[equity]] dimension: graph-based, modular learning could either democratize access to personalized pathways or entrench fragmentation if not supported by sound pedagogy and institutional infrastructure. The paper is best read alongside work on [[educational-policy-ai]] and [[ai-governance-education]] that addresses how such frameworks are governed and resourced. ## Connected Concepts - [[curriculum-design]] - [[educational-policy-ai]] - [[generative-ai]] - [[higher-ed]] - [[instructional-design]] - [[knowledge-graph]] - [[lifelong-learning]] - [[personalized-learning]] - [[self-regulated-learning]] - [[student-modeling]] - [[transfer-of-learning]] ## Connected Articles - [[ai-uk-higher-education-policy-2026]] — AI in UK higher education policy - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review - [[pchl-he-framework-genai-content-creation-2026]] — PCHL-HE framework - [[xie-hillm-cd-2026]] — HILLM curriculum design ## Citation Szekely, S., Gal-Ezer, J., & Harel, D. (2026). [*Rethinking Higher Education: From Fixed Curricula to Learnity Graphs*](https://arxiv.org/abs/2608.08543). arXiv:2608.08543 (cs.CY). --- ## [LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning](https://edtechdev.github.io/aied/articles/learnmate2-llm-adaptive-learning/) > LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning **Wang, Lee, & Mutlu (2026)** — University of Wisconsin-Madison. CHI-related publication. LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning ## Summary LearnMate^2 addresses the **personalization gap** in online learning: while online platforms offer widespread access, they lack the personalized guidance that characterizes effective [[intelligent-tutoring]] systems. The system provides three core capabilities: 1. **Personalized study plans** — tailored learning paths based on individual learner profiles 2. **Real-time contextual assistance** — in-context support during learning sessions via [[conversational-ai-tutors-framework]] 3. **Adaptive learning activities** — dynamic tasks responding to learner performance, implementing [[adaptive-learning]] principles **Iterative development and evaluation:** - **Preliminary study (n=24)** assessed effectiveness, informed system refinements - **Comparative evaluation (n=16)** against a state-of-the-art online learning platform plus an LLM - **Results:** LearnMate^2 improved both learning outcomes and user experience vs. the baseline The study demonstrates that LLM-powered [[personalized-learning]] can bridge the guidance gap in open online education. This connects to the broader [[adaptive-learning]] literature and extends findings from [[learnmate2-llm-adaptive-learning]] prior iterations. The system's integration of study planning, real-time assistance, and adaptive activities represents a more holistic approach than single-function AI tools, aligning with the [[agentic-ai]] vision of integrated educational AI. The work also contributes to understanding how [[scaffolding]] can be implemented at scale in digital environments, complementing [[ai-metacognition-stem-review]] findings on metacognitive support tools. ## Connected Concepts - [[adaptive-learning]] - [[agentic-ai]] - [[personalized-learning]] - [[scaffolding]] - [[adaptive-learning]] - [[agentic-ai]] - [[generative-ai]] - [[higher-ed]] - [[llm]] - [[metacognition]] ## Connected Articles - [[ai-metacognition-stem-review]] — AI Tools Scaffolding Metacognition in STEM - [[codify-socratic-tutoring-programming]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalab... - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... ## Citation Wang, X. J., Lee, C. P., & Mutlu, B. (2026). [LearnMate^2: Design and evaluation of an LLM-powered personalized and adaptive support system for online learning](https://arxiv.org/abs/2605.06257). arXiv:2605.06257. https://doi.org/10.1145/3800645.3812972 --- ## [LearnOpt: Recovering the Latent Cognitive Structure of Standardized Examinations via Knowledge Graphs and Constrained Optimization](https://edtechdev.github.io/aied/articles/learnopt-exam-cognitive-structure/) Standardized examinations are typically treated as uniform syllabus coverage problems. LearnOpt recovers stable latent cognitive structures diverging systematically from official syllabi, using LLM-tagged questions and constrained optimization. Applied to 9 years of NEET questions (n=1,496) and JEE Advanced questions. Finds NEET latent skill distribution is stable within syllabus regimes (KL 0.004-0.032) but shifts significantly after syllabus rationalization (KL=0.040, p=0.0005). JEE Advanced is dominated by multi-concept integration (80.9%). Formulates study planning as a knapsack-variant optimization with Bayesian Knowledge Tracing. ## Key Findings - Standardized exams have recoverable latent cognitive structures; multi-concept integration dominates higher-tier exams (JEE: 80.9%). - LearnOpt builds an **exam knowledge graph from LLM-tagged questions** and extracts a **five-category latent skill distribution**, applied to nine years of NEET questions (2016–2024, n=1,496). - NEET's latent skill distribution is **stable within a syllabus regime** (consecutive-year KL divergence 0.004–0.032 for 2016–2021, non-significant under permutation testing) but **shifts significantly with NCERT's 2023 syllabus rationalization** (pooling 2016–2021 vs 2023–2024 gives KL=0.040, p=0.0005), with Elimination/Negation questions rising from roughly 20–29% to 31–35%. - Within either regime, **subject predicts skill profile more strongly than year**; exam tier shapes latent cognitive structure more than subject, which shapes it more than time within a regime. - Study planning is formulated as a **knapsack-variant optimization over prerequisite-aware subgraphs with Bayesian Knowledge Tracing**; evaluation with one real and two synthetic mastery profiles shows the skill-weighted objective produces a modest but real reordering of recommended topics over a mastery-conditioned frequency baseline. - The JEE Advanced profile is dominated by **Multi-concept Integration (80.9% vs. 33.3% for NEET)**, with a JEE-vs-NEET divergence (KL=0.505) exceeding NEET's largest cross-subject divergence. ## Study Design & Method LearnOpt treats standardized examinations not as uniform syllabus coverage problems but as **adversarial systems with stable latent cognitive structures** that diverge systematically from official syllabi. The pipeline tags historical questions with LLMs, assembles a knowledge graph, extracts skill distributions, and optimizes time-bounded study plans. The JEE analysis uses the single-correct MCQ subset of JEEBench (110 of 515 problems, 2016–2023) because only that response type is structurally comparable to NEET's single-correct format. Code, knowledge graph, and annotated dataset are released publicly. ## Implications for AI in Education The piecewise-stable latent structure means exam preparation can be modeled as **skill acquisition over an inferred structure** rather than uniform syllabus coverage, supporting personalized study planning for high-stakes exams such as NEET and JEE. The LLM-tagging pipeline also shows a practical use of generative models for exam analytics, while the optimization framework connects exam data to [[knowledge-tracing]]-style mastery estimation and adaptive study recommendations. ## Connected Concepts - [[knowledge-tracing]] - [[affective-computing]] - [[human-in-the-loop-ai]] - [[pedagogical-agent]] - [[self-regulated-learning]] - [[personalized-learning]] - [[adaptive-learning]] - [[formative-assessment]] ## Connected Articles - [[history-aware-student-simulation]] — Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[physiological-signals-exam-outcomes-ml]] — Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning - [[ai-partner-science-epistemic-vigilance]] — AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation - [[dynamic-skill-matching-capstone-teams]] — Improving Capstone Team Outcomes through Dynamic Skill Matching and Preference Alignment - [[edtech-design-time-generative-ui]] — The Missing Layer: Why EdTech Needs Design-Time Generative UI, Not Just Runtime Personalization ## Citation Joy Bose, Om Thomas (2026). [LearnOpt: Recovering the Latent Cognitive Structure of Standardized Examinations via Knowledge Graphs and Constrained Optimization](https://arxiv.org/abs/2606.15349). arXiv:2606.15349. arXiv preprint. --- ## [LecturaAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching](https://edtechdev.github.io/aied/articles/lecturaagents-multi-agent-teaching/) > **Jaward Sesay, Yue Yu, Siwei Dong, Yemin Shi, Guangyao Chen, Borje F. Karlsson** (2026). arXiv cs.CL **Jaward Sesay, Yue Yu, Siwei Dong, Yemin Shi, Guangyao Chen, Borje F. Karlsson** (2026). arXiv cs.CL ## Key Findings - Hierarchical multi-agent framework (ProfessorAgent + subordinate agents) enables end-to-end adaptive embodied teaching. - TASA algorithm aligns teaching actions with learner profiles. - Outperforms baselines on lecture quality, embodiment, assessment, and personalization. - Given a lecture prompt or learning materials plus a learner profile, a **ProfessorAgent leads a collaborative team of specialized agents** through research, planning, design, evaluation, and embodied delivery of lecture and study content that adapts to the individual learner. - The framework provides students with access to **real-time adaptive, personalized teaching and study sessions**, addressing a gap in prior educational agent frameworks that focused mainly on lecture content automation and simulation. - Evaluation across **high school, undergraduate, and graduate-level courses** using sample-specific rubric-based analysis, with generated lecture materials and teaching actions assessed and validated by **expert educators**, shows consistent gains over existing approaches. ## Architecture & Method At its core, LecturaAgents mirrors a professor–student relationship: a ProfessorAgent orchestrates specialized agents across the full teaching pipeline, from researching content to planning, designing, evaluating, and delivering instruction. Two mechanisms distinguish it from prior work. First, an **adaptive embodied teaching mechanism** lets the ProfessorAgent execute visible, pedagogically motivated teaching actions — handwriting, highlighting, underlining — over content in a teaching environment while speaking. Second, the **Teaching Action-Speech Alignment (TASA) algorithm** employs salience-based heuristics and temporal semantic segmentation to generate coherent teaching action sequences aligned with learner profiles. The multimodal, embodied dimension is what positions the framework as pedagogically grounded [[personalized-learning]] rather than mere content generation. ## Relevance to AI in Education This paper contributes directly to understanding how AI systems interact with learners in authentic educational settings. It introduces hierarchical multi-agent architectures for embodied, personalized teaching that adapts lecture content and actions to individual learners, positioning LecturaAgents as a pedagogically well-grounded framework for personalized learning at scale. The expert-validated rubric evaluation across educational levels makes the framework relevant to [[intelligent-tutoring]] and [[pedagogical-llm-training]] research on embodied and multimodal instruction. ## Connected Concepts - [[personalized-learning]] - [[pedagogical-llm-training]] - [[pedagogical-agent]] - [[affective-computing]] - [[lifelong-learning]] - [[socratic-method]] - [[affective-tutoring]] - [[teacher-ai-competency]] ## Connected Articles - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[cognitive-agent-compilation]] — Cognitive Agent Compilation for Explicit Problem Solver Modeling - [[llm-student-modeling-memory]] — LLM Student Modeling and Long-Term Memory Architecture ## Citation Jaward Sesay, Yue Yu, Siwei Dong, Yemin Shi, Guangyao Chen, Borje F. Karlsson (2026). [LecturaAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching](https://arxiv.org/abs/2606.16428). arXiv:2606.16428. arXiv cs.CL. --- ## [Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work](https://edtechdev.github.io/aied/articles/less-deliberate-teams-llm/) > **Sehrish Basir Nizamani, Zannah Ziew, Saad Nizamani, Khyati Goyal** — ACM SIGCSE Virtual 2026, submitted 29 Jun 2026 **Sehrish Basir Nizamani, Zannah Ziew, Saad Nizamani, Khyati Goyal** — ACM SIGCSE Virtual 2026, submitted 29 Jun 2026 Semester-long study of 96 CS students shows LLM usage dropped 42.7 percentage points when students moved from individual to team work; team context reduces deliberate LLM engagement. ## Key Contributions - Semester-long study of 96 CS students shows LLM usage dropped 42.7 percentage points when students moved from individual to team work; team context reduces deliberate LLM engagement. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[mixed-reality-engineering-learning]] — Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning - [[cogtax-cognitive-taxonomy]] — CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use - [[gaze-informed-ai-children]] — Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration - [[tactile-statistical-graphs-accessibility]] — Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education ## Citation Sehrish Basir Nizamani, Zannah Ziew, Saad Nizamani, Khyati Goyal (2026). [Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work](https://arxiv.org/abs/2606.30860). arXiv:2606.30860. ACM SIGCSE Virtual 2026, submitted 29 Jun 2026 --- ## [Let''s Chat: Leveraging Chatbot Outreach for Improved Course Performance](https://edtechdev.github.io/aied/articles/lets-chat-chatbot-outreach-2026/) # Let's Chat: Chatbot Outreach for Course Performance > Meyer, Page, Mata et al. (2026) ran two pre-registered RCTs at Georgia State University testing a **non-generative** academic chatbot that texted students 2–3 customized nudges per week in large-enrollment online courses. It raised the probability of earning an A or B by **4 percentage points** — driven entirely by women in Microeconomics (+7 grade points, +11 pp A/B, −10 pp DFW) — via a task-completion channel (tutoring attendance, homework completion), with no spillover to other courses. ## Design - Two RCTs (REES-registered) in GSU's Introduction to American Government and Principles of Microeconomics — large-enrollment, asynchronous online courses - Half of students got a **Mainstay chatbot**: 2–3 scheduled text messages/week (due dates, nudges for missing assignments, encouragement), personalized ("Hi FIRSTNAME") and targeted (differentiated by missing-work status); 24/7 AI responses from a curated content knowledge base, TA fallback for unanswered questions - Note: the bot is **non-generative AI** — rule/AI-answered from a pre-programmed knowledge base — which is precisely why it's relevant: minimal hallucination risk at scale ## Results - **+4 pp likelihood of earning an A or B**, similar across both courses - **Microeconomics women: +7 grade points** vs control women; **+11 pp A/B**; **10 pp less likely to DFW**. No treatment effects for men - Mechanisms: treated students **more likely to attend university tutoring**; suggestive homework-completion gains in Micro (weekly-assignment course); no assignment effects in Government - **No spillover**: no effects on other courses that term or on next-semester enrollment/performance — course-specific gains without (yet) developed study habits - **82% of surveyed students recommended continuing the bot** and expanding it ## Why it matters - Large-enrollment online courses have documented negative outcomes; low-touch, scalable outreach is one of the few levers that work - Contrasts with other low-touch outreach null results (Oreopoulos & Petronijevic 2019) — **timing and customization matter**; nudges work best on ongoing, dynamic tasks (weekly assignments) rather than one-shot inputs - Shows the channel matters: chatbot outreach moved students to *human* tutoring — a complementarity rather than substitution story - A useful counterpoint to [[genai-can-harm-teaching-rct-2026]]: this is student-facing, non-generative, low-stakes outreach, where AI assistance had positive effects — the harm findings are about generative teacher-facing tools used for delegation ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[student-experience]] - [[llm]] - [[rag]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[genai-availability-grades-satisfaction]] — Generative AI Availability, Grades, and Student Satisfaction at a Large University - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams ## Citation Meyer, K. E., Page, L. C., Mata, C., Smith, E., Walsh, B. T., Fifield, C. L., Tyson, M., Eremionkhale, A. E., Evans, M., Frost, S., & Jung, E. E. (2026). [Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance. NBER Working Paper No. 35397. NBER](https://www.nber.org/papers/w35397) --- ## [Leveraging complex systems: Leading for transformative change](https://edtechdev.github.io/aied/articles/leveraging-complex-systems-leading-for-transformative-change/) > **Synthesis:** Dawson and Pardo (2026) argue that generative AI (GenAI) is precipitating a systemic, paradigmatic transformation of education — not a passing fad — and that traditional bureaucratic leadership structures are ill-suited for its pace, scale, and sociotechnical nature. They introduce the **SPARK framework** (Systems, Problem, Analysis, Research, and Knowledge brokerage), a pragmatic model that operationalises Complexity Leadership Theory (CLT) to help educational leaders navigate the tension between institutional stability and systemic innovation, translating GenAI-enhanced pedagogy from isolated pilots into scalable, institutionally embedded practices. ## Key Findings - Education is understood as a **complex adaptive system** undergoing simultaneous epistemological and structural instability; GenAI destabilises foundational assumptions about knowledge production, learner agency, assessment validity, and the role of educators as epistemic authorities. - The paper layers three theoretical lenses: **Kuhn's paradigm shifts** (why education faces epistemic rupture), **Carlota Perez's techno-economic transitions** (how systemic changes unfold over time, with institutional lag creating turbulence), and **Uhl-Bien's Complexity Leadership Theory** (how leaders navigate instability by balancing bureaucratic stability with emergent innovation). - **CLT** distinguishes three leadership functions — *administrative* (stability, efficiency), *adaptive* (fostering innovation in response to complexity), and *enabling* (creating conditions for the other two to interact productively in "adaptive space"). The critical leadership task is judging when to reinforce administrative stability and when to enable adaptive space, maintaining "productive disequilibrium." - The **SPARK framework** comprises five interdependent components: **Systems** (mapping formal/informal structures, power dynamics, and leverage points), **Problem** (diagnosing adaptive challenges rather than surface technical problems), **Analysis** (data collection and recursive sense-making to surface hidden patterns), **Research** (integrating empirical/theoretical knowledge as an evidence base and feedback mechanism), and **Knowledge Brokerage** (activating and connecting networks of influence to scale innovation from pilots to systemic change). - Drawing on Centola and Macy's "complex contagions," the paper argues that high-risk innovations like GenAI require **multiple, reinforcing exposures from trusted social networks** (faculty learning communities, cross-functional teams, student-led initiatives) to be adopted — change diffuses through trust-based, socially embedded networks, not linear knowledge flow. ## Study Design & Method This is a theoretical/conceptual paper. Drawing on Kuhn's theory of paradigm shifts, Perez's techno-economic framework, Uhl-Bien's Complexity Leadership Theory, and related work (Relational Leadership Theory, adaptive leadership, Meadows' leverage points, Centola & Macy's complex contagions), the authors develop the SPARK framework as a pragmatic operationalisation of CLT for the AI era. Each SPARK component is mapped onto the CLT leadership functions (e.g., Systems→administrative, Problem→adaptive, Knowledge Brokerage→enabling). The paper is grounded in prior empirical work on learning analytics adoption (Colvin et al., 2015; Dawson et al., 2018) demonstrating that technical success alone is insufficient for institutional change. No primary data were collected. ## Implications for AI in Education The paper shifts attention from technical implementation of GenAI to the **leadership and systemic conditions** required for sustainable, equitable transformation. For institutional leaders, SPARK provides a practical toolkit to map complex systems, reframe institutional challenges, mobilise data and research, and broker knowledge across actor networks. It addresses pressing leadership challenges in the GenAI era — assessment integrity, personalised learning, and the ethics of human–machine collaboration — arguing these require collective sense-making, cross-disciplinary dialogue, and distributed agency across staff and students rather than technical fixes. It cautions that algorithmic personalisation risks narrowing educational purposes and displacing the relational and collective dimensions of learning, and that academic-integrity concerns demand pedagogical redesign (rethinking what counts as evidence of learning) alongside technical detection. It connects to [[higher-ed]], [[learning-analytics]], [[generative-ai]], and institutional-change research. ## Limitations As a conceptual paper, SPARK is a proposed framework that has not been empirically validated; the authors present it as a practice-oriented toolkit rather than a tested model. It draws on examples from prior learning-analytics adoption research (mainly Australian higher education) rather than new data on GenAI adoption specifically. The framework's abstractions (complex contagions, leverage points, adaptive space) require contextual translation by leaders, and the paper does not provide detailed case studies of SPARK in use. The focus is on higher education leadership; applicability to other sectors (K-12, vocational) is not detailed. ## Connected Concepts - [[higher-ed]] - [[generative-ai]] - [[learning-analytics]] - [[personalized-learning]] ## Connected Articles - [[institutional-change-framework-ai]] — Institutional Change Framework for AI - [[ai-in-the-wild-college]] — AI in the Wild: College Contexts - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[state-policy-teacher-ai]] — State Policy and Teacher AI - [[stanford-evidence-base-ai-k12-2026]] — The Stanford Evidence Base for AI in K-12 - [[genai-higher-education-systematic-review-2026]] — GenAI in Higher Education: A Systematic Review ## Citation Dawson, S., & Pardo, A. (2026). [*Leveraging complex systems: Leading for transformative change*](https://doi.org/10.1016/j.caeai.2026.100563). *Computers and Education: Artificial Intelligence*. --- ## [A systematic review of generative AI in education: Empirical insights from a human–AI interaction perspective](https://edtechdev.github.io/aied/articles/liang-genai-systematic-review-human-ai-2026/) > **Synthesis:** Liang, Yang, Sha, Gašević, Yan & Chen (2026) systematically review 56 empirical studies on GenAI in education through the AIED-HCD framework, analyzing three human–AI interaction modes along dimensions of human control and AI automation. They find that practice remains cautious toward high-AI-automation modes, but a high-control + high-automation mode is emerging as a trend — suggesting the future is not AI replacing humans but calibrated human–AI complementarity. This BJET review synthesizes 56 empirical studies on GenAI in education, uniquely applying the AIED-HCD framework which conceptualizes human–AI interaction along two dimensions: human control and AI automation. Three interaction modes emerge: (1) low AI automation + high human control (teacher-led), (2) balanced, and (3) high automation + high human control (emerging trend). A sensitivity analysis validates the robustness of findings across modes. The review identifies that while practice remains cautious toward high-automation modes, the simultaneous presence of high human control with high AI automation represents a promising direction. - 56 empirical studies systematically reviewed through AIED-HCD human–AI interaction framework - Three interaction modes identified along human control × AI automation dimensions - High-control + high-automation mode emerging as promising direction — not AI replacement but complementarity - Most current practice remains in lower-automation modes with strong teacher/learner oversight - Sensitivity analysis confirms findings robust across interaction modes ## Connected Concepts - [[prompt-engineering]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[curriculum-design]] - [[plagiarism-detection]] - [[student-experience]] - [[administrator]] - [[equity-in-ai-education]] ## Connected Articles - [[generative-ai-enhanced-learning-experiences-for-computational-thinking-a-systema]] — Generative AI-enhanced learning experiences for computational thinking: A systematic scoping review and design guidelines - [[raza-farooq-aied-review-2020-2025]] — Review of Artificial Intelligence in Education from 2020 to 2025 - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] — Learning-to-learn in the age of generative AI: A scoping review and conceptual framework - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[quest-ai-inquiry-preservice-teachers]] — Effects of an AI-supported inquiry model on AI literacy and authentic performance: A quasi-experimental study with preservice teachers ## Citation Liang, Z., Yang, K., Sha, L., Gašević, D., Yan, L., & Chen, G. (2026). [A systematic review of generative AI in education: Empirical insights from a human–AI interaction perspective](https://doi.org/10.1111/bjet.70055). --- ## [Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education](https://edtechdev.github.io/aied/articles/llm-automated-assessment-student-self-explanations/) This paper presents a rigorous empirical comparison between [[llm|LLM]]-based and semantic similarity methods for [[automated-grading|automated assessment]] of student self-explanations in programming education. The task is framed as binary classification — determining whether a student's explanation of a worked-example step is correct or incorrect. Worked examples — step-by-step problem solutions — are a well-established [[scaffolding]] technique, and their effectiveness increases when students are prompted to self-explain each step. However, manually assessing these [[self-regulated-learning|self-explanations]] doesn't scale. The prevailing approach has been to compare student responses to reference explanations using semantic similarity metrics, but recent advances in large language models raise the question of whether LLM-based scoring now outperforms these traditional methods. The authors address a critical gap: high-quality, domain-specific datasets with balanced class distributions for automated scoring tasks. Their contribution is both methodological (a rigorous comparison framework) and empirical (which approach works better, and under what conditions). - **Binary classification framing**: Self-explanations scored as correct or incorrect, a practical framing for real-world deployment in [[intelligent-tutoring|intelligent tutoring systems]] - **Dataset contribution**: Domain-specific labeled data for programming self-explanations with balanced classes - **Method comparison**: LLM-based scoring versus semantic similarity methods, with systematic evaluation - **Practical implications**: Guidance for building [[feedback-loop|automated feedback]] systems in [[programming-its|programming education]] ## Connection to Wiki This work extends the [[automated-grading]] landscape by addressing a specific gap: assessment of open-ended self-explanations rather than final answers or code submissions. It complements research on [[automatic-short-answer-grading]] and [[structured-llm-feedback-programming]] by focusing on the formative, metacognitive dimension of student learning rather than summative evaluation. ## Connected Concepts - [[llm]] - [[automated-grading]] - [[scaffolding]] - [[self-regulated-learning]] - [[intelligent-tutoring]] - [[feedback-loop]] ## Connected Articles - [[programming-its]] - [[automatic-short-answer-grading]] - [[structured-llm-feedback-programming]] ## Citation Lekshmi-Narayanan, A.-B., Hassany, M., & Brusilovsky, P. (2026). [Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education](https://arxiv.org/abs/2605.21614). arXiv:2605.21614. --- ## [Are LLM-based Chatbots Good Enough to Support Computer Science Students in Multiple-Choice Exercises?](https://edtechdev.github.io/aied/articles/llm-chatbots-cs-multiple-choice/) Investigates LLM chatbots' performance on 70 MCQs for a university CS lecture on interactive visual data analysis, comparing with student performance. GPT-4o and GPT-5 significantly outperformed smaller models. A user study in two courses showed that presenting ChatGPT answers with explanations did NOT generally improve student performance. ## Key Findings - The authors developed **70 multiple-choice questions (MCQs)** for a university lecture on interactive visual data analysis and evaluated several LLM-based chatbots using **different prompt designs**. - **GPT-4o and GPT-5 achieved the best results**, significantly outperforming smaller models on the MCQ set. - Chatbot performance was compared with **students' performance** on the same questions, situating model accuracy relative to learner capability. - A **user study in two lectures** (interactive visual data analysis and computer vision) investigated how chatbot-generated answers and explanations affect students' performance. - The user study found that **presenting ChatGPT answers together with an explanation does not improve students' performance in general** — a counterintuitive result for chatbot-assisted learning. ## Study Design & Method The evaluation proceeded in two phases. First, a technical benchmark: multiple LLM-based chatbots solved the 70 MCQs under different prompting strategies, with results compared against students' own performance to calibrate what "good enough" means. Second, an educational user study: students in two university CS courses were given chatbot answers with explanations, and their performance was measured against conditions without such support. This two-part design separates raw model competence from actual learning impact. ## Implications for AI in Education The headline implication is that **model accuracy does not translate automatically into student learning**: even when chatbots answer correctly, exposing students to answers plus explanations failed to improve their MCQ performance. For CS education, this cautions against treating chatbot outputs as ready-made study aids; the value of LLM support likely depends on how it is integrated into exercises and feedback. The large gap between frontier and smaller models also matters for tool selection in [[higher-ed]] and [[cs-education]] contexts, as does the finding that MCQ-style support may need to be redesigned to produce measurable gains. ## Connected Concepts - [[cs-education]] - [[higher-ed]] - [[administrator]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[affective-computing]] - [[agentic-ai]] - [[ai-tutoring]] ## Connected Articles - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[shame-guilt-ai-regulation-computing-education]] — Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education - [[evaluating-interactivity-automated-assessment-ai-generated-explorable-explanations]] — Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations ## Citation Markos Stamatakis, Omkar Gavali, Joshua Berger, Christian Wartena, Anett Hoppe, Ralph Ewerth (2026). [Are LLM-based Chatbots Good Enough to Support Computer Science Students in Multiple-Choice Exercises?](https://arxiv.org/abs/2606.15919). arXiv:2606.15919. arXiv preprint. --- ## [Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety](https://edtechdev.github.io/aied/articles/llm-children-reading-story-generation/) # Children's English Reading Story Generation via SFT of Compact LLMs ## Key Finding: Small Fine-Tuned Models Beat Large Zero-Shot Models Using an expert-designed children's reading curriculum and stories generated by GPT-4o and Llama 3.3 70B as training data, the authors fine-tuned three different 8B-parameter LLMs. **The fine-tuned 8B models outperformed zero-shot GPT-4o and Llama 3.3 70B on difficulty-related metrics** while showing negligible safety issues. ## The Controllability-over-Scale Paradigm The core philosophy is **controllability over scale**: a compact, affordable model can be fine-tuned to target specific reading levels and error patterns, something large general-purpose models cannot do out of the box. This has major implications for [[equity-in-ai-education]] — schools could run these models locally at low cost rather than depending on expensive API-based LLMs. ## Educational Deployment The generated stories were designed for use by teachers, parents, and children in classrooms and at home. The controllable difficulty enables [[personalized-learning]] at scale — matching reading materials to individual student proficiency levels. The safety guarantees address concerns in [[ai-tutor-safety-harms]]. ## Methodological Contribution Fine-tuning designs were systematically compared, with the curriculum-derived training data providing curriculum alignment that general-purpose models lack. The quantitative and qualitative evaluation framework provides a template for assessing AI-generated educational content. ## Connected Concepts - [[equity-in-ai-education]] - [[personalized-learning]] ## Connected Articles - [[ai-tutor-safety-harms]] ## Citation Shen, Q., Cao, F., Yao, M., Gilda, S., Dorr, B. J., & Leite, W. L. (2026). [Children's English reading story generation via supervised fine-tuning of compact LLMs with controllable difficulty and safety](https://arxiv.org/abs/2605.13709). Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026). arXiv:2605.13709. --- ## [Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work](https://edtechdev.github.io/aied/articles/llm-cognitive-diagnosis-handwritten-math/) > **MathCog** benchmark (3,036 teacher-annotated diagnostic verdicts, 639 handwritten responses, 18 LLMs): all models severely underperform (macro F1 < 0.5) — over-attributing evidence, overthinking minimal cues, hallucinating nonexistent evidence ([[hallucination-risk]]) — calling for evidence-aware architectures and [[human-in-the-loop-ai|teacher-in-the-loop]] designs ([[knowledge-tracing]], [[multimodal]], benchmark). # Benchmarking LLMs for Diagnosing Cognitive Skills from Handwritten Math ## Summary This paper introduces **MathCog**, a benchmark dataset of 3,036 teacher-annotated diagnostic verdicts across 639 student handwritten math responses to 110 problems. Evaluating 18 LLMs, the authors find that **all models severely underperform** (macro F1 < 0.5), with systematic failure modes: over-attributing evidential strength, overthinking minimal cues, and hallucinating nonexistent evidence. Performance degrades sharply when student evidence is vague or implicit. The study calls for evidence-aware architectures and teacher-in-the-loop designs. ## Key Contributions 1. **MathCog Benchmark** — First benchmark for cognitive skill diagnosis from handwritten math, grounded in TIMSS 2019 cognitive framework with Evident/Vague evidential strength labels 2. **Systematic LLM Evaluation** — 18 models spanning reasoning, multimodal, and text-only architectures, all showing F1 < 0.5 3. **Error Taxonomy** — Five systematic error patterns identified: evidence misidentification, rubric misinterpretation, over-inference, inconsistency, and hallucination 4. **Evidential Calibration Metrics** — Introduces OverAttr and FalseAttr to quantify models' tendency to over-claim evidential confidence ## Core Findings ### Universal Underperformance No model achieves F1 ≥ 0.5. The best performers (GPT-4o-img at 0.448, DeepSeek-R1 at 0.442) still fail on nearly half of diagnostic decisions. Accuracy (mean 0.680) is misleading due to class imbalance — most student responses provide Evident Yes evidence, inflating accuracy. ### Evidence Sensitivity Gap All models perform worse when student evidence is Vague (implicit, incomplete, or context-dependent). Multimodal models show a larger Evident-to-Vague performance drop than text-only models, suggesting visual inputs may amplify over-interpretation rather than improve evidential calibration. ### Systematic Error Patterns - **Evidence Over-Attribution (OverAttr = 0.58):** Models label 58% of vague-evidence cases as "Evident" - **Evidence False-Attribution (FalseAttr = 0.585):** 58.5% of incorrect diagnoses are accompanied by false claims of evidential confidence - **Hallucination:** Models fabricate evidence quotes not present in student handwriting - **Over-inference:** Drawing strong diagnostic conclusions from minimal or ambiguous cues ## Implications for AI in Education ### Evidence-Aware Design Current LLMs lack the ability to calibrate confidence based on evidential quality. Educational AI systems need explicit mechanisms to **detect when evidence is insufficient** and communicate uncertainty rather than fabricate confident diagnoses. This connects to broader challenges in [[ground-truth-reliability-aied]] and [[hallucination-risk]]. ### Teacher-in-the-Loop Architecture The authors advocate for [[human-in-the-loop-ai]] systems where LLM predictions support rather than replace teacher judgment. This aligns with [[teacher-role]] research on AI augmentation vs. automation, and with findings from [[over-reliance]] studies showing that uncalibrated AI outputs can degrade human diagnostic accuracy. ### Benchmarking Beyond Accuracy MathCog demonstrates that standard accuracy metrics obscure critical failure modes in educational AI. The evidential calibration metrics (OverAttr, FalseAttr) introduced here complement evaluation approaches in benchmark design and [[ai-ed-evaluation]]. The dataset's Evident/Vague distinction parallels concerns in [[cognitive-offloading]] about when learners (and AI systems) should recognize the limits of available information. ### Connection to LLM Fallacy Research This paper directly extends work on [[llm-fallacy-misattribution]] — the hallucination of evidence and over-attribution patterns documented here are a specific educational manifestation of broader LLM reasoning failures. The finding that reasoning models (DeepSeek-R1, o1-Preview) show no systematic advantage echoes questions about whether chain-of-thought reasoning actually improves diagnostic reliability. ### Handwritten Math Assessment MathCog builds on existing [[llm-handwritten-math-grading]] work but shifts focus from answer correctness to cognitive skill diagnosis — a more ambitious and educationally meaningful task. The TIMSS cognitive framework provides a theoretically grounded alternative to ad-hoc grading rubrics. ## Limitations - Korean middle-school math only; generalizability to other languages, grade levels, and subjects unknown - 3,036 verdicts across 639 responses — moderate dataset size - Only TIMSS "Knowing" and "Applying" domains covered; "Reasoning" skills excluded due to problem set characteristics - Korean-to-English machine translation may introduce artifacts - Static benchmark; does not capture iterative diagnostic processes teachers use in practice ## Connected Concepts - [[math-education]] - [[hallucination-risk]] - [[human-in-the-loop-ai]] - [[knowledge-tracing]] - [[multimodal]] - [[teacher-role]] - [[over-reliance]] - [[ai-ed-evaluation]] - [[cognitive-offloading]] ## Connected Articles - [[ground-truth-reliability-aied]] - [[llm-fallacy-misattribution]] - [[llm-handwritten-math-grading]] ## Citation Kim, Y., Jin, H., Doh, H., Kim, E., Jung, D., Kim, S., Choi, K., Son, J., & Kim, J. (2025). [*Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work*](https://arxiv.org/abs/2504.00843). arXiv:2504.00843. --- ## [Using LLMs to Detect Growth in Computational Thinking in Introductory Physics](https://edtechdev.github.io/aied/articles/llm-computational-thinking-physics-2026/) > **Synthesis:** Savage, Shanker, Michlitsch & Rebello (2026) investigate using LLMs to evaluate students' written explanations of computational physics problems at scale. Establishing a human-coded baseline grounded in CT literature, they found significant growth in Data Practices and Computational Problem-Solving Practices. The LLM successfully mirrored human evaluations for these constructs, but both human raters and the LLM struggled with more complex constructs like Systems Thinking. This work demonstrates that LLMs offer a viable, scalable method for assessing computational thinking in large-enrollment physics courses. As computation becomes more central to physics education, scalable methods to assess authentic computational thinking (CT) are critically needed. This study establishes a human-coded baseline grounded in CT literature, identifies significant pre/post growth in Data Practices and Computational Problem-Solving Practices, and demonstrates that an LLM can mirror human evaluations — scaling CT assessment across large datasets. Notably, both human raters and the LLM struggled with more complex constructs like Systems Thinking, revealing the limits of current automated approaches. - LLMs successfully mirrored human coding for Data Practices and Computational Problem-Solving Practices - Both human raters and LLMs struggled with Systems Thinking — revealing construct complexity - The approach scales CT assessment to large-enrollment physics courses where manual coding is infeasible - Submitted to Physics Education Research Conference (PERC) 2026 ## Connected Concepts - [[physics-education]] - [[computational-thinking]] - [[stem-education]] - [[automated-grading]] - [[educational-measurement]] - [[higher-ed]] ## Connected Articles - [[hashmi-socratic-physics-chatbot-2025]] - [[ai-scoring-language-bias-physics]] ## Citation Savage, S., Shanker, A., Michlitsch, G., & Rebello, N. S. (2026). [Using LLMs to Detect Growth in Computational Thinking in Introductory Physics](https://arxiv.org/abs/2608.06200). --- ## [Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review](https://edtechdev.github.io/aied/articles/llm-critical-thinking-teamwork-review/) > **Synthesis:** Can Large Language Models Foster Critical Thinking, Teamwork, and Problem-Solving Skills in Higher Education?: A Literature Review ## Key Findings - Systematic literature review following the **PRISMA 2020 protocol**, searching the **Web of Science Core Collection** for studies published **2023–2024**; of **203 studies screened, 22 articles** were included (10 from search string A, 12 from search string B), and the review is **registered in PROSPERO (CRD420251165731)**. - LLMs often produce **incomplete or incorrect responses, prompting students to question, verify, and improve** the information — creating validation-and-correction cycles that the reviewed studies link to enhanced critical thinking and mental independence. - LLMs act as **catalysts for collaboration**: supporting idea generation, organization, and peer feedback, simulating rubric-based evaluations and expert reviews, and democratizing access to knowledge — promoting more equitable collaborative learning, especially in diverse and large-class settings. - For problem-solving, LLMs help students **explore alternative solutions, incorporate interdisciplinary perspectives, and simulate authentic real-world scenarios** — e.g., clinical and ethical dilemmas in health sciences, and prototype testing, debugging, and refinement in STEM. - LLMs deliver **scalable, timely, personalized feedback** in large courses (near-instant responses reduce delay and dependence on instructor availability) and can **reduce faculty workload** by automating feedback on routine tasks such as drafts, quizzes, and structured assignments. - LLMs support **assessment reform**: generating rubrics and varied assessment items aligned with learning outcomes, and shifting assessment from factual recall toward scenario-based, competency-based evaluation. ## About the Review The review addressed four research questions (RQ1–RQ4) on how LLMs can foster critical thinking, collaborative skills, and problem-solving; bridge theory and practice; deliver personalized feedback in large courses; and support assessment development. Search criteria limited records to peer-reviewed English-language journal articles published between January 2023 and December 2024 (IC1–IC4), with exclusions for duplicates, non-journal document types, non-English language, and early access (EC1–EC4). Of 165 records entering the inclusion/exclusion stage, 117 were excluded and 48 papers were retrieved for eligibility assessment against five criteria (C1–C5): explicit teaching–learning focus, direct use of LLMs by students, pedagogical or subject-specific application in higher education, practical or empirical evidence of learning outcomes, and structured data or evaluation. The 48 eligible documents spanned **35 countries** — led by the United States (n = 9), Germany (n = 8), Australia (n = 6), and England (n = 6) — with Frontiers Media SA and MDPI each publishing seven of the analyzed documents. Methodological rigor was enforced through the PRISMA 2020 protocol, the structured eligibility criteria, and PROSPERO registration. ## Main Findings - **Critical thinking:** students learn to evaluate the accuracy, consistency, and trustworthiness of LLM-generated content; effectively framing prompts improves their understanding and analytical skills, engaging them in comparison, synthesis, and assessment. - **Collaboration:** LLMs enrich teamwork by aiding idea generation and peer feedback, simulating expert or peer roles from various fields, and lowering participation barriers in large and diverse classes. - **Theory–practice gap:** simulations, hands-on project work, and instant alternative solutions on error accelerate learning cycles and support self-management and independent learning via anytime/anywhere access. - **Feedback at scale:** LLM-supported feedback extends beyond one-way correction to become a social, iterative process, and enables students to generate practice tests and self-assessments that strengthen metacognitive skills. - **Assessment:** scenario-based and problem-focused evaluation lets students demonstrate applied skills in realistic settings (engineering, health care, business), shifting the assessment culture from knowledge reproduction to applied understanding. - **Systemic implications:** the authors argue LLMs can transform curricula, feedback, and assessment at institutional and policy levels — improving educational equity, workforce readiness, and innovation — provided institutions establish ethical, equitable, and sustainable adoption frameworks. ## Implications for AI in Education The review positions LLMs not merely as instructional tools but as **catalysts for systemic improvement** in [[higher-ed]]: embedding LLMs in curricula and assessment can move teaching beyond memorization toward reasoning, hands-on learning, and competency-based evaluation. For instructors, the evidence supports using LLM imperfections deliberately — having students verify, critique, and refine AI outputs turns the technology's limitations into [[critical-thinking]] exercises. For institutions, the review calls for policies that incorporate [[generative-ai]] literacy into curricula, prepare faculty for responsible use, and protect academic integrity. The findings on personalized, scalable feedback connect to [[assessment]] and feedback research in large-enrollment courses, while the collaborative and problem-solving benefits align with [[collaborative-learning]] frameworks. The authors caution that impact depends on faculty, managers, and policymakers adopting the tools within broader educational reform. ## Limitations The authors explicitly list four limitations: (a) the search relied on the Web of Science Core Collection without considering other databases; (b) inclusion was restricted to scientific journal publications, excluding other document types; (c) the included documents did not focus on challenges of LLM adoption in teaching and learning, such as ethical concerns or privacy; and (d) the review covered only 2023 and 2024, the first two years of the technology's emergence. No formal risk-of-bias or quality-assessment instrument was applied to the included studies beyond the PRISMA 2020 protocol and the structured eligibility criteria. ## Connected Concepts - [[higher-ed]] - [[collaborative-learning]] - [[critical-thinking]] - [[assessment]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[automated-essay-scoring]] - [[plagiarism-detection]] ## Connected Articles - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for Developing an AI-Assisted Collaborative Learning Model to Enhance Critical Thinking and Problem-Solving Skills in Higher Education - [[ai-collaborative-learning-skills-impacts]] — Artificial Intelligence and Collaborative Learning: Impacts on Creativity, Critical Thinking, and Problem-Solving - [[ai-collaborative-learning-systematic-review]] — A systematic review of AI-powered collaborative learning in higher education: Trends and outcomes from the last decade - [[genai-higher-education-systematic-review-2026]] — Generative AI in Higher Education: A Systematic Review of Opportunities, Challenges, and Pedagogical Innovations (2022–2025) - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights ## Citation Martínez-Peláez, R., Mena, L. J., Toral-Cruz, H., Ochoa-Brust, A., González Potes, A., Flores, V., Ostos, R., Ramírez Pacheco, J. C., Félix, R. A., & Félix, V. G. (2025). [*Can large language models foster critical thinking, teamwork, and problem-solving skills in higher education? A literature review*](https://doi.org/10.3390/systems13111013). --- ## [LLMs for Culturally Relevant K-12 Pedagogy](https://edtechdev.github.io/aied/articles/llm-cultural-relevance-k12/) > Explores LLMs to support K-12 teachers in designing culturally relevant pedagogy. An exploratory pilot with four K-12 teachers found the CulturAIEd tool enhanced teachers' confidence in identifying opportunities for cultural responsiveness in learning activities and in making culturally responsive modifications to existing activities. Addresses equity gaps in AI educational tools by centering culturally relevant content. ## Key Findings - **CulturAIEd** is an LLM-powered system designed to help teachers contextualize **AI literacy activities through Culturally Relevant Pedagogy (CRP)** principles, combining student demographic characteristics with rubric-driven guidance in the generative process. - Teachers struggle to implement CRP due to **time, training, and resource gaps**; the study asks whether LLMs can lower these barriers (RQ1: influence on teachers' ability to design culturally responsive AI literacy activities; RQ2: perceived strengths and opportunities of such tools). - In an **exploratory pilot with four K-12 teachers**, CulturAIEd **enhanced teachers' confidence** in identifying opportunities for cultural responsiveness in learning activities and in making culturally responsive modifications to existing activities. - Teachers **valued the streamlined integration of student demographic information and immediate actionable feedback**, which they associated with high implementation efficiency. - The study positions this as among the first efforts to empower K-12 educators to design **culturally responsive AI literacy activities with low effort using LLMs**, situating AI as a partner in CRP. ## Study Design & Method The study combines a design component — building CulturAIEd around CRP/CRT frameworks, including a CRT checklist and demographic customization layered into the LLM's generative process — with an exploratory pilot conducted in preparation for a future mixed-methods study. The pilot with four K-12 teachers examined how the tool influenced their confidence, efficiency, and pedagogical strategies in making learning activities culturally responsive. The authors situate the work against Ladson-Billings' CRP framework (academic success, cultural competence, sociopolitical consciousness) and Geneva Gay's Culturally Responsive Teaching. ## Implications for AI in Education The findings suggest LLMs can help teachers integrate **culturally relevant components into instructional practice efficiently**, directly addressing common implementation barriers of limited time, curricular rigidity, and discomfort with cultural topics. Because AI literacy is an emerging subject with few resources tailored to diverse cultural and demographic contexts, tools like CulturAIEd point toward **culturally adaptive AI curricula** rather than the generalized approaches that dominate current programs. The work connects [[culturally-relevant-pedagogy]], [[ai-literacy]], and [[k-12]] teacher support, with implications for [[teacher-role]] and [[curriculum-design]] practice. ## Connected Concepts - [[culturally-relevant-pedagogy]] - [[ai-literacy]] - [[k-12-ai-education]] - [[curriculum-design]] - [[teacher-ai-competency]] - [[teacher-role]] - [[student-experience]] - [[equity-in-ai-education]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[post-covid-ict-career-aspirations]] — Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills? - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[gaide-vibe-coding-k12-teachers]] — A Guiding Framework for K-12 Teachers in Creating AI-powered Learning Technologies through Vibe Coding - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 ## Citation Wang, J., Xiao, R., Hou, X., Li, H., Tseng, Y. J., Stamper, J., & Koedinger, K. (2025). [*LLMs to Support K-12 Teachers in Culturally Relevant Pedagogy: An AI Literacy Example*](https://arxiv.org/abs/2505.08083). arXiv:2505.08083. --- ## [LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning](https://edtechdev.github.io/aied/articles/llm-design-problems-hot-pjbl/) Introduces 'design problems' (DPs): concise, scenario-based prompts that require applying knowledge in transfer contexts, generated with LLMs to assess higher-order thinking (HOT) in project-based learning. Traditional PjBL assessments often fail to capture HOT, especially transfer; DPs target that gap. Bridges [[generative-ai]] generation with [[formative-assessment]] and [[active-learning]], linking to [[scaffolding]] of complex tasks and [[higher-ed]]/[[cs-education]] contexts. It contributes a concrete method for scaling HOT assessment and informs [[generative-ai]] used for evaluation rather than just content delivery. ## Key Findings - **Design problems (DPs)** are concise, scenario-based prompts that require applying project concepts in **new situations**, targeting higher-order thinking (HOT) that traditional PjBL assessments often fail to capture, especially in transfer contexts. - **Surveys of 31 instructors** showed that instructors value DPs for assessing HOT, but **creation effort is a barrier** to adoption. - An evaluation of **80 LLM-generated DPs** showed LLMs can produce **high-quality prompts with strong expert agreement**, effectively lowering the creation barrier. - Students **rated DPs from different LLMs similarly**, and their performance on DP tasks showed **negligible correlation with traditional project grades**, suggesting DPs capture **distinct aspects of higher-order thinking** rather than duplicating existing measures. - **Keystroke data suggested deeper cognitive engagement** through planning and revision behaviors while students worked on DP tasks. ## Study Design & Method The study triangulates three perspectives: instructor perceptions (surveys with 31 instructors), LLM generation capability (80 generated DPs evaluated for quality and expert agreement), and student experience (performance data plus keystroke logs). The negligible correlation between DP performance and traditional project grades is the key psychometric signal: it indicates the assessment captures a different construct — transfer-oriented higher-order thinking — than project artifacts alone. ## Implications for AI in Education DPs appear to be a **useful complement to traditional assessments**, particularly in situations where AI use or collaboration may undermine individual learning: because DPs demand application in novel scenarios, they are harder to complete by simply reusing project artifacts or generated code. The strong expert agreement on LLM-generated prompts makes scalable HOT assessment feasible, and the keystroke evidence connects DP work to deeper engagement. This positions LLM-generated DPs as a practical instrument for [[formative-assessment]] and for evaluating transfer in project-based computing education. ## Connected Concepts - [[generative-ai]] - [[formative-assessment]] - [[active-learning]] - [[scaffolding]] - [[higher-ed]] - [[cs-education]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation - [[mllm-scientific-visualization-literacy]] — Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework ## Citation Ahmad D. Suleiman, Daqing Hou, Maliha Noushin Raida (2026). [LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning](https://arxiv.org/abs/2607.11032). arXiv:2607.11032. arXiv preprint. --- ## [Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content](https://edtechdev.github.io/aied/articles/llm-detecting-llm-generated-content-education/) As students increasingly use [[llm]]s to draft written responses and program code, this study asks whether LLMs can reliably detect their own generated content across educational task types — programming exercises, reflective writing, and short-answer questions. Using authentic student responses alongside multiple LLM-generated variants, the authors evaluate detection under varied prompting strategies and output formats. Detection proves **highly task-dependent**: it is substantially more reliable for programming tasks and longer reflective responses, but performs poorly for short-answer questions, where LLMs frequently judge their own outputs as *more* human-like than authentic student work. Prompt framing and response verbosity strongly affect detectability in reflective writing, with minor prompt variations sharply reducing accuracy, while programming detection is comparatively robust. The results highlight both the promise and the limits of LLM self-detection for [[academic-integrity]], cautioning against standalone reliance and complementing dedicated ai-detection and [[plagiarism-detection]] work. They connect to identity-detection challenges in [[socially-fluent-ai-identity-detection]] and student-side dynamics in [[student-rationalization-ai-writing]]. ## Connected Concepts - [[llm]] - [[academic-integrity]] - [[plagiarism-detection]] ## Connected Articles - [[socially-fluent-ai-identity-detection]] - [[student-rationalization-ai-writing]] ## Citation Leinonen & Denny (2026). [Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content](https://arxiv.org/abs/2607.20446). arXiv:2607.20446. arXiv preprint (cs.CL). --- ## [From Evaluated Models to Evaluation Aids: A Multi-Evidence Study of LLM-Based Difficulty Calibration for Programming Examinations](https://edtechdev.github.io/aied/articles/llm-difficulty-calibration-programming-exams-2026/) > **Synthesis:** Yan, Xiong, Li & Chen (2026) reposition LLMs from benchmark targets to auxiliary evidence sources for interpreting programming-exam difficulty, showing that AI difficulty estimates correlate strongly with student pass rates across parallel-class finals (rho ≈ −0.87 at problem level) while explicitly bounding that these scales must not be used for individual student evaluation or automatic grade adjustment. ## Key Findings 1. **AI pass rate tracks student performance.** In a synchronous eight-problem final exam where ten models solved alongside 120 students, AI pass rate correlated positively with student pass rate (Spearman *rho* = 0.866), and a solving-based composite difficulty index correlated negatively with it (*rho* = −0.905). 2. **Strong problem-level calibration across exams.** Across 79 problems from 11 parallel-class final exams, AI overall difficulty correlated with problem-level pass rate at *rho* = −0.871 and with non-attempt rate at *rho* = 0.800; a 26-problem longitudinal data-structures sample gave −0.829 and 0.883. 3. **Boundary condition in introductory courses.** A 106-problem CS101 sample marked the limit: problem-level correlation weakened to *rho* = −0.552 and exam-level correlation across 16 exams was near zero, with cohort composition dominating exam-level outcomes. Exposure-discount and duplicate-problem perturbation tests did not change the direction of findings. 4. **Explicit use-and-abstention boundaries.** The single-reviewer design, unverifiable model identity, and review-output instability mean AI difficulty scales are suitable for problem validation, parallel-class fairness discussion, and longitudinal quality tracking — but must not drive individual student evaluation or automatic grade adjustment. ## Implications This study reframes the role of [[llm]]s in [[assessment]] from "evaluated object" to "evaluation aid," contributing a methodology that layers AI evidence with student performance, item exposure, and [[learning-analytics]] to interpret exam difficulty. It connects to the growing literature on [[psychometrically-aware-ai]] and [[confidence-aware-ai-assessment]], where model outputs are treated as one noisy signal among several rather than as ground truth. For [[cs-education]] and [[cs-education]] practice, the finding that AI difficulty correlates with student outcomes at the problem level supports using LLMs to flag mis-calibrated items across parallel sections and to track item quality longitudinally. The clear abstention guidance is the crucial guardrail: cohort composition effects in introductory courses and the fragility of single-reviewer estimates caution against high-stakes automation, aligning with [[human-in-the-loop-ai]] design principles. The work also illustrates the epistemic limits of [[item-response-theory]]-style difficulty estimation when grounded in model rather than human response data, and reinforces the need for verification and [[ai-ed-evaluation]] frameworks that keep AI in a supporting rather than deciding role. ## Connected Concepts - [[assessment]] - [[automated-assessment]] - [[cs-education]] - [[confidence-aware-ai-assessment]] - [[educational-measurement]] - [[ai-ed-evaluation]] - [[human-in-the-loop-ai]] - [[item-response-theory]] - [[learning-analytics]] - [[ai-ed-evaluation]] - [[cs-education]] - [[psychometrically-aware-ai]] ## Connected Articles - [[llm-item-difficulty-prediction]] — LLM item difficulty prediction - [[llm-psychometric-calibration-cdp]] — LLM psychometric calibration - [[agreement-not-quality-llm-coding-verification]] — Agreement not quality in coding - [[llm-chatbots-cs-multiple-choice]] — LLM chatbots for CS MCQs - [[measuring-llm-tutors-teach-vs-solve]] — Measuring LLM tutors ## Citation Yan, H., Xiong, J., Li, Y., & Chen, C. (2026). [*From Evaluated Models to Evaluation Aids: A Multi-Evidence Study of LLM-Based Difficulty Calibration for Programming Examinations*](https://arxiv.org/abs/2608.07523). arXiv:2608.07523 (cs.CY). --- ## [Exploring the Value of Diverse LLM Explanations in Introductory Programming](https://edtechdev.github.io/aied/articles/llm-diverse-explanations-programming/) Bernstein, Denny, Leinonen et al. (2026) investigate whether providing students with multiple, diverse LLM-generated explanations of code (rather than a single 'best' explanation) improves comprehension in introductory programming. Their findings show that exposure to diverse explanations significantly outperforms single-explanation conditions on measures of conceptual understanding and code comprehension. This challenges the common design assumption that AI-generated educational content should converge on a single 'correct' explanation, instead suggesting that [[llm]]-generated [[feedback-loop]] diversity supports deeper learning by exposing students to multiple perspectives. The study connects to [[scaffolding]] theory, where multiple representations support the gradual transfer of responsibility from tool to learner. It also informs [[active-learning]] pedagogy by providing a concrete implementation strategy for AI-assisted instruction. The work has implications for how [[student-experience]] of programming education can be enhanced through deliberately varied AI-generated content, relevant to [[stem-education]] course design. ## Connected Concepts - [[llm]] - [[feedback-loop]] - [[scaffolding]] - [[active-learning]] - [[student-experience]] - [[stem-education]] ## Connected Articles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[flowcode-ai-creative-coding]] — Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education - [[chat-debugging-llm-hardware-education-2026]] — WIP: Chat-Debugging: Large Language Model as a Hardware Debugging Assistant - [[llm-automated-assessment-student-self-explanations]] — Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Seth Bernstein, Paul Denny, Juho Leinonen, Kush Patel, Rayhona Nasimova, Matt Littlefield, Stephen MacNeil (2026). [Exploring the Value of Diverse LLM Explanations in Introductory Programming](https://arxiv.org/abs/2606.28882). arXiv:2606.28882. cs.HC (SIGCSE Virtual 2026). --- ## [From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions](https://edtechdev.github.io/aied/articles/llm-educational-question-cognitive-depth/) LLM-generated educational questions show varying cognitive depth; models excel at factual recall but struggle with higher-order thinking questions per Bloom's taxonomy. ## Synthesis From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions investigates llm-generated educational questions show varying cognitive depth; models excel at factual recall but struggle with higher-order thinking questions per bloom's taxonomy. This work connects to existing research on a4l-analytics-pipeline by demonstrating that Abstract:While LLMs show promise in automating educational content creation, their ability to generate questions that stimulate higher-order thinking remains understudied. This work evaluates six widely-used LLMs through a Bloom's Taxonomy lens, focusing on their capacity to transcend rote memor.... ## Connected Concepts - [[automated-essay-scoring]] - [[formative-assessment]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[human-in-the-loop-ai]] - [[cs-education]] - [[plagiarism-detection]] - [[open-source]] ## Connected Articles - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[choi-anchor-aes-prompting-2025]] — Anchor Is the Key: Toward Accessible Automated Essay Scoring with Large Language Models Through Prompting ## Citation Xiaolong Wang, Zhe Zhao, Song Lai, Chaoli Zhang, Zijie Geng, Yu Tong, Ye Wei, Qingsong Wen (2026). [From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions](https://arxiv.org/abs/2606.18257). arXiv:2606.18257. --- ## [LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles](https://edtechdev.github.io/aied/articles/llm-educational-simulation-adhd/) > Gonnermann-Müller, Haase & Leins (2026) evaluate whether **LLM-generated student personas simulating ADHD profiles** maintain stable and realistic behavioral patterns over time. This addresses a critical question for using LLMs in educational research and teacher training: can simulated learners reliably represent neurodivergent students? # LLM Educational Simulations for ADHD Student Personas ## Core Contribution Gonnermann-Müller, Haase & Leins (2026) evaluate whether **LLM-generated student personas simulating ADHD profiles** maintain stable and realistic behavioral patterns over time. This addresses a critical question for using LLMs in educational research and teacher training: can simulated learners reliably represent neurodivergent students? ## Why This Matters Using LLMs to simulate students is an emerging practice in educational research, but the **temporal stability** of these simulations — especially for neurodivergent profiles — has been underexamined. If LLM-generated personas drift or become inconsistent, they cannot serve as valid proxies for real students in: - Teacher training simulations - [[adaptive-learning]] testing - [[intelligent-tutoring]] system evaluation - [[learning-analytics]] research methodology ## Connections to Wiki This work extends the [[llm-student-modeling-memory]] discourse on how LLMs represent learners over time, but applies it to **simulation validity** rather than tutoring personalization. The focus on ADHD profiles connects to broader [[student-experience]] research and highlights gaps in [[stanford-evidence-base-ai-k12-2026]] — the Stanford SCALE review found few studies with adequate causal inference for special education populations. The simulation methodology also raises questions about [[ai-tutor-safety-harms]] — if tutoring systems are tested on simulated neurodivergent learners, do the safety assessments generalize? This echoes [[educational-vlm-evaluation]] concerns about AI systems that underperform with specific student populations. ## Open Questions - How do LLM-simulated ADHD profiles compare to [[multimodal-ai-tutoring]] systems that work with real neurodivergent students? - Can temporal stability be improved through prompt engineering or fine-tuning? - What is the ethical boundary for using simulated students in [[rct]] designs? ## Connected Concepts - [[adaptive-learning]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[student-experience]] - [[rct]] ## Connected Articles - [[llm-student-modeling-memory]] - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-safety-harms]] - [[educational-vlm-evaluation]] - [[multimodal-ai-tutoring]] ## Citation Leins, A.J.G.J.H.N., Evaluating, L.E.S., ADHD, T.S.P.S.A., Leins, J.G.J.H.N., Berlin, B.H.B., & Pokutta, T.K.S. (2026). [*LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles*](https://arxiv.org/abs/2605.06307) --- ## [To Facilitate or not to Facilitate: Human and LLM Facilitator Tendencies in Online Discussions](https://edtechdev.github.io/aied/articles/llm-facilitation-timing-online-discussions/) > **Dimitris Tsirmpas, Katerina Korre, John Pavlopoulos** — arXiv preprint (2026). ## Synthesis This study asks when (not just how) LLMs should facilitate online discussions, creating PEFK, a corpus standardizing and aggregating facilitation datasets, and running the first survey on facilitation timing with expert facilitators and LLM-as-a-judge models. Key asymmetry: humans are more cautious while LLMs are excessively eager to facilitate, although both are more certain when judging that facilitation is not needed. Corrective attempts found trained ModernBert classifiers more reliable than alternative LLM setups, though existing datasets impose a relatively low performance ceiling — a benchmark-quality finding for automated discussion facilitation. For online learning, the work informs when AI should intervene in discussion forums (MOOC-style and classroom), connecting facilitation timing to engagement and moderation research. ## Key Findings - The study defines **facilitation timing** as the decision of when to intervene in a discussion, observing that not intervening when needed can lead to topic derailment or escalation, while intervening when a discussion is going well can irritate participants. - The authors create **PEFK**, a corpus **standardizing and aggregating all relevant facilitation datasets**, enabling direct comparison across prior work. - They are the first to run a **survey on facilitation timing**, executed with expert facilitative participants and **LLM-as-a-judge models**. - **Humans are more cautious** about intervening, while **LLMs are excessively eager to facilitate**; both are more certain when judging that facilitation is **not** needed. - **Trained ModernBert classifiers perform more reliably** than alternative LLM setups for predicting facilitation needs, although **current datasets impose a relatively low performance ceiling**. ## Study Design & Method Automating facilitation has been attempted with encoder-only classifiers, and LLMs have more recently been championed as the eventual solution; however, prior work indicated LLM facilitators are too eager to intervene, rendering them unusable as autonomous agents — a finding the authors contrast with human tendencies for the first time. The study operationalizes what facilitation is, observes when humans decide to facilitate, and compares those decisions with LLM decisions. Corrective alternatives (different LLM setups) and classifier training on established datasets are then evaluated against the aggregated PEFK corpus. ## Implications for AI in Education For online learning environments — MOOC-style forums and classroom discussion spaces — the work clarifies that **the timing of AI intervention is as important as its content**. LLMs' excessive eagerness to facilitate suggests autonomous moderation agents need calibration toward human caution, and the modest ceiling of existing datasets indicates that better annotation infrastructure is needed before facilitation timing can be reliably automated. The findings connect facilitation timing to [[collaborative-learning]] and to [[human-in-the-loop-ai]] design in educational discourse platforms. ## Connected Concepts - [[human-in-the-loop-ai]] - [[collaborative-learning]] - [[administrator]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[affective-computing]] - [[prompt-engineering]] ## Connected Articles - [[spritz-ai-disciplinary-mediation-student-teams-2026]] — Exploring AI-Supported Disciplinary Mediation in Student Project Teams' Text-Based Communication - [[icap-cognitive-engagement-llm-agents]] — Measuring Cognitive Engagement in Collaborative Discourse with an Extended ICAP Framework: Comparing Human Annotation, In-Context Learning, and Reflective LLM Agents - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use - [[interactive-learning-dashboards-engagement]] — Interactive learning dashboards: rethinking learning visualisations as engagement tools - [[physiological-signals-exam-outcomes-ml]] — Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning - [[less-deliberate-teams-llm]] — Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work ## Citation Tsirmpas, D., Korre, K., & Pavlopoulos, J. (2026). [To facilitate or not to facilitate: Human and LLM facilitator tendencies in online discussions](https://arxiv.org/abs/2607.28643). arXiv:2607.28643. --- ## [The LLM Fallacy and Misattribution of Competence](https://edtechdev.github.io/aied/articles/llm-fallacy-misattribution/) > Three system properties enable the fallacy via two cognitive mediators: > The LLM fallacy is a **cognitive attribution error** in which users misinterpret LLM-assisted outputs as evidence of their own independent competence, producing a systematic divergence between **perceived and actual capability** (∆C). It is independent of output correctness — it persists whether the AI is accurate or hallucinating.^[[kim-llm-fallacy-misattribution-2026]] ## Distinctions from Related Phenomena | Concept | Focus | LLM Fallacy | |---|---|---| | **Hallucination** | System produces incorrect information | How the user *interprets* any output as self-generated competence | | **Automation bias** | Over-reliance on system during decisions | Self-perception of *personal* capability derived from outputs | | **Cognitive offloading** | Delegating mental effort to tools | Integration of outputs into user's *identity and self-evaluation* | | **Dunning-Kruger** | Internal miscalibration of skill | Specifically AI-mediated; requires tool interaction to emerge | ## Mechanisms Three system properties enable the fallacy via two cognitive mediators: **System Properties:** 1. **Opacity** — Users cannot trace how the model constructed the response; division of labor is invisible 2. **Fluency** — Polished, coherent output acts as a metacognitive cue for competence; users infer skill from surface ease rather than generative process 3. **Interactional immediacy** — Rapid response cycles bias toward fast, intuitive judgments over reflective evaluation **Cognitive Mediators:** - **Attribution ambiguity** — In iterative interactions, the boundary between user contribution and system generation becomes impossible to delineate; authorship is inferred from outcomes - **Cognitive outsourcing** — As the system assumes more workload, users engage less with underlying reasoning, weakening self-assessment accuracy > "Capability divergence (∆C) emerges from the interaction of system-level properties (opacity, fluency, immediacy), mediated by attribution ambiguity and cognitive outsourcing."^[[kim-llm-fallacy-misattribution-2026]] ## Manifestations in Education | Domain | Educational Example | |---|---| | **Computational** | Student produces working code via Copilot but cannot explain logic, debug independently, or adapt to new requirements | | **Linguistic** | Student generates fluent essay in a second language but cannot produce comparable prose unassisted | | **Analytical** | Student presents structured step-by-step math solution but cannot replicate reasoning when AI is unavailable | | **Creative / Epistemic** | Student reads AI summary of a topic and equates *access to information* with *conceptual mastery* (illusion of explanatory depth) | | **Professional signaling** | Resumes, portfolios, and interview answers reflect ability to prompt LLMs rather than independently acquired expertise | ## Relationship to Existing Wiki Concepts - [[metacognition]] — The LLM fallacy is a metacognitive calibration failure: students cannot accurately monitor their own understanding because fluent AI output creates false fluency signals - [[transfer-of-learning]] — Misattribution undermines transfer because students believe they have mastered material they have merely prompted; the gap between perceived and actual capability manifests as transfer failure - [[ai-tutor-safety-harms]] — SafeTutors' Cognitive (fluency illusion) and Ethical-Epistemic (misrepresentation) dimensions are tutoring-specific expressions of the fallacy - [[self-regulated-learning]] — Attribution ambiguity disrupts the self-evaluation phase of SRL, preventing accurate causal attribution and adaptation ## Implications for Education **Assessment integrity:** Completed assignments may reflect LLM capability rather than student learning. Observable outputs are unreliable proxies when AI mediation is invisible (both to human and automated evaluators). **Metacognitive scaffolding needed:** Interventions should help users correctly attribute contribution — e.g., requiring students to explain AI-generated content in their own words before submission, or using "explain-to-a-peer" protocols. **Tool design:** Systems that surface their reasoning process (reducing opacity) or require iterative refinement by the user (reducing immediacy) may mitigate the fallacy. ## Connected Concepts - [[llm-cognitive-diagnosis-handwritten-math]] - [[metacognition]] - [[self-regulated-learning]] - [[higher-ed]] - [[k-12]] - [[llm]] - [[rag]] - [[scaffolding]] ## Connected Articles - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[transfer-of-learning]] — AI Learning Transfer - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[authentic-assessment]] — Authentic Assessment - [[brookings-ai-students-report]] — A New Direction for Students in an AI World: Prosper, Prepare, Protect - [[contextual-sycophancy-ai-literacy]] — The Hidden Cost of Contextual Sycophancy: an AI Literacy Intervention in Human-AI Collaboration - [[educational-llm-alignment]] — Educational LLM Alignment - [[educational-vlm-evaluation]] — Educational VLM Evaluation - [[genai-meta-analysis-programming-learning]] — A meta-analysis of the effect of generative AI on productivity and learning in programming - [[kim-llm-fallacy-misattribution-2026]] — LLM Fallacy Misattribution in Education - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[pedagogy-ai-mistakes]] — The Pedagogy of AI Mistakes: Fostering Higher-Order Thinking - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named ## Citation Kim, H., Yu, H., & Yi, H. (2026). [*The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows*](https://arxiv.org/abs/2604.14807). arXiv:2604.14807. --- ## [LLM-Generated Feedback in Introductory Programming: A Classroom Study](https://edtechdev.github.io/aied/articles/llm-feedback-programming-classroom/) Presents a **large-scale classroom study** (N=215 students, 6,693 submissions across 17 labs) deploying AI-generated feedback through a randomized protocol in an introductory Python programming course. Students received one of three conditions: natural language hints, AI-generated failing test cases, or no AI feedback (control). The resulting dataset, **ProgFeed**, captures fine-grained temporal learning trajectories. **Key findings:** Natural language feedback is significantly associated with higher completion rates and faster convergence to correct solutions. Test case feedback shows heterogeneous effects that depend critically on feedback validity. The form of [[ai-feedback-quality|AI-generated feedback]] matters — evaluating feedback quality, not just its presence, is essential for understanding pedagogical impact. This study provides one of the largest empirical validations of LLM-based [[feedback-loop|automated feedback]] in authentic programming classrooms, with direct implications for [[automated-grading|automated grading systems]] and [[formative-assessment|formative assessment]] design in [[stem-education|CS education]]. ## Connected Concepts - [[ai-feedback-quality]] - [[feedback-loop]] - [[automated-grading]] - [[formative-assessment]] - [[stem-education]] ## Connected Articles - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[llm-automated-assessment-student-self-explanations]] — Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education ## Citation Heickal, H., & Lan, A. (2026). A Classroom Study of LLM-Generated Feedback Intervention in Introductory Programming. [arXiv:2606.08807](https://arxiv.org/abs/2606.08807). Accepted at IRAISE 2026. --- ## [Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs](https://edtechdev.github.io/aied/articles/llm-handwritten-math-grading/) Automated grading systems have enabled scalable assessment for many response types, but handwritten mathematics remains a barrier due to the complexity of multi-step solutions. Vision-capable large language models (LLMs) offer new opportunities here, yet their reliability in authentic instructional settings remains poorly understood. This paper presents an empirical evaluation of a vision-capable LLM-based grader for handwritten mathematical work. Using a single LLM call that combines transcription and rubric-based evaluation, the system was tested on student work from two university STEM courses. The grader achieved high overall accuracy at the rubric-item level, with 87% of errors in the best model attributed to transcription failures rather than rubric misapplication. The authors categorize common error modes — image quality issues, hallucinated content, and incorrect handling of equivalent expressions — providing guidance for system design, prompt refinement, and deployment. This work extends the [[automated-grading]] pipeline to the previously unexplored domain of handwritten mathematics, complementing systems like [[lata-ferpa-compliant-local-llm-autograder]] that focus on typed LaTeX submissions. Unlike typed-response graders that achieve near-perfect rubric alignment, the vision pipeline introduces new failure modes around handwriting recognition that echo the diagnostic precision challenges identified in [[llm-tutoring-feedback-diagnosis-gap]]. The finding that the LLM can correctly apply instructor rubrics when transcription succeeds suggests that [[multimodal-ai-tutoring]] approaches can bridge the gap between visual input and pedagogical assessment, while remaining sensitive to the limitations documented in [[educational-vlm-evaluation]]. ## Connected Concepts - [[automated-grading]] ## Connected Articles - [[lata-ferpa-compliant-local-llm-autograder]] - [[llm-tutoring-feedback-diagnosis-gap]] - [[multimodal-ai-tutoring]] - [[educational-vlm-evaluation]] ## Citation Jacob Levine, Miguel Aenlle, Craig Zilles, Matthew West, Mariana Silva (2026). [Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs](https://arxiv.org/abs/2605.19043). arXiv:2605.19043. International Conference on AI in Education (AIED 2026). --- ## [A review of intervention designs of LLM Integration in Undergraduate Computer Science Education](https://edtechdev.github.io/aied/articles/llm-intervention-design-cs-review/) ## Key Findings This scoping review analyzed **13 experimental studies** on LLM integration in undergraduate [[cs-education]], examining how intervention design choices shape learning outcomes. The central finding: **LLM effectiveness depends less on the model itself than on pedagogical design**. ### Three Intervention Archetypes | Type | Studies | Results | |------|---------|---------| | **Task and Coding Assistant** | 8 | Mixed — Java code quality improved (p < 0.005), but broader academic performance showed no significant difference | | **Virtual Tutor or Peer** | 3 | **All three** showed significant improvements — semester-long integrations with scaffolded feedback consistently improved [[computational-thinking]] and performance | | **Exam and Quiz Help** | 2 | Mixed depending on implementation | ### The "Tool Frustration" Paradox A striking finding: students using [[generative-ai]] tools **without adequate [[scaffolding]] and prompt literacy training** reported significantly higher frustration than controls (p = 0.008, median frustration 14 vs. 9), even when performance was equivalent. This mirrors concerns in the [[over-reliance]] and [[critical-engagement-code-completion]] literature — LLM access without pedagogical support can actively harm the learning experience. ### Design Patterns That Work The review identifies four design elements that distinguish effective interventions: 1. **Sustained [[scaffolding]]**: Guided explanations, problem decomposition, and gradual reduction of support as competence grows — consistent with Vygotskian principles also discussed in [[instructional-design]]. 2. **Transparent interaction patterns**: Students need to understand *how* the LLM is reasoning, not just receive answers. 3. **Explicit meta-skill instruction**: [[prompt-engineering]] literacy must be taught — students cannot intuit effective prompting strategies. 4. **Assessment redesign**: Emphasize code evaluation and prompt crafting over code generation, as also recommended in [[reshaping-cs-education-genai]]. ### Language and Methodological Gaps Java interventions showed more consistent gains; Python — despite dominance in CS1 — lacks sufficient experimental isolation. The review also documents critical methodological weaknesses: inconsistent outcome operationalization, variable control group definitions, and chronic **underreporting of effect sizes and confidence intervals** — a concern that connects to broader efficacy-study design standards. ## Relevance to AI in Education This review is valuable because it shifts the conversation from "do LLMs work?" to "**what design choices make LLMs effective?**" The evidence strongly supports [[scaffolding]]-based approaches over simple tool access, reinforcing findings across the [[genai-meta-analysis-programming-learning]] literature. The "tool frustration" paradox is an important contribution — it suggests that poorly designed [[generative-ai]] integration can be worse than no integration at all. For [[higher-ed]] contexts, the review provides actionable guidance: semester-long Virtual Tutor designs with structured feedback outperform short-term coding-assistant interventions. This aligns with [[code-review-genai-cs1]] work on structured feedback and the broader [[cs-education]] push toward [[computational-thinking]] over syntax mastery. ## Connected Concepts - [[computational-thinking]] - [[generative-ai]] - [[higher-ed]] - [[ai-education]] - [[prompt-engineering]] - [[reshaping-cs-education-genai]] - [[scaffolding]] - [[llm]] ## Connected Articles - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[critical-engagement-code-completion]] — To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion Tools Using Behavioral Signals and Attentio... - [[genai-meta-analysis-programming-learning]] — A meta-analysis of the effect of generative AI on productivity and learning in programming - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education ## Citation Vissapragada, A. (2026). [*A review of intervention designs of LLM Integration in Undergraduate Computer Science Education*](https://osf.io/preprints/edarxiv/knhyz_v1). EdArXiv preprint. --- ## [Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction](https://edtechdev.github.io/aied/articles/llm-item-difficulty-prediction/) This paper introduces Epi2Diff (Episode to Difficulty), a framework that maps LLM reasoning traces into cognitively grounded episode sequences for predicting human item difficulty in [[assessment|educational assessment]]. The authors argue that difficulty should be viewed not only as a property of item text but also as an observable consequence of problem-solving burden. By analyzing reasoning traces from large reasoning models (LRMs), Epi2Diff extracts compact cognitive episodes that capture reasoning scale, effort allocation, and state transitions — enabling interpretable [[student-modeling|student modeling]] without costly human calibration. The work connects to [[knowledge-tracing-irt|knowledge tracing and IRT]] by offering a process-level view of item difficulty that complements traditional outcome-based models. It has implications for [[adaptive-learning|adaptive learning systems]], where more precise difficulty estimates enable better [[personalized-learning|personalized item selection]], and for [[learning-analytics|learning analytics]], where reasoning trace analysis can provide instructors with fine-grained diagnostic information about which cognitive steps students find challenging. This approach represents a novel intersection of LLM benchmarking and [[assessment]] design, suggesting that reasoning models can serve as cognitive proxies for human test-takers in [[k-12|K-12]] and [[higher-ed|higher education]] settings. It connects to [[transfer-of-learning|AI learning transfer]] by examining how model reasoning processes mirror human cognitive processes during problem-solving. ## Connected Concepts - [[assessment]] - [[student-modeling]] - [[adaptive-learning]] - [[personalized-learning]] - [[learning-analytics]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[knowledge-tracing-irt]] - [[transfer-of-learning]] ## Citation Chenguang Wang, Ming Li, Xinyue Zeng, Zhuochun Li, Hong Jiao (2026). [Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction](https://arxiv.org/abs/2606.28186). arXiv:2606.28186. Computation and Language (cs.CL). --- ## [Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models](https://edtechdev.github.io/aied/articles/llm-judged-helpfulness-pedagogy-signal/) Pre-registered study auditing whether general-purpose helpfulness rubrics can distinguish direct answer-giving from pedagogical guidance in LLM tutors. Uses deterministic detectors for answer leakage and next-turn independent work across three tutor models. Finds that helpfulness ratings conflate genuine pedagogical scaffolding with simply giving correct answers. ## Key Findings - Within each of three tutor bases, the authors compare **conversational and pedagogical policies** instantiated with the same underlying model, paired with one fixed weak simulated student, and audited under a **pre-registered protocol**. - On the primary base under the frozen, condition-blind judge **Claude Opus 4.8**, the policies **do not differ significantly in helpfulness but are perfectly rank-separated under the pedagogy rubric** (Cliff's |δ| = 0.10 vs. 1.0). - Across two judges (Opus and the prospectively specified **GPT-5.6 Sol**), **pedagogy contrasts retain their direction** where detected, whereas the **helpfulness ordering is judge-contingent**, reversing between judges on two of three bases. - In an Opus-only ablation, seven primary-base policies **span 2.3 points in mean judged pedagogy within a 0.25-point band of mean judged helpfulness**, showing helpfulness collapses nearly all policy variance. - **Answer-revealing turns are followed by less independent student work on every base**, a result that is judge-invariant by construction and validated by deterministic detectors of answer leakage and next-turn independent work. ## Study Design & Method The audit uses **deterministic process measures** — detectors for answer leakage and next-turn independent work — alongside LLM judges, over **1,179 confirmatory answer-phase tutor turns**. Each session follows a fixed five-phase protocol: six training problems, then immediate, interference, and delayed probes, plus transfer probes; within a training problem the tutor and student alternate for a fixed number of turns. Because the two policies share the same underlying model and student, any difference is attributable to the policy itself, making the design a controlled test of whether helpfulness rubrics carry pedagogical signal. ## Implications for AI in Education The central conclusion is that **general-purpose helpfulness is not a reliable pedagogy signal** in this controlled setting: a rubric tuned to "helpful" answers cannot distinguish a tutor that scaffolds from one that leaks the answer. Tutor evaluation should therefore **pair pedagogy-targeted rubrics with deterministic process measures** such as answer leakage and next-turn independent work. For [[ai-tutoring]] and [[benchmark]] design, this argues against relying on preference-based helpfulness judgments and toward measurement of student agency and independent work. ## Connected Concepts - [[ai-tutoring]] - [[pedagogical-llm-training]] - [[affective-tutoring]] - [[automated-question-generation]] - [[benchmark]] - [[personalized-learning]] - [[knowledge-tracing]] - [[socratic-ai-dialogue]] ## Connected Articles - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[rethinking-scaffolding-llm-tutors]] — Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[structrag-diagram-reasoning-ai-tutoring]] — Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents ## Citation Shuyi Fan, Boyuan Deng, Mengyu Xu, Jiale Liu, Hongyang Zhang (2026). [Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models](https://arxiv.org/abs/2607.28128). arXiv:2607.28128. cs.CL, cs.AI, cs.CY. --- ## [The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty](https://edtechdev.github.io/aied/articles/llm-misconception-difficulty-easy-trap/) LLMs systematically underestimate the difficulty of misconception-driven items ('The Easy Trap'). While LLM ratings show moderate rank correlation with empirical student difficulty (rho=0.52-0.70), they misclassify several fraction items as easy that are among the hardest for students (e.g., 34% correct). LLMs approximate curricular rather than cognitive difficulty. **Relevance to AI in Education:** This paper contributes to the understanding of [[automated-assessment]], [[personalized-learning]], and [[student-experience]]. The findings have implications for [[adaptive-learning]] systems, [[formative-assessment]] design, and the broader [[edtech-platform]] landscape. Future work should explore how these results generalize across [[stem-education]] and [[higher-ed]] contexts. This research connects to the growing body of work on [[ai-literacy]] and [[teacher-role]], highlighting both the promise and limitations of AI tools in educational settings. ## Connected Concepts - [[automated-assessment]] - [[personalized-learning]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[edtech-platform]] - [[stem-education]] - [[higher-ed]] - [[ai-literacy]] - [[teacher-role]] ## Connected Articles - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics ## Citation Amanda La Hadi, Muhammad Johan Alibasa, Guanliang Chen, A. Taufiq Asyhari (2026). [The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty](https://arxiv.org/abs/2607.26067). arXiv:2607.26067. EDM 2026 (Educational Data Mining Conference). --- ## [Exploring the Design Space of LLM-Based Programming Support in CS Education: A Scoping Review through the Lens of Assistance Governance](https://edtechdev.github.io/aied/articles/llm-programming-support-governance-cs-education/) This scoping review synthesizes 90 peer-reviewed [[llm]]-based programming support systems in [[cs-education]] to make explicit how each system bounds, enacts, and controls assistance — decisions the authors argue are usually left implicit. They introduce the **PEA framework**: Policy (what help is allowed or restricted), Enforcement (how boundaries are operationalized through interaction and system behavior), and Authority (who can configure, adapt, or override them at runtime). The analysis shows systems often share pedagogical goals but diverge sharply in enforcement mechanisms, while authority remains highly centralized in system logic — few tools give learners or instructors runtime control. PEA offers a comparative vocabulary that connects to [[genai-assessment-governance]] and [[genai-policies-higher-ed-computing]], and its emphasis on bounded, configurable help echoes [[structured-llm-feedback-programming]] and [[llm-feedback-programming-classroom]]. The governance lens reframes [[scaffolding]] as a design-space question of control rather than a fixed pedagogical feature. ## Connected Concepts - [[llm]] - [[cs-education]] - [[scaffolding]] ## Connected Articles - [[genai-assessment-governance]] - [[genai-policies-higher-ed-computing]] - [[structured-llm-feedback-programming]] - [[llm-feedback-programming-classroom]] ## Citation Kim, Monisha, Wu & Smith IV (2026). [Exploring the Design Space of LLM-Based Programming Support in CS Education: A Scoping Review through the Lens of Assistance Governance](https://arxiv.org/abs/2607.21257). arXiv:2607.21257. arXiv preprint (cs.HC). --- ## [Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach](https://edtechdev.github.io/aied/articles/llm-psychometric-calibration-cdp/) Proposes Cognitive Diagnostic Profiling (CDP), a zero-shot framework that dramatically improves LLM-simulated examinee alignment with human test-takers. With CDP, IRT difficulty Spearman correlations rose from 0.24 to 0.90, and RMSE fell from 6.31 to 0.90. Makes LLM-simulated examinees practical for operational test development. **Relevance to AI in Education:** This paper contributes to the understanding of [[automated-assessment]], [[personalized-learning]], and [[student-experience]]. The findings have implications for [[adaptive-learning]] systems, [[formative-assessment]] design, and the broader [[edtech-platform]] landscape. Future work should explore how these results generalize across [[stem-education]] and [[higher-ed]] contexts. This research connects to the growing body of work on [[ai-literacy]] and [[teacher-role]], highlighting both the promise and limitations of AI tools in educational settings. ## Connected Concepts - [[automated-assessment]] - [[personalized-learning]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[edtech-platform]] - [[stem-education]] - [[higher-ed]] - [[ai-literacy]] - [[teacher-role]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics ## Citation Wenjie Zhou, Yunting Liu, Renjiao Tang, Mark Wilson (2026). [Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach](https://arxiv.org/abs/2607.26317). arXiv:2607.26317. arXiv preprint. --- ## [Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition](https://edtechdev.github.io/aied/articles/llm-reasoning-traces-metacognition/) # LLM Reasoning Traces & Metacognition This preregistered between-subjects study (N=559) provides the first rigorous evidence that [[llm]] reasoning traces — increasingly common in AI interfaces — do not improve performance and can actively impair it. More critically, they create a dangerous [[metacognition|metacognitive]] blind spot: participants substantially overestimate their performance regardless of trace format. ## Key Findings - **Summary traces** preserved task performance at the no-trace baseline while elevating trust and hedonic appeal — changing how users *feel* without helping them perform. - **Full traces** from a verbose open-weight model actually *impaired* performance relative to answer-only baselines. - **No trace format** supported calibrated self-evaluation — metacognitive overestimation was universal. - **Hedonic appeal**, not trust, carried the indirect path to overestimation, consistent with a processing-fluency account: the pleasant experience of reading traces inflates confidence without improving understanding. ## Connection to AIED These findings have profound implications for [[intelligent-tutoring]] and [[ai-peer-feedback-systems|AI feedback systems]]. If students feel more confident after seeing AI reasoning but don't actually learn better, then simply exposing AI reasoning in educational interfaces may create an [[over-reliance]] trap. The paper's recommendation — that calibration should be scaffolded by interactions that elicit users' own reasoning first — directly aligns with [[self-regulated-learning]] principles and [[cognitive-offloading-speedup-illusion|cognitive offloading research]] showing that AI use can reduce active engagement. ## Contrast with Assessment Governance While [[genai-assessment-governance|GenAI assessment governance]] focuses on *when* to allow AI in evaluation, this paper addresses *how* AI explanations affect learning — suggesting that even well-designed AI transparency features can backfire without metacognitive scaffolding. ## Connected Concepts - [[llm]] - [[metacognition]] - [[intelligent-tutoring]] - [[over-reliance]] - [[self-regulated-learning]] ## Connected Articles - [[ai-peer-feedback-systems]] - [[cognitive-offloading-speedup-illusion]] - [[genai-assessment-governance]] ## Citation Fernandes, D., Buschek, D., Tankelevitch, L., Kosch, T., & Welsch, R. (2026). [Explaining too much? Understanding how large language model reasoning traces influence performance and metacognition](https://arxiv.org/abs/2605.25856). arXiv:2605.25856. cs.HC. --- ## [Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers: A Mixed-Methods Study at a Minority-Serving R1 University](https://edtechdev.github.io/aied/articles/llm-reliance-types-undergrad/) Hossain (2026) develops a typology of LLM reliance among undergraduate writers at a minority-serving R1 institution, identifying four distinct profiles: strategic scaffolders who use AI for idea generation and structure, critical editors who revise AI output substantially, passive acceptors who submit AI-generated text with minimal changes, and uncritical delegators who offload entire assignments. The mixed-methods design combines survey data with qualitative interviews, revealing that self-efficacy and prior academic preparation are the strongest predictors of the reliance type. The [[over-reliance]] types (passive acceptors and uncritical delegators) were disproportionately represented among students with lower writing self-efficacy, raising [[equity]] concerns. The findings directly inform [[ai-literacy]] curriculum design by identifying which student populations need targeted support. The study contributes to [[writing-education]] research by mapping how [[student-experience]] of AI writing tools varies across a diverse student body, challenging one-size-fits-all AI policies in [[higher-ed]]. ## Connected Concepts - [[over-reliance]] - [[equity]] - [[ai-literacy]] - [[writing-education]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[student-rationalization-ai-writing]] — It''s OK Because...": The Wild West of Student Rationalization of AI Use in Academic Writing - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration: A Longitudinal Analysis of Prompting Strategies - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs ## Citation Shahin Hossain (2026). [Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers: A Mixed-Methods Study at a Minority-Serving R1 University](https://arxiv.org/abs/2606.28749). arXiv:2606.28749. cs.CY / cs.AI / cs.HC. --- ## [LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments](https://edtechdev.github.io/aied/articles/llm-sentiment-analysis-education-research/) **LLM-Assisted Sentiment Analysis for Mixed-Methods Education Research** demonstrates how LLMs can serve as scalable qualitative research assistants, enabling researchers to investigate multiple demographic variables simultaneously rather than being limited to simple binary comparisons. Using 151 longitudinal written reflections from a study abroad program, the authors show that LLM-assisted sentiment analysis combined with statistical testing can uncover granular patterns: prior experience living abroad was the only personal variable that significantly impacted students' sentiments about their language and communication behaviors. This workflow bridges computational and qualitative methods, suggesting that LLMs can reduce the bottleneck of manual qualitative coding without replacing the interpretive depth of thematic analysis. The approach has implications for [[higher-ed]] research methodology, complementing existing [[learning-analytics]] pipelines and extending mixed-methods capabilities beyond what has been possible with [[automated-grading]] and [[formative-assessment]] systems alone. The paper connects to discussions about [[faculty-development]] in equipping researchers with AI literacy for methodological innovation, and relates to [[ai-literacy]] as both a tool for researchers and a consideration in how computational methods change the practice of qualitative inquiry. - LLM-assisted sentiment analysis enables comparison across 7 identity/lived-experience variables simultaneously - Only prior experience living abroad significantly impacted students' communication sentiments - The workflow preserves qualitative depth while adding statistical power - Implications for [[learning-analytics]] and [[student-experience]] research methodology ## Connected Concepts - [[higher-ed]] - [[learning-analytics]] - [[automated-grading]] - [[formative-assessment]] - [[faculty-development]] - [[ai-literacy]] - [[student-experience]] ## Connected Articles - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration: A Longitudinal Analysis of Prompting Strategies - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis ## Citation Xiomara Gonzalez, Gabriella Coloyan Fleming, Andrew Katz, Maya Denton, Jessica Deters (2026). [LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments](https://arxiv.org/abs/2605.27403). arXiv:2605.27403. arXiv preprint. --- ## [What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics](https://edtechdev.github.io/aied/articles/llm-student-misconception-identification/) # Using LLMs to Identify Student Misconceptions ## Synthesis This paper presents a systematic two-stage methodology for surfacing student misconceptions at scale. Drawing on 3,802 medical student enrollments across 5 biomedical science courses (9 course periods, 40-50 quizzes each), Parker and Zavala-Cerna first use quantitative quiz-level performance metrics to identify challenging topics, then deploy LLMs to analyze quiz questions, student response patterns, and lecture transcripts in combination — extracting the specific misconceptions underlying poor performance. The quality of LLM-identified misconceptions was rated as excellent by subject matter experts, and faculty interviews confirmed that data-driven topic identification aligned with, and extended, instructors' own classroom observations. This is significant because it demonstrates that LLM-based analysis can surface insights invisible in performance data alone — bridging the gap between [[learning-analytics]] dashboards (which show *what* is going wrong) and qualitative pedagogical reasoning (which explains *why*). The approach connects to several established themes in the wiki. It operationalizes [[formative-assessment]] by enabling targeted, misconception-specific interventions rather than generic remediation. It advances [[personalized-learning]] by providing the diagnostic foundation needed for adaptive systems to respond to individual conceptual gaps. And it extends [[knowledge-tracing-irt|knowledge tracing]] beyond binary correct/incorrect signals to the richer space of specific misconception categories. For [[student-experience]], the implications are direct: students struggling with the same topic for different conceptual reasons would receive different remediation. For [[ai-literacy]], the paper models how instructors can leverage AI outputs as hypotheses to be validated by expert judgment, rather than as authoritative diagnoses — a responsible-use pattern relevant to faculty development. ## Connected Concepts - [[learning-analytics]] - [[formative-assessment]] - [[personalized-learning]] - [[student-experience]] - [[ai-literacy]] ## Connected Articles - [[knowledge-tracing-irt]] ## Citation Parker, M. J., & Zavala-Cerna, M. G. (2026). [What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics](https://arxiv.org/abs/2605.00294). Education and Information Technologies. arXiv:2605.00294. --- ## [LLM Student Modeling and Long-Term Memory Architecture](https://edtechdev.github.io/aied/articles/llm-student-modeling-memory/) > Current AI tutoring systems treat each session as independent. Adaptive systems use real-time knowledge tracing (e.g., [[knowledge-tracing-irt|IRT-based models]]) but rarely retain a longitudinal student model that evolves across semesters. Longitudinal personalization is essential for effective scaffolding because: > PersonaVLM demonstrates a general-purpose framework for long-term user personalization—chronological memory extraction, evolving personality inference, and persona-aligned response generation—that maps directly onto the challenge of building AI tutoring systems that remember a student's learning history across months or years.^[[nie-personavlm-long-term-personalization-2026]] ## The Challenge in Educational Contexts Current AI tutoring systems treat each session as independent. Adaptive systems use real-time knowledge tracing (e.g., [[knowledge-tracing-irt|IRT-based models]]) but rarely retain a longitudinal student model that evolves across semesters. Longitudinal personalization is essential for effective scaffolding because: 1. **Preferences shift** — A student may initially prefer worked examples, later shift to Socratic questioning, then return to examples under stress 2. **Expertise builds** — [[self-regulated-learning|SRL skills]] develop over time, changing what scaffolding is appropriate 3. **Transfer depends on history** — Whether [[transfer-of-learning|AI-assisted gains persist]] may depend on whether the tutor *remembers* past learning and spacing ## The PersonaVLM Architecture (General Framework) PersonaVLM (Nie et al., 2026) proposes a two-stage agent architecture for long-term personalization: ### Stage 1: Response (Real-Time) Multi-step reasoning with targeted memory retrieval: - Decides *whether* to retrieve, *what* to retrieve, and *when* (time period + keywords) - Retrieves from procedural, semantic, and episodic memory in parallel - Generates answer aligned with user's inferred personality ### Stage 2: Update (Asynchronous) Post-response memory and personality maintenance: - **Personality Evolving Mechanism (PEM):** Infers Big Five (OCEAN) traits per turn, updates via Exponential Moving Average with cosine-decay smoothing (early interactions weighted more, later stabilized) - **Four memory types:** - *Core memory:* Foundational attributes (latest version only) - *Semantic memory:* Event-independent knowledge, updated every turn - *Episodic memory:* Time-stamped events with summaries - *Procedural memory:* Goals, habits, recurring behaviors ## Relevance to AI in Education While PersonaVLM was evaluated on general assistant tasks, its architecture addresses a gap in educational AI: **most tutoring systems lack longitudinal student memory**. The implications are: | Educational Need | PersonaVLM Mapping | |---|---| | Persistent learner profile across sessions | Core memory + procedural memory | | Evolving preference for explanation style | Semantic memory + PEM personality alignment | | Remembering past misconceptions | Episodic memory with time-stamped retrieval | | Calibrating to emotional state (frustration, motivation) | PEM neuroticism/extraversion tracking | ### Connection to Existing Tutoring Research - [[knowledge-tracing-irt]] provides real-time ability estimation but is session-constrained; integrating chronological episodic memory (as in PersonaVLM) could enable longitudinal knowledge modeling - [[self-regulated-learning]] frameworks identify regulation phases (planning, monitoring, reflection) that could map to Core/Procedural/Episodic memory types - [[tutoring-specific-vs-general-ai|tutoring-specific design]] requires maintaining pedagogical guardrails even as the system personalizes; PersonaVLM's alignment mechanism could be constrained to educational (not just social) personality dimensions ## Limitations for Education 1. **No educational evaluation:** Persona-MME benchmark tests general personal assistant scenarios, not tutoring 2. **Privacy concerns in K-12:** Longitudinal student memory raises FERPA/COPPA questions; PersonaVLM's self-contained pipeline (no API dependency) mitigates this but local deployment remains infrastructure-heavy 3. **Bias risk:** Personality inference from limited student interaction may stereotype; EMA smoothing helps but doesn't eliminate it 4. **Personality vs. competence:** Big Five alignment optimizes for user satisfaction, not necessarily learning outcomes—these can conflict (e.g., a student prefers easy answers, but learning requires productive struggle) ## Open Questions 1. What educational personality/adaptation dimensions should replace/modify Big Five? (e.g., academic goal orientation, prior knowledge state, metacognitive monitoring accuracy) 2. How does longitudinal memory interact with spaced repetition and forgetting curves? 3. Would a tutoring system with PersonaVLM-style memory produce better [[transfer-of-learning|transfer outcomes]] than episodic-only systems? 4. What are the pedagogical guardrails needed to prevent personalization from becoming over-accommodation? ## Connected Concepts - [[self-regulated-learning]] ## Connected Articles - [[knowledge-tracing-irt]] - [[nie-personavlm-long-term-personalization-2026]] - [[transfer-of-learning]] - [[tutoring-specific-vs-general-ai]] ## Citation Nie, C., Fu, C., Zhang, Y., Yang, H., & Shan, C. (2026). [*PersonaVLM: Long-Term Personalized Multimodal LLMs*](https://arxiv.org/abs/2604.13074). arXiv:2604.13074. --- ## [Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators](https://edtechdev.github.io/aied/articles/llm-student-simulation-misconception-faithfulness/) # LLM Student Simulation and Misconception Faithfulness ## Synthesis This paper exposes a critical failure mode in using LLMs as simulated students for [[intelligent-tutoring]] development and evaluation. The authors introduce **misconception faithfulness** — the property that a simulated student holds a coherent, misconception-driven belief state and updates it *only* when feedback addresses the underlying misconception — and show that across seven LLMs (4B to 120B parameters), simulators exhibit near-zero faithfulness. The core finding is a **sycophantic failure mode**: when given any corrective signal, LLM simulators abandon their assigned misconception persona and re-solve the problem from internal knowledge. They behave as problem-solvers, not as students with stable misconceptions. Using the novel **Selective Flip Score (SFS)**, the authors quantify this: simulators flip their answers at similarly high rates regardless of whether feedback is targeted, misaligned, or generic. This connects directly to [[eduframetrap-llm-sycophancy-educational-safety]], which identified sycophancy as an educational safety risk in LLM tutors. Here the sycophancy is inverted: simulated *students* capitulate to feedback rather than maintaining authentic misconception-driven behavior. Both papers together establish sycophancy as a bidirectional problem in AIED — affecting both tutor and student roles. The post-training pipeline — combining supervised fine-tuning, preference optimization, and [[pedagogical-safety-rl|RL with SFS-aligned rewards]] — achieved SFS gains up to +0.56, demonstrating that misconception faithfulness is trainable. This has implications for [[ai-tutor-safety-harms]]: if student simulators used for tutor safety testing are themselves unfaithful, safety evaluations conducted on them may systematically miss harm patterns that real students would exhibit. For [[student-experience]] and benchmark development, this paper motivates a paradigm shift from static output matching toward interactive, belief-aware student modeling — a theme that also resonates with [[llm-student-modeling-memory]] and the behavioral evaluation framework in [[ai-tutor-behavioral-evaluation]]. ## Connected Concepts - [[intelligent-tutoring]] - [[student-experience]] ## Connected Articles - [[eduframetrap-llm-sycophancy-educational-safety]] - [[pedagogical-safety-rl]] - [[ai-tutor-safety-harms]] - [[llm-student-modeling-memory]] - [[ai-tutor-behavioral-evaluation]] ## Citation Do, H., Sonkar, S., & Sachan, M. (2026). [Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators](https://arxiv.org/abs/2605.12748). arXiv:2605.12748. --- ## [Can LLMs Effectively Simulate Human Learners? Teachers' Insights from Tutoring LLM Students](https://edtechdev.github.io/aied/articles/llm-student-simulation-teacher-insights/) > **Synthesis:** Semi-structured interviews with 12 teachers who tutored LLM-simulated students (MathDial dataset) reveal key authenticity gaps: overly complex language, lack of emotions, unnatural attentiveness, and logical inconsistency. The study categorizes four real-world student behavior types along scaffolding and presence dimensions, and provides design guidelines for building higher-fidelity LLM student simulations. ## Methodology Martynova et al. interviewed 12 teachers who had extensively interacted with LLM-simulated students during collection of the MathDial dialogue tutoring dataset. The study used a mixed-method approach grounded in two frameworks: - **Community of Inquiry (CoI)** — capturing social and cognitive presence in learning interactions - **Scaffolding theory** — effective teaching through graduated support Teachers tutored LLM students in K-12 math problem-solving dialogues, then rated realism and described deviations from authentic student behavior. ## Key Findings ### Authenticity Gaps in LLM Students | Issue | Description | |---|---| | Language complexity | Responses too technical, lengthy, and formal for K-12 students | | Emotional absence | Lack of frustration, fear, embarrassment, or disengagement | | Unnatural attentiveness | Students too engaged; never lose focus or go silent | | Logical inconsistency | Knowledge jumps without gradual building; no forgetting | | No question-asking | Teachers had too much control over discussion flow | ### Four Student Behavior Categories The study classifies real-world student behaviors along two dimensions: | | High Scaffolding Needs | Low Scaffolding Needs | |---|---|---| | **Social Presence** | Short/simple writing, negative emotions, disengagement | Asking questions, disagreeing with teacher | | **Cognitive Presence** | Gradual knowledge-building, memory/forgetting | Changing tactics based on feedback | LLMs captured the bottom-right quadrant reasonably well but failed to represent the other three categories. ## Design Guidelines 1. **Diverse personalities** — model Big Five personality traits to produce varied engagement levels and emotional responses 2. **Gradual knowledge building** — integrate knowledge tracing to avoid unrealistic knowledge jumps 3. **Model forgetting** — account for memory decay over time 4. **Promote question-asking** — use context-aware triggers for the LLM student to ask questions 5. **Vary language complexity** — regulate response length, formality, and introduce age-appropriate errors 6. **Allow disengagement** — let simulated students lose focus or stay silent, providing authentic teaching challenges ## Significance - **Teacher training**: more realistic LLM student simulations enable scalable practice for pre-service and in-service teachers - **Validation gap**: only 3% of studies simulating learners do post-factum validation — this study provides a framework for it - **MathDial** is the only publicly available dataset of real teacher/LLM-student interactions - Addresses the growing trend of using unvalidated LLM simulations in educational contexts ## Connected Concepts - [[k-12]] - [[knowledge-tracing]] - [[llm]] - [[scaffolding]] ## Connected Articles - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[ai-in-the-wild-college]] — AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue ## Citation Learners?, C.L.E.S.H., Students, T.I.F.T.L., Daheim1,2, D.M.J.M.N., Sachan1, Ö.N.Y.X.Z.M., Fraser, E.Z.T.D.S., many, L.L.M.O., & aims, F.B.H.L.S.A.P.U.T.S. (2026). [*Can LLMs Effectively Simulate Human Learners? Teachers' Insights from Tutoring LLM Students*](https://aclanthology). Innovative Use of NLP for Building Educational Applications) **DOI:** https://aclanthology --- ## [A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large Language Models: An Exploratory Study of Educational Potential](https://edtechdev.github.io/aied/articles/llm-tts-dialogue-lesson-generation/) **Gendo Kumoi, Fumie Watanabe, Tota Suko, Takashi Ishida, et al. (2026)** - arXiv preprint (IEEE). arXiv preprint. ## Key Findings - Proposes a human-in-the-loop LLM+TTS pipeline that generates Expert-Novice dialogue lessons, augmenting rather than replacing educators - see [[generative-ai]]. - Three-stage workflow (LLM slide/narration generation -> educator review -> automated audiovisual integration) parallels [[scaffolding]] design. - Builds on cognitive apprenticeship theory to structure dialogue narration, supporting [[active-learning]]. - A quasi-experiment with 245 first-year high school students suggests educational potential of dialogue-based lessons. - Relates to [[personalized-learning]] and prior [[ai-generated-slides-student-perception]] work on AI-generated instructional material. - Informs [[pedagogical-llm-training]] by showing how LLMs can be steered toward pedagogically sound lesson generation. ## Connected Concepts - [[generative-ai]] - [[scaffolding]] - [[active-learning]] - [[personalized-learning]] - [[pedagogical-llm-training]] ## Connected Articles - [[ai-generated-slides-student-perception]] ## Citation Kumoi, G., Watanabe, F., Suko, T., Ishida, T., et al. (2026). [A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large Language Models: An Exploratory Study of Educational Potential](https://arxiv.org/abs/2607.12235). arXiv:2607.12235. --- ## [What out-of-the-box LLMs can(t) do in law? A Turing test in Italian exams for lawyers, judges and notaries](https://edtechdev.github.io/aied/articles/llm-turing-test-italian-legal-exams-2026/) > **Synthesis:** This paper reports a blind Turing Test evaluating leading LLMs on three Italian professional legal examinations: the Bar exam, Judges exam, and Notary exam. LLMs generated full written papers that were anonymised and graded by expert examiners using real examination rubrics. Results show marked variance across models and tasks: some LLMs match or exceed human passing thresholds on certain exam sections, while all models struggle with tasks requiring deep legal reasoning, jurisdiction-specific knowledge, and nuanced argumentation. The study highlights both the promise and the current limits of LLMs in high-stakes professional assessment contexts, raising implications for AI's role in legal education and certification. The article reports on a blind Turing Test experiment, assessing the performance of out-of-the-box leading LLMs on three Italian legal professional exams: the Bar, Judges and Notary exams. Leading LLMs were asked to generate full written exam papers, which were made indistinguishable from human submissions and anonymously evaluated by expert examiners, using the same criteria applied in real examinations. Results reveal marked differences across both models and tasks. While some LLMs match or exceed top human performance in adversarial legal argumentation and doctrinal analysis, all models fail in the notary exam, which requires goal-directed legal planning under strict formal and substantive constraints. Beyond ranking models, the study identifies task-specific strengths, limitations and reports a blind Turing Test evaluating leading LLMs on three Italian professional legal examinations: the Bar exam, Judges exam, and Notary exam. LLMs generated full written papers that were anonymised and graded by expert examiners using real examination rubrics. Results show marked variance across models and tasks: some LLMs match or exceed human passing thresholds on certain exam sections, while all models struggle with tasks requiring deep legal reasoning, jurisdiction-specific knowledge, and nuanced argumentation. The study highlights both the promise and the current limits of LLMs in high-stakes professional assessment contexts, raising implications for AI's role in legal education and certification. ## Connected Concepts - [[benchmark]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[automated-essay-scoring]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[open-source]] - [[cs-education]] ## Connected Articles - [[machines-misread-pedagogical-quality]] — Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[llm-computational-thinking-physics-2026]] — Using LLMs to Detect Growth in Computational Thinking in Introductory Physics - [[ground-truth-reliability-aied]] — Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education - [[tutoring-effectiveness-index]] — The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals ## Citation Bertoli, Germana et al. (2026). [What out-of-the-box LLMs can(t) do in law? A Turing test in Italian exams for lawyers, judges and notaries](https://arxiv.org/abs/2608.06166). arXiv:2608.06166. --- ## [Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most](https://edtechdev.github.io/aied/articles/llm-tutoring-feedback-diagnosis-gap/) ## Key Finding LLM tutors achieve near-ceiling on correct steps but systematically over-reject valid-suboptimal reasoning and over-validate incorrect solutions — precisely where adaptive tutoring matters most. ## Synthesis This paper exposes a critical diagnostic blind spot in LLM-based tutoring agents. Across seven models and 10,836 solution-feedback pairs in propositional logic, LLMs performed near-perfectly on clearly correct steps but systematically misfired on the cases that matter most for adaptive tutoring: they over-rejected valid-but-suboptimal reasoning and over-validated incorrect solutions. These failures persisted regardless of solution context, suggesting architectural limitations rather than insufficient information. Alarmingly, even when models correctly diagnosed a step, they often failed to produce pedagogically actionable feedback — revealing a gap between diagnostic accuracy and instructional effectiveness. The authors propose hybrid architectures where knowledge-graph-grounded models handle precise diagnosis while LLMs support open-ended [[scaffolding]] and dialogue. This finding directly complements the behavioral evaluation framework from [[ai-tutor-behavioral-evaluation]], which also found that pedagogical quality alone is insufficient — students must actually act on feedback. Together, these papers suggest that current LLM tutors need both better diagnostic precision AND better feedback-actionability to serve as effective [[intelligent-tutoring]]. ## Connected Concepts - [[scaffolding]] - [[intelligent-tutoring]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] ## Citation preprint, A. (2026). [*Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most*](https://arxiv.org/abs/2605.16207) --- ## [Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education](https://edtechdev.github.io/aied/articles/llm-unlearning-math-privacy/) > **Synthesis:** Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education ## Key Findings - The study applies **gradient-based LLM unlearning** to **three models** — Math-GPT-J, Llama-Lora (Llama-1), and Llama-2-QLora (Llama-2), open-source math-focused LLMs extended from the authors' prior work — pre-trained on approximately **3 million data points** from an Algebra I online discussion forum (Algebra Nation) between students and professional tutors. - **PII and harmful content were detected on the training data** and targeted for unlearning in **two different orders** (PII-first and harmful-content-first, labeled P→H and H→P), producing two unlearned variants per base model. - The PII forget dataset was dominated by **person names (93.28%)**, reflecting the forum context; the harmful-content forget set was dominated by **harassment (10,367 messages)**, with toxic content the smallest category (861 messages). - Before unlearning, PII-containing output rates were substantial (Llama-1: 17.8%; Llama-2: 15.8%; GPT-J: 15.7%; clean-data baselines 13.9%, 11.5%, 16.5%); after the harmful→PII unlearning order, rates fell to **0.1% for all three models**, while the PII→harmful order was somewhat less effective (Llama-1: 1.0%; Llama-2: 1.8%; GPT-J: 0.2%). - After unlearning, the **rates of PII-containing output and harmfulness substantially decreased** compared to the pre-trained models; on the external RealToxicityPrompts benchmark, harassment, offensive, and toxic output rates fell to **0.0% across all three model families** — interpreted as cross-dataset consistency, since baseline rates on that benchmark were already low. - **Utility was maintained** on both **single-label and multi-label downstream math classification tasks**: F1 scores for unlearned models remained comparable to pre-trained baselines across training-instance conditions (roughly 0.80–0.84, e.g., Llama-1 pre-trained 0.809 vs. unlearned H→P 0.809 at n = 100). - Sensitivity analyses showed the PII reductions were robust across learning rates and unlearning sample sizes (20K–150K), though larger forget samples did not produce monotonic improvement — effectiveness was not simply a function of using more unlearning examples. - The findings demonstrate a practical path to making LLM-based math tutors **more responsible and privacy-preserving** while retaining strong performance on math-related tasks. ## Study Design & Method Online mathematics learning platforms increasingly adopt LLMs for scalable, on-demand support, but pre-trained models may **reproduce private information from training data or generate harmful language**. The study first detects PII and harmful content on the ~3M-point Algebra I tutoring corpus, applies gradient-based unlearning in two orders (harmful→PII and PII→harmful), and then compares the generated outputs of the unlearned models with those of the pre-trained model in terms of **PII-containing output rate and harmful rate**, including on the external RealToxicityPrompts benchmark for generalization. Finally, the unlearned models are evaluated on two math classification tasks (single-label and multi-label) to confirm that utility survives, alongside learning-rate and unlearning-sample-size sensitivity analyses. The order effect matters theoretically: because the second unlearning stage updates the same parameters again, it can strengthen or partially reverse the first stage's changes, so the optimal sequence may not generalize uniformly across model families. ## Implications for AI in Education For responsible deployment in educational settings, the work shows that **post-hoc unlearning is a viable complement to data curation**: models already trained on sensitive or unsafe classroom data can be steered toward safer behavior without retraining from scratch. The trade-off between safety and utility is managed rather than eliminated — privacy and harmfulness drop substantially (to 0.1% PII output rates under the harmful→PII order and 0.0% toxic rates on the external benchmark) while math-task performance is maintained. Practitioners should treat unlearning order as a design decision rather than a detail, since the two orders produced different privacy–safety balances. This supports [[pedagogical-safety]] and [[privacy]]-conscious design of LLM-based tutors in [[k-12]] mathematics contexts, where student data are especially sensitive, and connects to the broader [[ai-governance-education]] conversation about model-level mitigations alongside data curation and deployment policy. ## Limitations The findings are limited to the Algebra I context in which unlearning was applied and evaluated, so results and procedures may not transfer directly to other subject areas, grade levels, learning platforms, or deployment contexts where PII-like and harmful output risks differ. The PII-containing output rate captures classifier-detected PII-like spans in generated responses and should not be interpreted as direct evidence of memorized training-data leakage — exact or approximate target matching has its own reliability problems, so reductions are best read as decreases in PII-like generation under the study's configuration. Only gradient-based unlearning was examined; other unlearning approaches remain to be benchmarked in education, and future work should add extraction tests, membership-inference evaluations, and target-specific reproduction analyses. ## Connected Concepts - [[pedagogical-safety]] - [[reinforcement-learning]] - [[privacy]] - [[k-12]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[affective-tutoring]] - [[ai-governance-education]] ## Connected Articles - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[child-safety-genai]] — Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[hazra-safetutors-pedagogical-safety-2026]] — SafeTutors: Pedagogical Safety in AI Tutoring - [[vetting-dual-llm-safety-education]] — VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI - [[knowledge-distillation-ai-tutor-evaluation]] — Knowledge Distillation for Automated AI Tutor Evaluation ## Citation Li, C., Gülfidan, G., & Zhang-Kopf, Y. (2026). [*Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education*](https://doi.org/10.1016/j.caeai.2026.100642). --- ## [Artificial intelligence, cognitive offloading and implications for education](https://edtechdev.github.io/aied/articles/lodge-loble-cognitive-offloading-2026/) > **Synthesis:** Lodge & Loble (2026) provide a comprehensive report on the cognitive science behind AI use in education, arguing that the core risk of generative AI is not plagiarism but cognitive offloading — students outsourcing the mental work required for durable learning. They distinguish beneficial offloading (freeing capacity for higher-order thinking) from detrimental outsourcing (bypassing desirable difficulties), introduce the concept of metacognitive laziness and a new metacognitive equity gap, and propose pedagogical strategies to move from cognitive atrophy to augmentation. This report from the Australian Network for Quality Digital Education investigates the risk that students — especially novice learners — will outsource too much cognitive work to AI, short-circuiting the mental effort required for deep, sustainable learning. Drawing on cognitive load theory and the science of learning, Lodge and Loble distinguish beneficial cognitive offloading (freeing capacity for higher-order thinking) from detrimental outsourcing (bypassing "desirable difficulties" that build durable knowledge). They identify a **performance paradox** where AI makes tasks feel easy but undermines learning, a **metacognitive laziness** that creates an illusion of competence, and a new **metacognitive equity gap** where already-advantaged students are better positioned to leverage AI effectively. The report proposes pedagogical strategies for moving from cognitive atrophy to cognitive augmentation. - **Beneficial vs. detrimental offloading:** Offloading lower-order tasks can free cognitive capacity, but outsourcing the struggle of learning itself prevents knowledge consolidation - **Performance paradox:** AI-assisted tasks feel fluent but produce "false mastery" — students perform well in the moment but retain less - **Bypassing desirable difficulties:** The friction that makes learning hard (retrieval, elaboration, generation) is precisely what AI eliminates - **Metacognitive laziness:** Students overestimate what they've learned because AI handled the cognitive heavy lifting - **Equity gap:** Students with strong prior knowledge and metacognitive skills leverage AI better — widening existing divides - **80% of Australian students** already use AI; two-thirds of early secondary teachers use it (OECD 2025) ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[k-12]] - [[metacognition]] - [[teacher-role]] - [[rag]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming ## Citation Lodge, J. M., & Loble, L. (2026). [Artificial intelligence, cognitive offloading and implications for education](https://doi.org/10.71741/4pyxmbnjaq.31302475). --- ## [LUDIA: A Design and Evidence Statement](https://edtechdev.github.io/aied/articles/ludia-udl-ai-thought-partner-2026/) > **Synthesis:** LUDIA is a no-cost, private, multilingual AI thought partner that connects educators with the Universal Design for Learning (UDL) framework. This statement describes the August 2026 relaunch rebuilt for privacy (no accounts, no cookies, no data collection), accessibility (WCAG 2.2 Level AA), and scale (13 languages, public-good architecture). The tool is positioned as a thought partner rather than content generator, and is evaluated against four 2026 guides from UNICEF, the European Commission, the EdTech Quality Collaborative, and the European EdTech Alliance. ## Key Findings LUDIA is a no-cost, private, multilingual AI thought partner that connects educators with the Universal Design for Learning (UDL) framework. This statement describes the August 2026 relaunch rebuilt for privacy (no accounts, no cookies, no data collection), accessibility (WCAG 2.2 Level AA), and scale (13 languages, public-good architecture). The tool is positioned as a thought partner rather than content generator, and is evaluated against four 2026 guides from UNICEF, the European Commission, the EdTech Quality Collaborative, and the European EdTech Alliance. The work contributes to understanding of [[universal-design-for-learning]] in educational contexts, with implications for [[accessible-learning]], [[professional-training]]. ## Connected Concepts - [[universal-design-for-learning]] - [[accessible-learning]] - [[professional-training]] - [[instructional-design]] - [[equity]] - [[edtech-platform]] ## Connected Articles - [[dyslexlens-dyslexic-learners-ai]] - [[embodied-string-learning-blindness-low-vision-musicians]] ## Citation## Citation Stark, B. (2026). [*LUDIA: A Design and Evidence Statement*](https://doi.org/10.35542/osf.io/3nmgp_v1). EdArXiv preprint. --- ## [Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation](https://edtechdev.github.io/aied/articles/machines-misread-pedagogical-quality/) Tseng et al. (2026) investigate human-machine alignment in LLM-based pretest question evaluation — a critical bottleneck for scalable AI-assisted assessment. Their AI-assisted workflow combines automated generation, rubric-based evaluation, and iterative selection. Through a 2×2 experimental design varying rubric operationalization and evaluation mode, they find that human-machine disagreements are systematic rather than random, rubric revision has a larger effect on alignment than rationale-first evaluation, and the two interventions are complementary. The core insight is that scalable AI-assisted pretesting depends not only on generation capability but crucially on how pedagogical quality is operationalized for machine interpretation. This work contributes to [[ai-ed-evaluation]] by providing empirical evidence for aligning [[llm]] judgment with human pedagogical standards in [[formative-assessment]] contexts, and has direct implications for [[automated-grading]] and [[assessment]] system design. ## Connected Concepts - [[ai-ed-evaluation]] - [[llm]] - [[formative-assessment]] - [[automated-grading]] - [[assessment]] ## Connected Articles - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[automated-formative-assessments-a-level-sciences]] — The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs ## Citation Pei-Yu Tseng, Mahir Akgun, Peng Liu (2026). [Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation](https://arxiv.org/abs/2606.23629). arXiv:2606.23629. arXiv:2606.23629 (cs.HC) --- ## [MBP-KT: Learning Global Collaborative Information from Meta-Behavioral Pattern for Enhanced Knowledge Tracing](https://edtechdev.github.io/aied/articles/mbp-kt-meta-behavioral-knowledge-tracing/) ## Analysis This paper proposes **MBP-KT**, which transforms raw learner interaction sequences into structured **meta-behavioral patterns** before extracting collaborative signals. Raw sequences contain redundant noise; by decomposing interactions into distinct behavioral patterns (success-streaks, struggle-recovery, hesitation), the model captures higher-order learning dynamics. The parameter-free global extraction module makes this broadly applicable — extracted representations can be injected into any downstream KT architecture. This connects to [[neural-symbolic-knowledge-tracing]] by introducing structured behavioral representations, and to [[adaptive-learning]] by providing a model-agnostic enhancement layer. ## Key Findings - **MBP-KT** transforms raw learner interactions into **meta-behavioral patterns**, moving from low-level response sequences to structured summaries of how a learner behaves over time. - The model extracts **global collaborative information** — signals that emerge across learners rather than within a single learner's sequence — to enhance knowledge tracing. - The global extraction module is designed **without additional parameters**, keeping the enhancement computationally lightweight and easy to add. - The method provides **universal injection strategies** that work with **any downstream knowledge tracing architecture**, making it a model-agnostic enhancement layer rather than a standalone replacement. - By capturing both behavioral structure and cross-learner collaborative signal, MBP-KT addresses a limitation of standard sequence-based knowledge tracing, which typically ignores what other learners' trajectories reveal about the current learner's state. ## Implications for AI in Education Knowledge tracing is the backbone of [[adaptive-learning]] systems, and MBP-KT's contribution is architectural: a parameter-free way to fold richer behavioral and collaborative information into existing knowledge tracing models. For practitioners, this means improved mastery estimation without redesigning their KT stack — relevant to [[student-modeling]] and to the deployment of [[knowledge-tracing]] in adaptive tutoring platforms. ## Connected Concepts - [[adaptive-learning]] ## Connected Articles - [[neural-symbolic-knowledge-tracing]] ## Citation Jia et al. (2026). [MBP-KT: Learning Global Collaborative Information from Meta-Behavioral Pattern for Enhanced Knowledge Tracing](https://arxiv.org/abs/2605.08697). arXiv:2605.08697. arXiv preprint. --- ## [Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact](https://edtechdev.github.io/aied/articles/measuring-llm-tutors-teach-vs-solve/) Studies whether public LLM tutoring benchmarks distinguish learning-supportive behavior from mere answer production. Proposes a lightweight diagnostic based on the gap between solving-oriented and pedagogy-oriented benchmark performance. Using MathTutorBench, shows correlation between solving and pedagogy composites is only r=0.421 across 8 models, with several models shifting rank when evaluated on pedagogy. Benchmarks reward guiding questions, calibrated hints, and non-disclosive scaffolding. Recommends reporting solving and pedagogy scores separately. ## Key Findings - Stronger task-solving ability does **not necessarily imply stronger learning support**: a response that looks helpful can be pedagogically harmful by giving away the answer. - Using public **MathTutorBench** leaderboard results across **eight publicly reported models**, the correlation between solving-oriented and pedagogy-oriented composite scores is only **r = 0.421**, and **several models shift meaningfully in rank** when evaluation moves from solving to pedagogy. - Analysis of the public **TutorBench sample** shows that agency-relevant behaviors are **explicitly encoded in benchmark rubrics**, especially in active-learning settings that reward **guiding questions, calibrated hints, and non-disclosive scaffolding**. - The proposed diagnostic is **lightweight**: it is based on the gap between solving-oriented and pedagogy-oriented benchmark performance, requiring no new data collection. - The authors argue that **educational-impact evaluation should not treat task success as a sufficient proxy for learning support**. ## Study Design & Method The diagnostic exploits the fact that public tutoring benchmarks (MathTutorBench, TutorBench) score models on multiple rubrics. By separating rubric items into solving-oriented and pedagogy-oriented composites, the authors compute a per-model gap that reveals whether a model's benchmark standing reflects teaching quality or merely answer production. The correlational analysis across eight models quantifies how partially aligned the two dimensions are, while the rubric analysis identifies which specific behaviors — guiding questions, calibrated hints, non-disclosive scaffolding — benchmarks already reward. ## Implications for AI in Education For the [[benchmark]] community and for AI tutor deployment, the findings argue for **reporting solving-oriented and pedagogy-oriented scores separately** and for making disclosure-sensitive, student-agency-preserving criteria more explicit. A model that tops a solving leaderboard should not be assumed to be a good tutor; evaluation infrastructure must measure learning support directly. This connects to [[scaffolding]] and to the design of [[ai-tutoring]] systems where the goal is not the fastest answer but durable student understanding. ## Connected Concepts - [[ai-tutoring]] - [[pedagogical-llm-training]] - [[agentic-ai]] - [[affective-tutoring]] - [[scaffolding]] - [[help-seeking]] - [[pedagogical-agent]] - [[affective-computing]] ## Connected Articles - [[llm-judged-helpfulness-pedagogy-signal]] — Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[human-ai-collaboration-prerequisite-functions]] — What do you mean by human-AI collaboration: Prerequisite functions and the affordances needed to achieve it - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[rethinking-scaffolding-llm-tutors]] — Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents ## Citation Junyi Yao, Zihao Zheng, Baichuan Li (2026). [Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact](https://arxiv.org/abs/2606.16206). arXiv:2606.16206. arXiv preprint. --- ## [MedEasy: Designing AI Standardized Patients for Clinical Consultation Training](https://edtechdev.github.io/aied/articles/medeasy-ai-standardized-patients/) MedEasy multi-agent system simulates standardized patients with varying conditions for medical consultation training; outperforms script-based approaches in realism and adaptability. ## Synthesis MedEasy: Designing AI Standardized Patients for Clinical Consultation Training investigates medeasy multi-agent system simulates standardized patients with varying conditions for medical consultation training; outperforms script-based approaches in realism and adaptability. This work connects to existing research on accessible-learning by demonstrating that Abstract:AI standardized patients are becoming a setting for professional training in clinical consultation. This paper presents MedEasy, a multi-agent system that organizes virtual-patient practice through patient dialogue, clinical actions, decision submission, documentation, and feedback. We firs.... ## Connected Concepts - [[socratic-ai-dialogue]] - [[lifelong-learning]] - [[ai-tutoring]] - [[math-education]] - [[affective-tutoring]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[help-seeking]] ## Connected Articles - [[ai-psychotherapy-training-avatars]] — Toward Accessible Psychotherapy Training Using AI-Driven Interactive Patient Avatars - [[supplynet-visual-exploratory-learning]] — SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation - [[astra-atco-training-simulator]] — ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI ## Citation Zhiqi Gao, Huarui Luo, Guo Zhu, Bingquan Zhang, Dongyijie Primo Pan, Yizhan Feng, Jiahuan Pei, Jie Li, Benyou Wang (2026). [MedEasy: Designing AI Standardized Patients for Clinical Consultation Training](https://arxiv.org/abs/2606.17512). arXiv:2606.17512. --- ## [MedGame: Storytelling Gamification Empowered by Large Language Models for Medical Education](https://edtechdev.github.io/aied/articles/medgame-llm-medical-education-gamification/) MedGame transforms static clinical cases into structured, executable storytelling games for medical education, moving beyond the localized question-answering and single-turn feedback that characterize most [[llm]] medical-training systems. It uses a dual-engine design: a Medical Narrative Designer synthesizes case-grounded clinical storylines with states and decision nodes, while a Story Director converts them into dependency-aware multimodal orchestration plans rendered by an interactive platform. The authors release **MedGame Bench**, a 5,000-case benchmark and evaluation protocol for Medical Narrative Generation and Story Direction; task-specific fine-tuning substantially improves open-source LLMs and narrows the gap with commercial models. A pilot student study finds learners perceive MedGame as more engaging and useful than text-only alternatives, extending decision-centered, immersive approaches seen in [[multimodal-ai-tutoring]] and [[intelligent-tutoring]]. Its use of [[generative-ai]] for scenario authoring connects to [[llm-tts-dialogue-lesson-generation]], and the engagement gains speak to engagement-metrics as a design target in professional training. ## Connected Concepts - [[llm]] - [[intelligent-tutoring]] - [[generative-ai]] ## Connected Articles - [[multimodal-ai-tutoring]] - [[llm-tts-dialogue-lesson-generation]] ## Citation Wu, Zhou, Ma, Chen, Gao, Lin, Wu, Gou, Liu, Lau & Dou (2026). [MedGame: Storytelling Gamification Empowered by Large Language Models for Medical Education](https://arxiv.org/abs/2607.21570). arXiv:2607.21570. arXiv preprint (cs.CL). --- ## [Memdora: Designing Cognitively-Grounded Flashcard Interactions for AI-Powered Spaced Repetition](https://edtechdev.github.io/aied/articles/memdora-ai-spaced-repetition/) Presents Memdora, a cross-platform AI spaced repetition system that addresses limitations of binary flip-and-rate flashcard interactions. Grounded in cognitive science evidence on retrieval practice, it enables richer interaction patterns and reduces context-switching by generating flashcards from reading material. Demonstrates improved retention compared to traditional SRS tools. ## Key Findings - Spaced repetition systems have robust effects on long-term retention, but existing tools reduce flashcard interaction to a single **binary gesture: flip and self-rate**, an impoverished model that fails to leverage decades of cognitive science evidence on retrieval practice. - Memdora contributes a **taxonomy of 17 cognitively-grounded interaction types** across three learning categories — **Language (6 types), By Heart (1 type with 3 retrieval modes), and Exam (10 types)** — each mapped to peer-reviewed cognitive science evidence displayed on every card. - A **unified AI generation pipeline** collapses card creation to a **single gesture at the point of reading**, across web, mobile, and three browser extensions (Chrome, Edge, Firefox), reducing the need to context-switch out of reading flow. - A **collaborative classroom layer** enables teachers to publish decks, assign them to students, and **track learning outcomes at the individual card level**. - An **effort-based behavioral reward system** incentivizes actual cognitive engagement rather than mere app presence, and the system integrates **FSRS-6**, the current state-of-the-art spaced repetition algorithm. - Memdora is **deployed publicly on iOS, Android, Web, and three browser extensions**, advancing beyond prior AI flashcard systems including SmartFlash and KARL. ## Design Rationale The design rationale treats the flashcard as a full retrieval-practice instrument rather than a passive review object. The forgetting curve first described by Ebbinghaus — roughly **70% of newly learned material forgotten within 24 hours** without review — motivates scheduling, while the interaction taxonomy maps each interaction type to peer-reviewed evidence, so learners see the cognitive rationale behind the activity. The effort-based reward system extends this by rewarding the actual work of retrieval rather than time spent in the app, connecting to [[self-regulated-learning]] and to retrieval-practice research. ## Implications for AI in Education Memdora illustrates how AI-powered spaced repetition can move beyond scheduling to **interaction design**: by generating cards at the point of reading and offering varied, evidence-grounded retrieval activities, the system lowers the cost of effective study behavior. The classroom layer makes individual-card learning outcomes visible to teachers, supporting data-informed instruction, while the taxonomy itself is a reusable framework for designing cognitively grounded practice in [[adaptive-learning]] systems. ## Connected Concepts - [[self-regulated-learning]] - [[adaptive-learning]] - [[affective-tutoring]] - [[pedagogical-llm-training]] - [[affective-computing]] - [[personalized-learning]] - [[math-education]] - [[higher-ed]] ## Connected Articles - [[simulating-learner-task-selection]] — Simulating Learners' Task-Selection Strategies and System Constraints in Mastery Learning - [[llm-educational-simulation-adhd]] — LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[gaze-informed-ai-children]] — Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration - [[xai-education-framework]] — Explainable Artificial Intelligence in Education (XAI-ED) - [[neural-symbolic-knowledge-tracing]] — Neural-Symbolic Knowledge Tracing ## Citation Ruiyang Zhang (2026). [Memdora: Designing Cognitively-Grounded Flashcard Interactions for AI-Powered Spaced Repetition](https://arxiv.org/abs/2607.25096). arXiv:2607.25096. cs.HC. --- ## [Metacognitive AI literacy: going beyond the AI skills gap agenda](https://edtechdev.github.io/aied/articles/metacognitive-ai-literacy-beyond-skills-gap-2026/) > **Synthesis:** Shapiro, Souto-Otero, and Watermeyer (2026) argue that conventional AI literacy frameworks anchored in functional skills acquisition fail to address the fundamental epistemological challenges posed by probabilistic, opaque algorithmic systems. They reconceptualize AI literacy as a **metacognitive social practice** that transcends individual competencies to encompass collective capacity for critical engagement with AI design, deployment, and governance. Drawing on case studies from higher education institutions and the Nordic financial sector, they illustrate how participatory co-design processes and experimental pedagogical spaces can cultivate metacognitive awareness and democratic agency — arguing that universities must evolve beyond skills transmission to become sites of collective intelligence that anchor AI literacy as a public good. ## Key Findings - **AI literacy is a paradigm shift, not an extension of digital literacy:** the probabilistic nature and epistemic opacity of contemporary AI systems (particularly large language models) render traditional rule-based digital literacy paradigms obsolete; learners must navigate probabilistic reasoning, continuously question outcomes, and consider AI's broader impacts. - **Reconceptualizes AI literacy as a metacognitive social practice** — transcending individual skill acquisition toward collective critical engagement with AI design, deployment, and governance, rather than just functional tool use. - **Case studies show participatory co-design and experimental pedagogical spaces cultivate metacognitive awareness:** examples include Aalborg (interdisciplinary project-based engineering), Aalto, Strathmore (GAEIA, engaging ~200 postgraduate students across Global North and South), Northeastern's "robot-proof" university / Humanics model, Stanford's sandbox environments, and a Nordic bank's agentic-AI implementation that used co-design as a vehicle for metacognition. - **Efficacious AI literacy requires institutional transformation:** universities must evolve beyond skills transmission to become sites of collective intelligence where learners interrogate algorithmic power structures and mobilize alternative AI futures; AI literacy is framed as a governance problem, not merely a technical or educational challenge. - The framework transforms universities across three dimensions: **epistemically** (beyond functional skills to interrogating how algorithmic systems reshape knowledge production), **civically** (cultivating ethical agency to contest power asymmetries), and **institutionally** (reclaiming legitimacy as custodians of collective intelligence rather than vendors of digital credentials). ## Study Design & Method This is a **conceptual/argumentative article** (not an empirical study) drawing on case studies and literature analysis. It synthesizes scholarship on digital/media literacies, critical pedagogy, algorithmic governance, and higher education transformation (including Giroux, Marginson, Williamson, Pangrazio, Crawford, Eubanks, and Watermeyer). The authors present illustrative case studies of AI literacy in practice across higher education institutions (Aalborg, Aalto, Strathmore, Northeastern, Stanford) and the adult continuing training sector (a Nordic bank's agentic-AI implementation), examining how metacognitive AI literacy is operationalized across pedagogical design, organisational governance, and socio-technical experimentation. ## Implications for AI in Education The article reframes [[ai-literacy]] away from functional skills acquisition toward a **metacognitive social practice** relevant to [[higher-ed]] and [[ai-education]]. It argues that embedding AI literacy requires more than adding AI courses or tools — learning environments must enable learners to monitor and adjust their strategies in response to uncertain or opaque results, fostering scientific scepticism rather than narrow functional proficiency, connecting to [[metacognition]] and [[critical-thinking]]. It positions universities as essential infrastructure for civic resilience, demanding a fundamental reconfiguration of university missions, governance structures, and pedagogical approaches so AI literacy serves as a public good rather than a private asset. The article's case studies offer concrete models (participatory co-design, experimental pedagogical spaces, challenge-based collaboration) for cultivating metacognitive awareness and democratic agency, relevant to [[teacher-role]] and [[ethics]] and to algorithmic governance. ## Limitations As a conceptual article, the argument is not empirically tested, and the case studies are illustrative rather than systematically evaluated. The authors acknowledge the tension that universities operate under neoliberal funding regimes and "efficiency logics" that marginalize critical pedagogy, which limits the practical feasibility of the proposed transformation. The Nordic financial-sector case is adult-learning-focused rather than higher-education-specific, and the institutional examples span very different national contexts without comparative assessment. The proposal for institutional transformation is programmatic rather than operationalized. ## Connected Concepts - [[ai-literacy]] - [[metacognition]] - [[critical-thinking]] - [[teacher-role]] - [[ethics]] - [[higher-ed]] - [[ai-education]] ## Connected Articles - [[possibility-ai-literacy-critical-editorial]] — The (Im)possibility of AI Literacy - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy Interaction Patterns - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy ## Citation Shapiro, H., Souto-Otero, M., & Watermeyer, R. (2026). [*Metacognitive AI literacy: Going beyond the AI skills gap agenda*](https://doi.org/10.1080/17439884.2026.2652638). *Learning, Media and Technology*. --- ## [Experiential Versus Instructional Approaches for Eliciting Metacognitive Awareness in AI-Assisted Learning](https://edtechdev.github.io/aied/articles/metacognitive-awareness-experiential-vs-instructional/) A quasi-experimental, short-term longitudinal study with 126 first-year engineering students comparing two ways of teaching students how to learn with generative AI: an experiential, hands-on session versus a classical instructional lecture. Metacognitive awareness \u2014 both knowledge of cognition (understanding effective AI-use strategies) and regulation of cognition (applying that knowledge in practice) \u2014 was measured before and after a two-hour session and tracked longitudinally. The design directly addresses a gap flagged across the wiki's [[metacognition]] thread: interventions that build awareness of one's own AI-assisted thinking, rather than just policing AI use.\n\nThe study speaks to the [[cognitive-offloading]] and [[over-reliance]] literature by testing whether metacognitive skills for AI-assisted learning are better acquired by doing (experiencing GenAI's strengths and failure modes firsthand) or by being told. It extends [[ai-metacognition-stem-review]] with primary quasi-experimental evidence in an engineering-course context and connects to [[self-regulated-learning]] as a proximal training target for durable, transferable AI-use strategies. ## Connected Concepts - [[metacognition]] - [[cognitive-offloading]] - [[over-reliance]] - [[self-regulated-learning]] ## Connected Articles - [[ai-metacognition-stem-review]] ## Citation Benazet i Montobbio, P., Rotter, J., & Hern\u00e1ndez-Leo, D. (2026). [Experiential Versus Instructional Approaches for Eliciting Metacognitive Awareness in AI-Assisted Learning](https://arxiv.org/abs/2607.20047). arXiv:2607.20047. arXiv preprint (cs.CY/cs.HC). --- ## [A Taxonomy of Metacognitive Learning Scenarios in Professional Contexts: Integrating Systems Theory with Empirical Constraints](https://edtechdev.github.io/aied/articles/metacognitive-learning-scenarios-taxonomy/) # Metacognitive Learning Scenarios Taxonomy This paper addresses a fundamental gap in [[metacognition]] research: the lack of systematic integration of metacognitive theories into scenario taxonomies capable of guiding AI-enhanced professional development. By synthesizing four major theoretical frameworks into a six-node open systems model, the authors create a rigorous taxonomy of metacognitive learning scenarios. ## Systems Model & Scenario Generation The six-node open systems model (Environment, Input, Processes, Structures, Output, Feedback) was used for combinatorial enumeration, generating 216 mathematically possible learning scenarios. Four sequential constraint-based filters — psychological plausibility, educational relevance, measurement feasibility, and intervention potential — reduced this to **24 priority scenarios**. These 24 scenarios distribute across three developmental tiers: - **Novice** (6 scenarios) — foundational metacognitive awareness - **Developing** (10 scenarios) — active strategy use and monitoring - **Expert/Adaptive** (8 scenarios) — flexible, context-sensitive metacognitive control ## Theoretical Gaps Identified The analysis revealed critical gaps in current metacognition theory regarding dynamic reconfiguration of monitoring-control relationships across expertise levels, the role of feedback topology in metacognitive development, and trade-offs between internal integration and external connectivity. These gaps have direct implications for designing [[scaffolding]] in [[intelligent-tutoring]] systems and [[adaptive-learning]] platforms. ## AI-Enhanced Professional Development The taxonomy enables targeted, scenario-specific professional development interventions and generates testable predictions. It provides a structured foundation for AI systems that scaffold metacognitive growth in [[professional-training]] and [[lifelong-learning]] contexts. This complements work on [[self-regulated-learning]] by operationalizing the progression from novice to expert metacognitive functioning in ways that AI systems can track and support. ## Connected Concepts - [[metacognition]] - [[scaffolding]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[professional-training]] - [[lifelong-learning]] - [[self-regulated-learning]] ## Connected Articles - [[codify-socratic-programming-tutor]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[curiobot-llm-tutoring-exploratory-learning]] — Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning ## Citation Gibson, D. C., Azukas, M. E., & Yilmaz Soylu, M. (2026). [A taxonomy of metacognitive learning scenarios in professional contexts: Integrating systems theory with empirical constraints](https://arxiv.org/abs/2605.24142). arXiv:2605.24142. cs.HC. --- ## [Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning](https://edtechdev.github.io/aied/articles/metacognitively-discordant-completion-genai-2026/) > **Synthesis:** This theoretical paper names a state it calls *metacognitively discordant completion* (MDC): a learner submits correct, complete work while holding a first-person awareness that understanding has not actually arrived. Arguing that no existing literature holds the three defining conditions together under one name, the author builds the construct by inheritance from metacognition research and by boundary against related concepts, framing GenAI's role as amplification rather than invention. ## Key Findings 1. **The construct.** MDC is the experiential state of a learner who has invested cognitive effort, holds a formed first-person awareness that understanding has not occurred (judged against their own operative standard), and releases the completion anyway. The conjunction of the three conditions is the construct. 2. **Not new to AI.** A pre-AI genealogy across five literatures establishes that the dissociation between completing and understanding is old; four cuts separate MDC from withdrawn effort, justification discourse, fluency illusions, and preprint-stage constructs. 3. **Sibling construct and boundary.** A sibling construct, the Absent Cognitive Baseline, marks the case where no verdict can form; frameworks from epistemic akrasia to academic dishonesty each lack one element of MDC. GenAI's part is amplification, not invention. ## Implications For [[metacognition]] and [[self-regulated-learning]] research, MDC sharpens the distinction between completion and understanding in AI-mediated work, and connects to [[cognitive-offloading]] and [[over-reliance]] concerns. The paper argues that existing integrity and assessment frames each miss one element of the state, suggesting why simple policy responses fail. This is relevant to [[academic-integrity]] and [[student-experience]] discussions and to AI-literacy design: it reframes the risk of GenAI not primarily as cheating but as the routine, aware pass-through of non-understanding. As a conceptual contribution, the author assigns evidence questions to a planned interpretative phenomenological study with graphicacy as the anchoring domain. ## Connected Concepts - [[metacognition]] - [[self-regulated-learning]] - [[cognitive-offloading]] - [[over-reliance]] - [[academic-integrity]] - [[student-experience]] - [[generative-ai]] - [[ai-education]] ## Connected Articles - [[absent-cognitive-baseline-2026]] — Absent cognitive baseline - [[metacognitive-learning-scenarios-taxonomy]] — Metacognitive learning scenarios taxonomy - [[ai-fallibility-warning-help-seeking]] — AI fallibility warnings and help-seeking - [[polished-artifacts-fragile-engagement-2026]] — Polished artifacts, fragile engagement - [[trust-reliance-ai-education-2026]] — Trust and reliance in AI education ## Citation Jia, Y. (2026). [*Metacognitively discordant completion and the aware pass-through of non-understanding in generative AI learning*](https://osf.io/preprints/edarxiv/wzjxf_v1/). EdArXiv preprint. --- ## [Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI](https://edtechdev.github.io/aied/articles/mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai/) **Tomohiro Nagashima, Lisa Siegrist, Niklas Scholz, Shintaro Sato, Martina Vincoli, Man Su (2026)** As AI technologies enter [[k-12]] classrooms, understanding how different stakeholders perceive these tools is critical. This paper identifies a significant trust gap between [[teacher-role]] and [[student-experience]] perspectives on control and agency in classroom AI systems. Students generally desire greater autonomy and flexibility when interacting with AI tools, while teachers prioritize oversight, monitoring, and structured control to maintain pedagogical alignment. These misalignments have direct implications for [[ai-literacy]] programs that must address both stakeholder perspectives to build effective [[student-experience]] in learning environments. ## Key Findings - The researchers conducted a **speed-dating study using storyboards** with **16 school students and 15 school teachers in Germany** to investigate alignments and misalignments in views on student-AI decision-making control in K-12 classrooms. - Through **explicit pair-matching analysis**, they found that students and teachers had **misaligned views** on several key topics, including **how much they trust AI** and the **social and emotional aspects of student learning with AI**. - Students emphasized the importance of their **own decision making** and of **human teachers as gatekeepers** of the classroom, arguing that teachers have a "better connection" to students and understand them as whole persons, not just through information detected by systems. - Teachers also reported **not fully trusting AI's decision making**, citing the risk of **incorrect assessment of students** (e.g., assigning tasks that do not match actual knowledge levels), and stressed the importance of being able to **intervene and override AI's automatic assignments**. - Contrary to students' preference for human teachers, **many teachers worried that students would trust AI more than teachers themselves** — the mirror image of the students' stated view. ## Study Design & Method The speed-dating study paired participants with a series of storyboard scenarios depicting classroom AI systems, eliciting rapid, repeated judgments across design situations. The pair-matching analysis then explicitly compared teacher and student responses on shared topics, revealing where views aligned and where they diverged. The design foregrounds the classroom as a **multi-stakeholder environment** in which the same technology is perceived differently depending on role. ## Implications for AI in Education The misalignments identified in this study complicate the common assumption that more AI personalization is uniformly desirable. Because students and teachers differ on trust, control, and the social-emotional dimensions of learning with AI, classroom AI design must negotiate **competing preferences** rather than optimize for a single user. The findings also highlight that **teacher-student relationships outside of AI use** shape how both groups view the technology, suggesting that trust-building and communication — not just interface design — are needed to fill the perspective gap. This connects to [[human-in-the-loop-ai]] and to the design of [[intelligent-tutoring]] systems in [[k-12]] settings. ## Connected Concepts - [[k-12]] - [[teacher-role]] - [[student-experience]] - [[ai-literacy]] ## Connected Articles - [[teacher-authored-prompts-student-ai-dialogue]] — Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[modular-educational-llm-agency]] — Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistance ## Citation Tomohiro Nagashima, Lisa Siegrist, Niklas Scholz, Shintaro Sato, Martina Vincoli, Man Su (2026). [Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI](https://arxiv.org/abs/2607.01506). arXiv:2607.01506. --- ## [MindCopilot: Towards Formalizing and Evaluating Granular Human-LLM Co-Writing](https://edtechdev.github.io/aied/articles/mindcopilot-llm-co-writing/) MindCopilot introduces a formal framework for evaluating human-LLM co-writing that shifts from output-only metrics (BLEU, ROUGE) to **interaction-aware evaluation**. The paper models co-writing as a **Human-in-the-Loop Markov Decision Process (HiL-MDP)**, where writing is a sequence of granular decisions: accept, edit, or reject each AI suggestion. The **Co-Writing Fidelity Suite** introduces two novel metrics — **Hierarchical Acceptance Rate** (quantifying acceptance across granularity levels) and **Knowledge-aware Editing Distance** (measuring cognitive effort of editing suggestions). Evaluated across 1,688 queries from 16 writing domains, with a 30-participant validation study. For [[writing-education]], this represents a paradigm shift. Traditional automated writing evaluation focuses on final text quality, missing the process-level decisions that constitute real learning. The HiL-MDP framework aligns with process-oriented writing pedagogy — capturing where students accept, modify, or reject AI suggestions provides rich diagnostic information beyond what a final essay grade reveals. The work connects to [[cyberscholar-genai-writing-feedback]] and [[structured-llm-feedback-programming]] by establishing that interaction structure (when/how suggestions appear) significantly affects user behavior. For [[ai-peer-feedback-systems]], the acceptance/editing metrics offer a template for evaluating collaborative feedback workflows. Accepted to IJCAI 2026, this work bridges [[human-in-the-loop-ai]] theory with practical writing tool design, advancing [[llm]]-based educational technology beyond static quality assessment toward dynamic interaction modeling. ## Connected Concepts - [[writing-education]] - [[human-in-the-loop-ai]] - [[llm]] ## Connected Articles - [[cyberscholar-genai-writing-feedback]] - [[structured-llm-feedback-programming]] - [[ai-peer-feedback-systems]] ## Citation Youqing Fang, Yinhao Tang, Yanan Sun, Jiangning Liu, Ziyi Wang, Xun Zhao, Bin Liu, Weiming Zhang, Kuikun Liu, Wenwei Zhang, Kai Chen (2026). [MindCopilot: Towards Formalizing and Evaluating Granular Human-LLM Co-Writing](https://arxiv.org/abs/2605.23535). arXiv:2605.23535. IJCAI 2026. - [[humanlike-ai-collaborative-writing]] — Anthropomorphic design choices amplify both benefits and costs of AI co-writing --- ## [Cognitive Offloading in Student–AI Collaboration: A Longitudinal Analysis of Prompting Strategies](https://edtechdev.github.io/aied/articles/misiejuk-cognitive-offloading-prompting-2026/) > **Synthesis:** Misiejuk, López-Pernas, Kaliisa, and Saqr (2026) analyze 281 prompts from 122 student submissions across four assignments to examine how prompting strategies reveal cognitive offloading in student–AI collaboration. Using qualitatively coded prompts and Co-Occurrence Network Analysis (CNA), they found that high-quality submissions demonstrated cohesive prompting patterns integrating contextual details, instructions, and polite language — leading to fewer disagreements and more effective task guidance — while low-quality submissions were characterized by disagreement and direct instructions with limited contextualization. Notably, across both groups a convergence toward low-effort, direct instruction emerged, suggesting AI "leveled" achievement by encouraging cognitive laziness and reducing the incentive for deeper cognitive engagement. ## Key Findings - **281 prompts from 122 submissions** were analyzed across four assignments in a course where students used LLMs to generate social network datasets. - **High-quality submissions** showed cohesive prompting with strong *context–instruction–output-specification–polite-language* connections (e.g., context–instruction 0.25, output specification–instruction 0.28), with students providing domain knowledge and contextual details rather than delegating interpretation to the LLM — resulting in fewer disagreements. - **Low-quality submissions** were characterized by *disagreement–instruction* and *disagreement–output-specification* patterns with limited contextualization, indicating students issued orders and reacted negatively when AI didn't deliver, without contributing their own knowledge. - **Cognitive offloading was asymmetric:** reactive codes (disagreement, agreement) reflect higher offloading — the student reacts to AI output rather than directing the interaction with their own reasoning. - **A "leveling" tendency emerged:** across both quality groups, prompting converged toward low-effort, direct instruction over time; high achievers accepted AI output with minimal critical engagement, and low performers' inertia was reinforced. The convenience of AI reduced the incentive for refinement and deeper cognitive effort, effectively lowering the standard of engagement to the cohort's minimal common denominator. - **Longitudinal divergence:** the similarity between high- and low-quality prompting patterns was high in Assignments 1–3 (Pearson correlations 0.952, 0.935, 0.982) but dropped sharply in Assignment 4 (0.485), as disagreement-dominated patterns became more prominent in low-quality submissions. ## Study Design & Method This longitudinal study analyzed student–AI interactions in a semester-long social network analysis course at a Finnish university. Students used LLMs to generate their own network datasets meeting specified criteria across four assignments. A total of 281 prompts from 122 submissions were qualitatively coded into six binary categories: *Instruction*, *Context*, *Output specification*, *Disagreement*, *Agreement*, and *Polite language*. **Co-Occurrence Network Analysis (CNA)** was applied to map how prompt-code combinations clustered within and across submissions, with separate networks for high- and low-quality submissions, subtraction networks to quantify differentiating connections, and centrality analysis (in-strength, diffusion). Longitudinal networks tracked patterns across the four assignments, with Pearson/Spearman correlations quantifying convergence or divergence between the quality groups over time. ## Implications for AI in Education The study contributes to understanding [[cognitive-offloading]] as observable patterns in student prompting, not just a theoretical concern. It shows high-quality AI use integrates active cognitive engagement (contextual prompting, specifying expectations) rather than avoiding AI, connecting to [[genai-performance-vs-learning|distinguishing performance gains from learning]] and the finding that *how* students use AI matters more than *whether* they use it. The "leveling" tendency and cognitive-laziness finding support [[over-reliance]] research and [[cognitive-load-theory]]: excessive offloading risks diminishing the germane cognitive load needed for transferable mental models. The contextual-prompting finding supports [[scaffolding]] approaches that teach students to prompt with context, and the longitudinal dimension shows prompting strategies evolve with practice, informing [[ai-literacy]] curriculum design and [[learning-analytics]] approaches for monitoring student–AI collaboration. ## Limitations The study is context-specific (a social network analysis course, LLM-based dataset generation), and the prompting patterns may not generalize to other task types or disciplines. The "leveling" tendency could reflect the specific task design (minimal-effort tasks with plausible instant outputs) rather than a general characteristic of LLM-assisted work, as the authors acknowledge. The sample is 281 prompts from 122 submissions at a single university. Prompt quality was inferred from output quality, and the relationship between prompting behavior and learning outcomes is not directly causally tested. ## Connected Concepts - [[cognitive-offloading]] - [[prompt-engineering]] - [[ai-literacy]] - [[cognitive-load-theory]] - [[over-reliance]] - [[student-experience]] - [[higher-ed]] - [[scaffolding]] - [[learning-analytics]] - [[self-regulated-learning]] ## Connected Articles - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning - [[genai-can-harm-teaching-rct-2026]] — GenAI Tools: Improving Teaching Quality? - [[instructional-guidance-genai-learning]] — Instructional Guidance and GenAI Learning - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work - [[genai-meta-analysis-programming-learning]] — Meta-Analysis of GenAI in Programming ## Citation Misiejuk, K., López-Pernas, S., Kaliisa, R., & Saqr, M. (2026). [*Cognitive offloading in student–AI collaboration: A longitudinal analysis of prompting strategies*](https://doi.org/10.1016/j.chbr.2026.101130). *Computers in Human Behavior Reports*, 22, 101130. --- ## [Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning](https://edtechdev.github.io/aied/articles/mixed-reality-engineering-learning/) > **Mohammad Abu Nasir Rakib, Sharmin Akter, Eshwara Prasad Sridhar, Somik Biswas, Md Rassel Raihan, Mahmudur Rahman** — submitted 1 Jul 2026 **Mohammad Abu Nasir Rakib, Sharmin Akter, Eshwara Prasad Sridhar, Somik Biswas, Md Rassel Raihan, Mahmudur Rahman** — submitted 1 Jul 2026 User study with 24 participants comparing classroom instruction, mixed-reality apps, and physical toolkits for Engineering Mechanics reveals multimodal learning improves engagement but complex visualizations remain challenging. ## Key Contributions - User study with 24 participants comparing classroom instruction, mixed-reality apps, and physical toolkits for Engineering Mechanics reveals multimodal learning improves engagement but complex visualizations remain challenging. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[cogtax-cognitive-taxonomy]] — CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education - [[less-deliberate-teams-llm]] — Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work - [[dynamic-skill-matching-capstone-teams]] — Improving Capstone Team Outcomes through Dynamic Skill Matching and Preference Alignment - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System ## Citation Mohammad Abu Nasir Rakib, Sharmin Akter, Eshwara Prasad Sridhar, Somik Biswas, Md Rassel Raihan, Mahmudur Rahman (2026). [Visualizing Engineering Fundamentals: Design of Mixed Reality and Physical Toolkits for Effective Learning](https://arxiv.org/abs/2607.00979). arXiv:2607.00979. submitted 1 Jul 2026 --- ## [Fostering machine learning literacy in senior primary education: Evaluating a structured pedagogical course design](https://edtechdev.github.io/aied/articles/ml-literacy-primary-education/) > **Synthesis:** Fostering machine learning literacy in senior primary education: Evaluating a structured pedagogical course design ## Key Findings - The authors designed and evaluated a **six-to-eight-hour machine learning course** for **752 senior primary students** (average age 11.36) across **seven Hong Kong primary schools**, targeting conceptual ML understanding rather than tool usage. - **Pre- and post-test results showed significant improvement in ML understanding**: the concept-test mean rose 65% (4.57 → 7.53) and the median doubled (4.00 → 8.00), with a **Wilcoxon signed-rank effect size of r = 0.55** (Z = −14.206, p < .001) across 679 matched pairs (pre M = 4.57, SD = 1.88; post M = 7.48, SD = 3.26). - Students came to comprehend **supervised learning and reinforcement learning**, including algorithms such as **k-nearest neighbours (KNN)** and **artificial neural networks (ANN)**, through **training robots in competitive circuit tasks** and **real-time algorithm visualization**. Item-level effect sizes ranged from 0.24 to 0.48, with the largest gains on items about ML paradigms, the ML workflow, and ANN structure (correctness rising from below 30% to nearly 60%); the hardest item — selecting optimal ML approaches for novel problems under resource trade-offs — reached only 36% correctness. - Thematic analysis of student interviews — 15 semi-structured focus groups with 47 randomly selected students across all seven schools — revealed that the structured pedagogical approach, blending **guided worksheets, hands-on activities, and iterative refinement** of data processing, parameter adjustment, and model training within the robots' software, enhanced both conceptual learning and engagement; "Development of Conceptual Understanding" was the most prevalent theme (31.38% of all coded segments). - Surprisingly, students developed **initial reflections on distinguishing between AI and human learning**, an early metacognitive outcome of the course. ## Study Design & Method The course design responds to a gap in K-12 AI literacy education, which the authors argue emphasizes tool usage over fundamental concepts even though AI literacy requires understanding how and why AI works. The structured approach combines guided worksheets with hands-on robot interaction, letting students experience the ML pipeline directly: they adjust data processing, tune parameters, and iteratively train models within the robots' software. The mixed-methods evaluation paired a pre/post concept test administered to the full cohort (Npre = 696; Npost = 727; 679 matched pairs analyzed with the non-parametric Wilcoxon signed-rank test) with 15 semi-structured focus group interviews (three to five students each, conducted via Zoom for about 30 minutes, with a course-content recap to mitigate recall bias) analyzed thematically using a codebook approach. There was no significant gender difference in initial knowledge (p = .270), with only a non-significant post-test trend (p = .051). ## Implications for AI in Education The results demonstrate the **feasibility and promise of teaching fundamental ML concepts to senior primary students at scale** through a structured course design. For educators, the study provides actionable guidance on how to allocate time and resources: a relatively short course can produce meaningful conceptual gains when hands-on training tasks are paired with structured scaffolding. The emergence of student reflections on AI-versus-human learning suggests early ML education can seed [[computational-thinking]] and critical dispositions toward AI, supporting [[ai-literacy]] goals in [[k-12]] education. The authors further suggest that an eight-to-ten-hour intervention might be optimal for deeper exploration, and that future iterations should adopt spiral-curriculum designs — combining concepts such as KNN and ANN in cross-concept challenges (e.g., choosing and defending the best algorithm for a new racing track) rather than teaching them as isolated lessons — an [[instructional-design]] refinement grounded in the observed "cognitive boundaries" between discrete knowledge and systematic integration. ## Limitations The authors identify several design constraints: the one-group pre-/post-test design lacks a control group, so external factors cannot be entirely ruled out; the use of identical test items for both assessments may have introduced testing effects that inflated post-test scores; variation in instructional duration (six to eight hours across schools) created an inconsistent intervention "dose," potentially affecting the depth of conceptual development; and the reliance on retrospective focus group interviews limits understanding of engagement to student perceptions rather than real-time behavioral data. ## Connected Concepts - [[ai-literacy]] - [[computational-thinking]] - [[instructional-design]] - [[k-12]] - [[teacher-role]] - [[faculty-development]] - [[affective-tutoring]] - [[automated-essay-scoring]] ## Connected Articles - [[computational-thinking-ai-agent-creation]] — Computational Thinking Development in AI Agent Creation: A Mixed-Methods Study - [[ai-literacy-continuum-higher-education]] — Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education - [[teacher-education-ai-literacy-sdt-2026]] — Teacher education for artificial intelligence literacy through a self-determination theory perspective - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 ## Citation Kong, S. C., & Wang, Q. (2026). [*Fostering machine learning literacy in senior primary education: Evaluating a structured pedagogical course design*](https://doi.org/10.1016/j.caeai.2026.100631). --- ## [Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy](https://edtechdev.github.io/aied/articles/mllm-scientific-visualization-literacy/) Multimodal large language models (MLLMs) are increasingly used to interpret visualizations, yet most evaluations remain chart-centric and offer limited insight into **scientific visualization (SciVis) literacy**. This study benchmarks six MLLMs (three closed-source, three open-source) on a standardized SciVis literacy assessment — 49 items spanning 18 scientific visualizations, 8 techniques, and 11 task types — and compares model performance against data from 485 human participants. Results show MLLMs do **not** exhibit uniform SciVis literacy. Gemini is the strongest model overall, exceeding the human mean on several evaluated subsets, while all open-source models fall below the human baseline. Performance is highly uneven across techniques and tasks: models do best on scientific illustration, search, and spatial understanding, but struggle on texture-based and integration-based visualizations and on fine-grained quantitative estimation. For [[stem-education]], this delineates where AI can responsibly support interpretation of scientific figures versus where it remains unreliable. The work contributes a reusable benchmark methodology and underscores that current [[generative-ai]] multimodal systems should not be treated as substitutes for human [[ai-literacy]] in reading scientific visualizations — a finding relevant to assessment design in [[higher-ed]] and to [[formative-assessment]] of visualization competence. ## Connected Concepts - [[stem-education]] - [[generative-ai]] - [[ai-literacy]] - [[higher-ed]] - [[formative-assessment]] ## Connected Articles - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education ## Citation Patrick Phuoc Do, Chau M. Ta, Chaoli Wang (2026). [Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy](https://arxiv.org/abs/2607.15176). arXiv:2607.15176. --- ## [Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistance](https://edtechdev.github.io/aied/articles/modular-educational-llm-agency/) The widespread adoption of AI chatbots in education will drastically change learning, making responsible deployment a critical concern. While large language models (LLMs) might have access to sources discussing insights from educational sciences, they are not particularly inclined to adhere to pedagogical concepts, risking negative effects on the learning process, such as a loss of transfer capabilities, critical thinking, or creativity. In this paper, we introduce an agentic AI chatbot architecture assisting students with exercise solving, specifically designed to contribute to more responsible AI use in education. We base our conceptual development on the identification of several desiderata for responsible LLM-based educational systems, argue for the structural shortcomings inherent in monolithic, out-of-the-box solutions, and instead suggest modularizing the agentic architecture. - **Modular over monolithic:** Gabelmann et al. argue that monolithic, out-of-the-box LLM solutions structurally impede responsible deployment in education because LLMs lack inherent pedagogical discipline. They propose modularizing the agentic architecture into stage-specific components for exercise solving, each capable of incorporating targeted pedagogical advice. - **Pedagogical desiderata:** The paper derives formal requirements (desiderata) for responsible LLM-based educational systems, arguing that systems must be controllable, transparent, and overseeable. This directly addresses [[over-reliance]] risks identified in studies showing that brief AI assistance impairs independent performance. - **Stage-specific scaffolding:** The modular architecture decomposes exercise-solving assistance into distinct stages — each module can enforce [[scaffolding]] strategies appropriate to that stage, preventing the [[correct-answer-trap-ai-tutor]] where LLM tutors fail to detect misconceptions when students arrive at correct answers via flawed reasoning. - **Accepted at AISoLA 2025** under the Responsible and Trusted AI track, positioning this work at the intersection of formal methods for AI safety and [[intelligent-tutoring]] system design. The modular approach resonates with calls for [[agentic-literacy-debt]] frameworks that ensure agent deployments are accompanied by appropriate governance. - **Teacher-in-the-loop design:** The architecture preserves a [[teacher-role]] by making the system's pedagogical decisions observable and auditable, rather than black-box. This connects to work on [[rubric-aware-grading-rec-cbm]] which similarly emphasizes interpretability in educational AI. ## Connected Concepts - [[over-reliance]] - [[scaffolding]] - [[intelligent-tutoring]] - [[teacher-role]] ## Connected Articles - [[correct-answer-trap-ai-tutor]] - [[agentic-literacy-debt]] - [[rubric-aware-grading-rec-cbm]] ## Citation Julius Gabelmann, Felix Jahn, Kevin Baum, Sophie van Rossum, Emely Wuenscher, Timo P. Gros, & Verena Wolf (2026). [Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistance](https://arxiv.org/abs/2605.30187). arXiv:2605.30187. AISoLA 2025 (Track: Responsible and Trusted AI). --- ## [From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents](https://edtechdev.github.io/aied/articles/mooc-to-maic/) > **A new paradigm for online education replacing MOOCs with LLM-driven multi-agent AI classrooms**, piloted at Tsinghua University with 100K+ learning records from 500+ students. MAIC uses specialized agents (Teacher, Assistant, Classmate, Analyzer) to deliver personalized, adaptive learning at scale. **ArXiv:** 2409.03512 **Submitted:** September 2024 ### Overview MAIC (Massive AI-empowered Course) proposes a new form of online education that addresses the fundamental tension between **scalability** (MOOC's strength) and **adaptivity** (MOOC's weakness). Traditional MOOCs serve thousands of learners through one pre-recorded video, struggling to personalize instruction. MAIC replaces this with an LLM-driven [[agentic-ai|multi-agent]] system that constructs an AI-augmented classroom, shifting from "one video for N students" to "N agents for 1 student". ### Architecture MAIC deploys a suite of specialized AI agents: - **Teacher Agent:** Delivers lectures and core instruction - **Assistant Agent:** Provides personalized offline mentoring and exercises - **Classmate Agents:** Engage in peer-like dialogue to stimulate discussion and questions - **Analyzer Agent:** Diagnoses student performance from quiz results and recommends prerequisite learning paths ### Key Innovations 1. **Unified LLM Foundation:** Unlike prior systems that used separate models for recommendation, dialogue, and assessment, MAIC builds all agents on a shared LLM foundation, enabling deeper integration across teaching and learning tasks. 2. **Course Creation at Scale:** The system can generate full course materials (slides, textbooks, exercises, videos) from instructor-provided course descriptions. Traditional MOOC production costs ~25K USD and 60 hours per course; MAIC reduces this to under 2 USD and 30 minutes. 3. **Personalization Loop:** Students receive adaptive quizzes, personalized feedback, and dynamically recommended learning paths based on their performance patterns. ### Pilot at Tsinghua University - **100,000+ learning records** from over **500 students** - Initial observations suggesting improved engagement compared to traditional MOOC formats ### Significance MAIC represents a convergence point for [[generative-ai]], [[rag]], and [[agentic-ai]] in education. It demonstrates how LLM-driven multi-agent systems can transform the MOOC paradigm from one-size-fits-all broadcasting to truly adaptive, personalized [[intelligent-tutoring]] at scale. The platform vision extends toward an open collaborative hub for AI-driven education research. ## Connected Concepts - [[agentic-ai]] - [[generative-ai]] - [[rag]] - [[intelligent-tutoring]] ## Connected Articles - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[lecturaagents-multi-agent-teaching]] — LecturaAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System ## Citation Yu, J., Zhang, Z., Zhang-li, D., Tu, S., Hao, Z., Li, R., ... Liu, Z., Liu, H., & Sun, M. (2024). [*From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents*](https://arxiv.org/abs/2409.03512). arXiv:2409.03512. --- ## [From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle](https://edtechdev.github.io/aied/articles/moodle-ai-tutoring-deep-learning/) > Ostrowska, Kukla & Majstrak (2026) present an AI tutoring system **integrated into the Moodle LMS** designed to scaffold students from surface-level fact recall to deep conceptual understanding through adaptive questioning and feedback. # AI Tutoring for Moodle: Surface to Deep Learning ## Core Contribution Ostrowska, Kukla & Majstrak (2026) present an AI tutoring system **integrated into the Moodle LMS** designed to scaffold students from surface-level fact recall to deep conceptual understanding through adaptive questioning and feedback. ## How It Works The system operates within Moodle's existing infrastructure, using LLM-based tutoring to: - Detect the **depth of student understanding** from their responses - Adaptively scaffold toward deeper comprehension via [[scaffolding]] techniques - Move students through [[formative-assessment]] cycles that promote conceptual mastery This **grounded approach** — embedding AI tutoring in an existing LMS rather than building standalone tools — addresses deployment barriers identified in the [[ai-tutor-effectiveness-review]]. Many intelligent tutoring systems fail to achieve real-world impact because they require new infrastructure; Moodle integration lowers the adoption threshold. ## Connections to the Wiki The system's focus on deep vs. surface learning connects to [[metacognition]] research — students must recognize when they have only surface understanding. The adaptive approach aligns with [[adaptive-learning]] but emphasizes **qualitative shifts** in understanding rather than quantitative difficulty adjustment. The Moodle deployment strategy echoes lessons from [[ai-peer-feedback-systems]] (AICoFe) about integrating AI tools into existing educational workflows. Unlike [[tutoring-specific-vs-general-ai]] debates about specialized tutors, this system shows how general LLMs can be **scaffolded into tutoring roles** within familiar platforms. The focus on deep understanding complements [[pedagogy-ai-mistakes]] work on using AI errors for higher-order thinking. ## Open Questions - How does in-LMS tutoring compare to standalone systems in learning outcomes? - Can the deep learning scaffolding generalize across disciplines beyond the tested contexts? - What is the instructor's role when AI handles adaptive questioning within Moodle? ## Connected Concepts - [[scaffolding]] - [[formative-assessment]] - [[metacognition]] - [[adaptive-learning]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[ai-peer-feedback-systems]] - [[tutoring-specific-vs-general-ai]] - [[pedagogy-ai-mistakes]] ## Citation Majstrak, A.A.O.M.K.G., Understanding:, F.S.L.T.D., Moodle, A.G.A.T.S.F., Skwarek, A.O.M.K.G.M.J.O.S.P.J., Wróblewska, A., Warsaw, F.O.M.A.I.S., scalability, R.G.L.E., & (LMS), M.S. (2026). [*From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle*](https://arxiv.org/abs/2605.06963) --- ## [Navigating the moral panic: encouraging appropriate use of GenAI in the classroom rather than condemning innovation as disruption](https://edtechdev.github.io/aied/articles/moral-panic-genai-classroom/) > **Jennifer M. Krebsbach & Victoria L. Cross (University of California, Davis)** — *Assessment & Evaluation in Higher Education* (Taylor & Francis). Open Access, CC BY 4.0. doi:10.1080/02602938.2026.2686727. **Jennifer M. Krebsbach & Victoria L. Cross (University of California, Davis)** — *Assessment & Evaluation in Higher Education* (Taylor & Francis). Open Access, CC BY 4.0. doi:10.1080/02602938.2026.2686727. ## Summary A **natural-experiment / design-based study** tracking eight iterations of a lower-division *Data Visualisation in the Social Sciences* course (n = 921 across six years) to test whether — and how — GenAI changes student learning. The authors frame faculty anxiety about GenAI as the latest in a series of "moral panics" (calculators, word processors, search engines, e-learning) and argue the productive response is to **teach and embed GenAI use**, not ban it. They compare three instructional conditions on two quiz types (knowledge vs. applied): - **pre-GenAI** (2019–2020, n = 3 cohorts) - **GenAI-available** (2023–2024, n = 3) — GenAI present but no pedagogical adaptation; some students used it, often ineffectively/unethically - **GenAI-integrated** (2025, n = 2) — explicit instruction + encouragement to use GenAI on the *applied* portion; GenAI banned on the *knowledge* portion (paper quiz) ## Method (key design) - Quizzes 3–6 analysed (first two dropped as orientation; Quiz 7 dropped as low-stakes). Item-level performance (% correct = difficulty; SD = variability) from the LMS. - **3 × 4 mixed-design ANOVA**: AI-availability (between, 3 levels) × quiz number (within, 4 levels). Small per-condition N (2–3 cohorts), so effect sizes (ω²) reported as the primary evidence. ## Key Findings ### Applied questions — "available" hurt, "integrated" recovered - **Main effect of GenAI availability:** F(2,5) = 5.85, p = 0.049, **ω² = 0.35** (GenAI availability accounts for 35% of variance in applied-question performance). - In the **GenAI-available** condition, applied performance was **significantly lower** than baseline on Quizzes 4, 5, 6. Because applied questions could *not* be answered directly by GenAI, the drop indicates students were **less prepared** — either unable to use GenAI effectively or unable to critically evaluate its output. - In the **GenAI-integrated** condition, applied performance **returned to ~pre-GenAI levels** (and exceeded baseline on one harder quiz). Teaching students to use GenAI for data summarising levelled the field. ### Knowledge questions — availability masked cheating; paper quiz revealed a deficit - Same main effect, **ω² = 0.35**. Knowledge performance stayed at baseline during GenAI-available, then **dropped below baseline** once delivered on paper in the integrated condition. - The authors interpret the *stable central tendency but elevated variability* during GenAI-available as evidence that **some students unethically used GenAI** to boost knowledge scores (heterogeneous use masked underlying learning differences). Moving knowledge quizzes to paper removed that opportunity and exposed that integrated-cohort students were **less prepared** — plausibly from over-reliance on GenAI to summarise content. ### Variability — the headline signal - **Applied-question variability:** F(2,5) = 64.84, p < 0.001, **ω² = 0.88** — GenAI availability accounted for **88%** of variability. Variability spiked in the GenAI-available condition, returned to baseline under integration. - **Knowledge-question variability:** availability × time interaction F(6,15) = 11.65, p < 0.001, ω² = 0.57; main effect ω² = 0.78. GenAI availability accounted for **78%** of the increase. Paper delivery produced the **lowest, most stable variability** → read as greater **equity** in the classroom. ### Student feedback (pilot, n = 28/149 responded) Mixed: 53% preferred the new split format (paper knowledge + take-home applied); common praise was reduced stress and more active calculation. Others preferred the old 25-min efficiency. ## Interpretation: design beats ban The integrated redesign **resolved both academic-integrity and authenticity concerns** by splitting the quiz: a high-integrity paper knowledge test + a high-authenticity, open-resource applied task where GenAI use was taught. The authors caution they likely over-learned the "moral panic" lesson — assuming universal, effective GenAI adoption — when in reality uptake was partial and often ineffective. Their conclusion: monitor our own hypotheses about student GenAI use, keep learning objectives central, and **design authentic assessments for the new environment** rather than condemn the technology. ## Connected Concepts - [[plagiarism-detection]] - [[reducing-ai-misuse]] - [[automated-essay-scoring]] - [[student-experience]] - [[ai-misuse-learning-harm]] - [[ai-literacy]] - [[student-misconceptions-ai]] - [[higher-ed]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights - [[genai-skill-bypass-literacy]] — The GenAI Skill Bypass: Mapping Divergent Pathways of University Students and Staff AI Literacy ## Citation Krebsbach, J. M., & Cross, V. L. (2026). Navigating the moral panic: encouraging appropriate use of GenAI in the classroom rather than condemning innovation as disruption. [Assessment & Evaluation in Higher Education](https://www.tandfonline.com/doi/full/10.1080/02602938.2026.2686727). https://doi.org/10.1080/02602938.2026.2686727 --- ## [MotiBo: The Impact of Interactive Digital Storytelling Robots on Student Motivation Through Self-Determination Theory](https://edtechdev.github.io/aied/articles/motibo-digital-storytelling-robots-motivation-2026/) > **Synthesis:** Fung and Lui (2026) examine the impact of MotiBo, an interactive digital storytelling system incorporating a human-like robot, on student engagement and creativity. Recognizing that storytelling can enhance motivation and engagement but that conventional methods often lack interactive elements, the study compares engagement across three modalities: paper-based, PowerPoint, and robot-assisted (MotiBo) storytelling. Using a quasi-experimental design with three groups of students over a five-day learning period, the study finds that students using MotiBo exhibit statistically significant improvements in behavioural and cognitive engagement compared with traditional methods. ## Key Findings - **Creativity is increasingly recognized as an important skill in education**, and storytelling can enhance motivation and engagement — but conventional storytelling methods often lack interactive elements. - **MotiBo is an interactive digital storytelling system incorporating a human-like robot** to add interactivity. - A **quasi-experimental design** compared engagement across three modalities: paper-based, PowerPoint, and robot-assisted (MotiBo) storytelling, with three groups of students over a five-day learning period. - Students using **MotiBo showed statistically significant improvements in behavioural and cognitive engagement** compared with traditional methods. ## Study Design & Method This is a **quasi-experimental study**. The researchers compared student engagement and creativity across three storytelling modalities: paper-based, PowerPoint-based, and robot-assisted storytelling using the MotiBo system (a human-like interactive digital storytelling robot). Three groups of students participated over a five-day learning period. The study measured behavioural and cognitive engagement, and considered creativity as an outcome, analyzing differences across the three conditions. ## Implications for AI in Education MotiBo demonstrates how [[social-robots]] can enhance [[student-engagement]] and [[motivation]] in storytelling-based learning, framed through [[self-determination-theory]]. It shows that adding an interactive, embodied robot to storytelling increases behavioural and cognitive engagement compared with paper and PowerPoint methods. This connects to [[educational-robotics]], [[storytelling-in-education|narrative learning]], and the promotion of [[creativity]] in education. For educators, it provides evidence that embodied, interactive digital storytelling can motivate learners, with implications for [[k-12]] and special/inclusive education settings. ## Limitations The study is quasi-experimental (non-random assignment) with a five-day learning period, limiting causal inference and insight into long-term effects. The specific sample and student population are not detailed in the abstract. Findings focus on engagement measures, and the relationship to deeper learning outcomes and creativity requires further investigation. ## Connected Concepts - [[social-robots]] - [[educational-robotics]] - [[motivation]] - [[self-determination-theory]] - [[student-engagement]] - [[storytelling-in-education]] ## Connected Articles - [[icub-humanoid-storytelling-llm-hri-2025]] — LLM-Powered Narrative HRI - [[robobuddy-llm-social-robots-classroom-2025]] — RoboBuddy: LLM-Powered Social Robots - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind: Robot-Mediated Role-Play ## Citation Fung, K. Y., & Lui, T. L. R. (2026). *MotiBo: The impact of interactive digital storytelling robots on student motivation through self-determination theory*. arXiv:2601.01218. --- ## [Multi-Agent Systems for Instructional Design](https://edtechdev.github.io/aied/articles/multi-agent-instructional-design/) > Embedding the Knowledge–Learning–Instruction (KLI) framework into multi-agent systems to act as sophisticated instructional designers for K-12 educators. ## The Prompting Gap in K-12 Education **Wang et al. (2025)** address a critical problem: **60% of K-12 teachers** now use AI for teaching tasks, but most lack the expertise to prompt effectively for high-quality instructional materials. ### The Problem - **Commercial LLMs** (ChatGPT, Gemini) lack embedded pedagogical theory - **Simple prompts** produce shallow, misaligned, or inconsistent materials ("a mile high and an inch deep") - **Advanced prompt engineering** requires time/expertise most teachers don't have ### The Solution Shift pedagogical expertise from **user's prompt** to **LLM's internal architecture** by embedding learning sciences principles directly into multi-agent systems. ## Three Systems Compared | System | Architecture | KLI Integration | Teacher Preference | |--------|--------------|-----------------|-------------------| | **SAS** (Single-Agent) | Baseline: simple "Generate learning activity" prompt | None (naive) | Lowest: "needs lots of editing" | | **MAS-Roles** | 5-agent sequential pipeline (KC → Learning → Instruction → Design → Feedback) | Embedded in agent roles | Middle: "generic, impersonal" | | **MAS-CMD** | 3-agent collaborative with conquer-merge discussion (Behaviorist, Constructivist, Aesthetic personas) | Collaborative discussion | **Highest: "fantastic, creative, classroom-ready"** | ## Key Findings ### Quantitative Results (QM K-12 Rubric) - **Small differences:** Statistical significance only on criterion 5.2 (p = .014) - **Low expert agreement:** Fleiss' κ ≈ 0 (poor to slight agreement across all criteria) - **Trend:** MAS-CMD > SAS > MAS-Roles (but not always statistically significant) ### Qualitative Results (20 Practicing Teachers) **MAS-CMD praised for:** - **Creativity:** "Fantastic ideas," "so creative" - **Real-world context:** Urban planning theme made geometry "meaningful" - **Completeness:** Included worksheets, exit tickets, teacher dialogue **Common critique:** Pacing issues ("a lot to unpack," needs to be "broken up into some days") ### Efficiency Trade-off | System | Time (s) | Tokens (k) | Requests | |--------|-----------|-------------|----------| | SAS | 25 ± 16 | ~3.4 | 1 | | MAS-Roles | ~75 | ~24 | 5 | | MAS-CMD | 272 ± 168 | ~71.6 | 13 | **Finding:** Highest quality requires ~11× more tokens and ~10× more time than baseline. ## Knowledge–Learning–Instruction (KLI) Framework The KLI framework (Koedinger et al., 2012) aligns three elements: 1. **Knowledge Components (KCs):** Facts, concepts, principles, procedures 2. **Learning Processes:** Memory/fluency, induction/refinement, understanding/sense-making 3. **Instructional Principles:** Spacing, testing, worked examples, prompted self-explanation **Embedding KLI into MAS:** - **MAS-Roles:** Sequential pipeline where each agent embodies one KLI component - **MAS-CMD:** Collaborative discussion among agents with different pedagogical personas ## Connection to Existing Work ### vs. [[principled-ai-education|Principled AI Education]] - Both emphasize: **augment, don't displace** learning sciences with AI - KLI framework operationalizes "educational goals → learning models → technology" - **Complementary:** Principled AI sets the framework; this paper implements it via MAS ### vs. [[agentic-workflows-education|Agentic Workflows]] - **Multi-agent collaboration:** MAS-CMD's conquer-merge discussion exemplifies agentic **planning + reflection** - **Teacher as beneficiary:** Unlike agentic tutoring (where students interact with agents), here agents serve *teachers* as instructional design assistants - **Scalability:** MAS offers "scalable path for creating high-quality educational content" ### vs. [[teachbench-llm-teaching-evaluation|TeachBench]] - **Complementary paradigms:** TeachBench evaluates *teaching ability*; this paper *designs instructional materials* - **Both use agents:** TeachBench (teacher-student agents); this paper (instructional designer agents) - **Both find:** Raw solving ≠ pedagogical quality; structured frameworks needed ## Implications for AI in Education ### For Tool Developers - **Embed pedagogy:** Don't rely on teachers to "prompt in" pedagogical quality - **Multi-agent > single-agent:** Collaborative discussion produces more creative, contextual outputs - **Trade-off awareness:** Highest quality requires computational cost; offer tierd options ### For K-12 Educators - **Beyond chatbots:** Specialized MAS tools can provide "print-ready" materials - **Teacher remains central:** AI as "instructional partner," not replacement - **Time savings:** Weekly AI users save ~5.9 hours/week; quality tools amplify this ### For Researchers - **Evaluation challenge:** Low expert agreement (κ ≈ 0) suggests need for better rubrics - **Quality ≠ rubric scores:** Teachers' qualitative preferences didn't always match statistical significance - **KLI as design principle:** Learning sciences frameworks can be operationalized in AI architectures ## Connected Concepts - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[help-seeking]] - [[metacognition]] - [[agentic-ai]] - [[reducing-ai-misuse]] - [[ethics]] - [[affective-tutoring]] ## Connected Articles - [[principled-ai-education]] - [[agentic-workflows-education]] - [[teachbench-llm-teaching-evaluation]] ## Citation Wang, J., Xiao, R., Hou, X., & Stamper, J. (2025). [*Enabling Multi-Agent Systems as Learning Designers: Applying Learning Sciences to AI Instructional Design*](https://arxiv.org/abs/2508.16659). arXiv:2508.16659. --- ## [Beyond the AI Tutor: Social Learning with LLM Agents](https://edtechdev.github.io/aied/articles/multi-agent-llm-social-learning/) ## Summary Most AI-based educational tools adopt a one-on-one tutoring paradigm, pairing a single LLM with a single learner. Yet decades of learning science — from Vygotsky's Zone of Proximal Development to Bandura's Social Learning Theory — suggest that multi-party interaction, through peer modeling, co-construction, and exposure to diverse perspectives, produces learning benefits that dyadic tutoring alone cannot. This paper reports two controlled experiments testing whether multi-agent LLM configurations can deliver those collaborative and observational benefits at scale. In a convergent math problem-solving study (N=315), learners who interacted with both an LLM tutor and error-prone LLM peers achieved the highest unassisted test accuracy. In a divergent composition study (N=247), a two-agent Duo of distinct frontier models matched the quality gains of single-model assistance while restoring ideational diversity to baseline, avoiding the idea-level homogenization that single-model support produced. The findings offer one of the first controlled investigations of multi-agent LLM learning environments. ## Key Contributions - In a **convergent problem-solving study (N=315)**, participants tackled SAT-level math problems in a 2×2 design varying the presence of an LLM tutor and error-prone LLM peers (making conceptual vs. arithmetic errors); those who interacted with **both a tutor and peers achieved the highest unassisted test accuracy**, suggesting peer observation has independent learning value even when an expert is present. - In a **divergent composition study (N=247)**, participants wrote argumentative and creative essays with no AI assistance, a single LLM (Claude or ChatGPT), or **both Claude and ChatGPT together**; while both LLM conditions improved essay quality, **only the two-agent condition avoided the idea-level homogeneity** that single-model assistance produced. - The writing results were statistically robust: an OLS model predicting combined essay quality from condition was significant (F(2,244)=21.91, p<.0001, R²=0.152), with both LLM conditions outperforming control (Control vs. Single: Δ=0.332, t(244)=6.21, p<.0001; Control vs. Duo: Δ=0.234, t(244)=4.21, p=0.0001), while Single and Duo did not differ significantly. - Idea homogeneity across participants was measured via bootstrapped similarity scores with permutation testing over 1,000 rounds, showing the **multi-model Duo condition restored ideational diversity to baseline levels**. - Across both studies, **peer-like configurations supported learner self-efficacy more than single authoritative agents did**, even when they produced lower objective performance — promising for learners whose primary barrier is motivation rather than knowledge. ## Study Design & Method The two experiments span the convergent–divergent spectrum of learning tasks (Guilford's distinction between arriving at one correct solution and generating multiple valid ones). **Experiment 1** is a pre-registered 2×2 between-subjects design on SAT-level math, crossing the presence of an LLM tutor with error-prone LLM peers. Three GPT-5.2 agents played distinct roles: **Bob**, a supportive tutor who guided with hints, clarifications, and follow-up questions rather than answers (consistent with Socratic tutoring); **Alice**, an arithmetic-error peer with strong conceptual understanding; and **Charlie**, a conceptual-error peer with accurate arithmetic. The two peers were designed to reflect the two most common categories of student math mistakes, creating opportunities for the diagnostic reasoning and schema conflict that learning-from-errors research predicts should deepen understanding. Each problem used an isomorphic variant in the lesson and test phases to control for item-specific memorization, and the primary outcome was unassisted accuracy on the test variants after a brief distractor. **Experiment 2** contrasts no AI, a single LLM, and a two-agent Duo (GPT-5.2 + Claude Opus 4.6) with complementary role specializations (e.g., one agent emphasizing imagination and voice, the other craft and structure), scoring essays by rubric validated against human raters (r=.87) and computing cross-participant idea similarity via SBERT embeddings with bootstrapped permutation testing. ## Implications for AI in Education The findings suggest that one-on-one AI tutoring is not the only effective paradigm: **orchestrating multiple LLM agents — as peers alongside a tutor or as role-specialized collaborators — can add measurable learning value**, particularly by restoring ideational diversity that single-model assistance suppresses. For [[ai-tutoring]] and [[collaborative-learning]] system design, this points toward multi-agent configurations in which learners observe peer agents struggle, disagree, and co-construct understanding, rather than receiving authoritative answers from a single assistant. Two design-relevant patterns stand out. First, **homogenization is not an inherent cost of AI assistance but a function of how agents are deployed** — a direct implication for [[writing-education]], where offering multiple architecturally distinct models can preserve ideational diversity while maintaining quality. Second, the observed **error-salience asymmetry** — participants rated the arithmetic-error peer (Alice) more harshly than the conceptual-error peer (Charlie) on nearly every dimension, calibrating trust by the observability of errors rather than their severity — is an important consideration for designing pedagogical peer agents. The self-efficacy benefit of peer-like agents also speaks to [[motivation]] and [[self-regulated-learning]]: configurations that support learners whose primary barrier is confidence rather than knowledge. ## Connected Concepts - [[ai-tutoring]] - [[collaborative-learning]] - [[scaffolding]] - [[writing-education]] - [[self-regulated-learning]] - [[motivation]] ## Connected Articles - [[collaborative-ai-tutoring]] - [[self-efficacy-tutoring-learning]] - [[ai-tutor-effectiveness-review]] - [[humanlike-ai-collaborative-writing]] ## Citation Kumar, H., Mu, Z. K. (J.), Vincentius, J., & Anderson, A. (2026). [*Beyond the AI Tutor: Social Learning with LLM Agents*](https://arxiv.org/abs/2604.02677). arXiv:2604.02677. --- ## [Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion](https://edtechdev.github.io/aied/articles/multi-site-vr-immersive-learning/) > This paper develops a VR-based immersive learning environment using Photon Fusion that allows teachers and students to be present in the same virtual space regardless of physical locations. The system enables real-time verbal communication and interaction with 3D learning materials, achieving stable real-time communication and state synchronization across multiple players. Evaluation demonstrates stable communication performance, good usability, and minimal VR sickness. This work connects to [[edtech-platform]] design for distributed education and to [[active-learning]] research by providing a technology platform for shared virtual classroom experiences. The system addresses a practical infrastructure gap in multi-site education scenarios where physical co-presence is not possible. ## Key Findings - A Photon Fusion-based architecture supports stable real-time communication and state synchronization among multiple players in a shared virtual classroom. - Teachers and students can occupy the same virtual space regardless of physical location, with two-way voice communication and shared manipulation of 3D models. - The design avoids the frame-rate instability of earlier peer-to-peer approaches (WebSocket/WebRTC with a server on a PC), where processing load depended on the head-mounted display (HMD) and degraded as simultaneous users increased. - Evaluation results show stable communication performance, good usability, and minimal VR sickness, confirming effectiveness as an immersive learning environment. - UI consistency was identified as an area needing improvement, and future work targets a wider variety of learning materials plus UI and network enhancements. ## Study Design & Evaluation The system targets support for up to twenty participants, aiming for high usability through improved operability compared with earlier systems whose complex operation methods, such as hand tracking, resulted in poor usability. The evaluation involved 10 undergraduate and graduate students from the same academic department, aged 21 to 24 (9 men, 1 woman); 3 participants had prior VR experience while 7 had none. In a one-hour experiment using the Meta Quest 3 headset, participants launched the application and entered the VR classroom to familiarize themselves with the system, took a 10-minute break, re-entered the classroom, and conducted a science experiment. Afterwards they completed the System Usability Scale (SUS) and the Simulation Sickness Questionnaire (SSQ), while the frame rate of the teacher's device was recorded to evaluate device load. ## Implications for AI in Education The system addresses a practical infrastructure gap in multi-site education: VR enables learning activities that are difficult to simulate in traditional classrooms, such as gaining a three-dimensional understanding of human structures or conducting chemistry experiments without physical constraints. By decoupling co-presence from physical location, platforms of this kind support distributed [[higher-ed]] and [[stem-education]] scenarios in which hands-on, immersive activities must be shared across sites. The documented trade-offs — usability, UI consistency, and motion sickness — offer a reference point for [[edtech-platform]] designers weighing the costs and benefits of real-time immersive environments against conventional video-based remote instruction. ## Connected Concepts - [[edtech-platform]] - [[active-learning]] ## Connected Articles - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training ## Citation Wataru, I., & Nguyen, D. V. (2026). [*Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion*](https://arxiv.org/abs/2606.10325). arXiv:2606.10325. --- ## [A multi-agent AI classroom based on dual-process reasoning hazards: a pilot with prospective physics teachers](https://edtechdev.github.io/aied/articles/multiagent-classroom-dual-process-physics-teachers-2026/) > **Synthesis:** Tufino (2026) pilots a simulated multi-agent AI classroom where five AI students each enact distinct dual-process theory (DPT) reasoning hazards, giving prospective physics teachers rare practice in responding to authentic student reasoning. Fifteen graduate students showed significant improvement in diagnostic scores (p=0.014, r=0.79), but during the simulation itself used predominantly uniform guiding questions — revealing a knowing-doing gap on the developmental trajectory toward responsive teaching. Responding productively to authentic student reasoning is among the most difficult teaching skills to develop, and prospective teachers get few opportunities to practice it. This pilot study created a simulated class of five AI students, each consistently enacting a distinct dual-process theory reasoning hazard. Fifteen graduate students in a physics teacher preparation course diagnosed vignettes before/after interacting with the simulated class, showing significant diagnostic improvement — but their actual questioning during the simulation revealed a gap between knowing DPT vocabulary and applying it in real-time. - Diagnostic scores improved significantly (Wilcoxon p=0.014, r=0.79) - During simulation, participants used predominantly uniform guiding questions - DPT vocabulary appeared in only 2 of 71 substantive teacher turns during the simulation - Seven of thirteen POST sheets used DPT vocabulary readily — revealing a knowing-doing gap - The simulation makes the developmental trajectory toward responsive teaching visible at transcript-level granularity ## Connected Concepts - [[physics-education]] - [[agentic-ai]] - [[professional-training]] - [[dual-process-theory]] - [[simulation]] - [[stem-education]] - [[higher-ed]] ## Connected Articles - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] ## Citation Tufino, E. (2026). [A multi-agent AI classroom based on dual-process reasoning hazards: a pilot with prospective physics teachers](https://arxiv.org/abs/2608.05820). --- ## [An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training](https://edtechdev.github.io/aied/articles/multimodal-affective-its-presentation/) ## Key Finding Closed-loop ITS with multimodal affective scoring (facial, vocal, textual, oculomotor) produced significant presentation skill gains (Cohen's d = 0.39-0.90, N=204) over 30 days. ## Synthesis This paper presents one of the most comprehensive closed-loop [[intelligent-tutoring]] for soft-skill training. The system operationalizes a seven-dimension BARS across facial, vocal, textual, and oculomotor inputs, using an XGBoost backbone for interpretable scoring that approaches expert-rater reliability (Spearman's rho 0.69-0.78). The three-layer feedback architecture — rubric-aligned scoring, audience-expressive diagnostics, and [[rag]] conversational coaching — creates a complete deliberate practice loop. With 204 adult learners and Cohen's d of 0.39-0.90 across all seven dimensions, this is among the stronger efficacy signals in ITS research. The interpretability requirement (feedback traceable to observable cues) directly addresses concerns raised in [[educational-llm-alignment]] about opaque AI feedback, while the multimodal approach extends beyond text-only systems like [[cyberscholar-genai-writing-feedback]]. The closed-loop architecture shares philosophical ground with [[ai-tutor-behavioral-evaluation]]'s call for behavioral feedback loops, and the retrieval-augmented coaching component parallels [[retrieval-augmented-tutoring-algorithm-kite]]'s approach to grounded tutoring. ## Connected Concepts - [[intelligent-tutoring]] - [[rag]] ## Connected Articles - [[educational-llm-alignment]] - [[cyberscholar-genai-writing-feedback]] - [[ai-tutor-behavioral-evaluation]] - [[retrieval-augmented-tutoring-algorithm-kite]] ## Citation K.-E, A.S.H.H. & Technologies, V.I.T.O.L. (2026). [*An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training*](https://arxiv.org/abs/2605.17468) --- ## [LLM-based Multimodal AI Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback](https://edtechdev.github.io/aied/articles/multimodal-ai-feedback-learning/) # Multimodal AI Feedback vs. Educator Feedback ## Core Finding **AI multimodal feedback matches educator feedback for learning while significantly outperforming it on student perceptions.** The authors built a real-time AI-facilitated multimodal feedback system integrating structured text, relevant slide references, and streaming AI audio narration. In a crowdsourced experiment, they compared it against fixed "business-as-usual" educator feedback. ## The System The multimodal feedback system combines three channels: 1. **Structured textual explanations** — targeted, specific feedback on student answers 2. **Dynamic slide references** — retrieval of the most relevant lecture slide for context 3. **Streaming AI audio narration** — spoken explanation to complement text This multimodal approach aims to provide timely, targeted feedback at scale — something that remains a major challenge in education. ## Results ### Learning Effectiveness - **Equivalent learning gains** between AI multimodal feedback and educator feedback - No significant difference in post-test performance ### Student Perceptions (AI significantly better) | Dimension | AI > Educator? | |-----------|:---:| | Clarity | ✓ | | Specificity | ✓ | | Conciseness | ✓ | | Motivation | ✓ | | Satisfaction | ✓ | | Reduced cognitive load | ✓ | | Correctness | — | | Trust | — | | Acceptance | — | AI feedback matched educator feedback on correctness, trust, and acceptance — but outperformed on every experiential dimension. ### Behavioral Engagement Patterns - **Multiple-choice questions**: Educator feedback encouraged more total submissions (students kept trying) - **Open-ended questions**: AI feedback lowered revision barriers — targeted suggestions promoted iterative improvement ## Significance This is a strong result for AI feedback systems: - **Equivalence on learning** is the bar most systems fail to clear — AI matched human educators - **Superior student experience** across 6 dimensions suggests AI can surpass humans on consistency, specificity, and clarity - **Scalability**: The system can provide real-time, context-aware support without instructor availability constraints - **Question-type effects**: The engagement pattern differences suggest adaptive strategies — AI may be better for open-ended work while human-like interaction helps for multiple-choice ## Methodological Notes - Online crowdsourcing experiment — participants recruited via a platform, not classroom students - Compared against **fixed** educator feedback (not live, not adaptive) — the AI system's adaptivity may partly explain its perceptual advantage - Single-session design — long-term effects unknown ## Open Questions - Would results hold in real classroom settings with live educator feedback rather than fixed, pre-written feedback? - Does the advantage persist over multiple sessions, or is there a novelty effect? - How does each modality (text vs. slides vs. audio) contribute to the overall effect? - Can multimodal AI feedback reduce the negative behavioral pathway identified in sequenced feedback studies (fewer resubmissions)? ## Connected Concepts - [[ai-feedback-quality]] - [[administrator]] - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[affective-computing]] - [[metacognition]] - [[self-regulated-learning]] - [[socratic-method]] ## Connected Articles - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[cyberscholar-genai-writing-feedback]] — Generative AI Feedback, English Writing and Teacher Rubrics: A Multiple-Case Study of CyberScholar - [[sequenced-ai-feedback-learning]] — Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning - [[aicode-collaborative-feedback-system]] — AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education ## Citation Zhao, C. Q., Cao, J., Lin, J., & Koedinger, K. R. (2026). [LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback](https://arxiv.org/abs/2601.15280). arXiv:2601.15280. Accepted to LAK 2026. --- ## [Multimodal AI Tutoring in STEM](https://edtechdev.github.io/aied/articles/multimodal-ai-tutoring/) > When LLMs process STEM problems that require interpreting diagrams, graphs, or schematics alongside text, their accuracy degrades substantially. This effect is: > General-purpose LLMs achieve near-ceiling accuracy (~96%) on text-only physics problems, but accuracy drops to **~74%** on image-rich multimodal problems due to the **Multimodal Interference Effect**. A simple structured dialogue intervention—requiring no retraining—recovers performance to **95%**.^[[syal-multimodal-dialogue-stem-2026]] ## The Multimodal Interference Effect When LLMs process STEM problems that require interpreting diagrams, graphs, or schematics alongside text, their accuracy degrades substantially. This effect is: - **Consistent across models:** Claude, Gemini, and ChatGPT all showed the drop - **Dominantly visual:** Visual Processing Errors were the most prevalent failure mode - **An equity concern:** Students working on image-rich problems—often those most needing support—receive less reliable AI assistance ## Error Taxonomy Syal et al. (2026) identified four empirically-derived failure modes on OpenStax physics problems: | Category | Prevalence | Definition | |----------|-----------|------------| | **Visual Processing Errors** | Highest | Failures to extract information from graphs, diagrams, or images | | **Context Misinterpretation** | Moderate | Wrong physics concepts or problem framing assumptions | | **Mathematical Computation Errors** | Low | Calculation or unit errors | | **Hybrid Errors** | Moderate | Combinations of the above | *Note: Visual and Hybrid errors appeared only in multimodal (not text-only) problems.*^[[syal-multimodal-dialogue-stem-2026]] ## The Structured Dialogue Intervention A three-step scaffolding protocol corrected **82% of all errors** and **100% of Visual Processing Errors**: 1. **Elicit visual description:** *"Describe what you observe in the figure/graph"* 2. **Correct observable elements:** Clarify visual properties (e.g., vector direction, graph intercept) **without providing physics principles or formulas** 3. **Re-prompt for solution:** *"Solve the problem"* This relies purely on prompt design—no model retraining, no specialized benchmarks, no GPU access required. Under-resourced institutions can implement it immediately. ## Relationship to Tutoring-Specific Design The structured dialogue intervention is structurally similar to [[tutoring-specific-vs-general-ai|tutoring-specific guardrails]]: it forces the LLM to reason step-by-step rather than jumping to answers. It also preserves productive cognitive load by making the student (or in this case, the LLM) attend to visual structure before solving.^[[syal-multimodal-dialogue-stem-2026]] ## Implications - **For STEM educators:** Image-rich problems remain a high-risk use case for raw LLM tutoring; structured prompting protocols dramatically improve reliability - **For tool designers:** Visual reasoning pipelines should not be treated as a solved capability; interface-level scaffolding may matter as much as model scale - **For equity:** The Multimodal Interference Effect disproportionately harms students relying on free/public AI tools for STEM support ## Open Questions 1. Does the intervention generalize to other STEM disciplines (chemistry, biology, engineering)? 2. What happens when the *student* directly implements the dialogue protocol vs. an intermediary system? 3. Do visual reasoning capabilities improve in newer model generations (e.g., GPT-5 multimodal)? ## Connected Concepts - [[pedagogical-agent]] - [[agentic-ai]] - [[ai-tutoring]] - [[math-education]] - [[affective-tutoring]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[help-seeking]] ## Connected Articles - [[syal-multimodal-dialogue-stem-2026]] - [[tutoring-specific-vs-general-ai]] ## Citation tasks:, T.W.A.V.L.W.T.F.M. & Preview, P. (2026). [*Multimodal AI Tutoring in STEM*](https://arxiv.org/abs/2605.04131) --- ## [Multimodal Item Parameter Estimation using Simulated Response Probabilities](https://edtechdev.github.io/aied/articles/multimodal-item-parameter-estimation-2026/) > **Synthesis:** This paper fine-tunes a multimodal large language model (Qwen3.5-based) to reconstruct multiple-choice model (MCM) and three-parameter logistic (3PL) item characteristic curves. By learning to reproduce students' systematic error patterns across a range of ability levels, the LLM implicitly captures underlying response probabilities and can approximate item difficulty on held-out test items directly from predicted option probabilities. ## Key Findings 1. **LLMs can learn IRT curves implicitly.** A multimodal LLM prompted and fine-tuned to replicate choice probabilities across a corpus of image-and-text multiple-choice items, conditioned on labeled ability levels, learns the response patterns encoded in 3PL and MCM curves. 2. **Direct difficulty estimation.** Item difficulty on a held-out test set can be approximated directly from the model's predicted option probabilities, without explicit parameter-fitting. 3. **Multimodal relevance.** Because items contain both image and text stimuli, the approach extends IRT-style estimation to multimodal assessment items. ## Implications For [[item-response-theory]] and [[educational-measurement]], this suggests a path to calibrate assessment items using LLMs that emulate student responding, which is relevant to [[psychometrically-aware-ai]] and to pre-testing items before deployment. It connects to [[student-modeling]] and [[automated-assessment]] work where models must reason about how learners of different abilities respond. The approach has practical implications for [[automated-question-generation]] pipelines and for scaling item calibration in [[llm]]-based assessment systems, though the work is presented as a technical proof-of-concept and would benefit from broader validation across item types and populations. ## Connected Concepts - [[item-response-theory]] - [[educational-measurement]] - [[psychometrically-aware-ai]] - [[student-modeling]] - [[automated-assessment]] - [[llm]] - [[multimodal]] - [[knowledge-tracing]] ## Connected Articles - [[ai-assessment-scale-reform]] — AI assessment scale reform - [[ai-assessment-human-tutors]] — AI assessment compared with human tutors - [[socratic-tests-conversational-assessment]] — Conversational assessment - [[tutoring-effectiveness-index]] — Tutoring effectiveness index ## Citation Ormerod, C., & Kim, Y. (2026). [*Multimodal item parameter estimation using simulated response probabilities*](https://arxiv.org/abs/2608.10154). arXiv:2608.10154. cs.CL. --- ## [Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning](https://edtechdev.github.io/aied/articles/multimodal-knowledge-graph-educational-reasoning/) > **Synthesis:** This paper introduces an evidence-grounded multimodal pipeline that constructs provenance-rich [[knowledge-tracing|knowledge graphs]] from lecture videos by integrating speech transcripts, slide OCR, and vision-language model analysis. Processing three neural-network lectures, the pipeline extracted 172 canonical concepts and 282 typed relationships with 90.38% endpoint coverage, achieving perfect retrieval accuracy on a preliminary test. The approach addresses a key challenge in [[ai-education|educational AI]]: converting rich multimodal lecture content into structured, queryable knowledge representations without losing the evidential provenance that makes them trustworthy. ## Pipeline Architecture The multimodal pipeline processes lecture videos through several stages: 1. **Transcription:** Speech-to-text conversion of lecture audio 2. **Semantic Anchor Selection:** Identification of key concept-bearing segments 3. **OCR Extraction:** Text extraction from slide content and diagrams 4. **Vision-Language Analysis:** Concept and relationship extraction with evidential grounding 5. **Validation and Canonicalization:** Cross-referencing mentions against multiple evidence sources 6. **Knowledge Graph Construction:** Typed relationships with provenance tracking ## Key Metrics | Stage | Volume | |-------|--------| | Frames processed | 3,118 | | Transcript segments | 756 | | Semantic anchors | 559 | | Concept mentions retained | 1,022 | | Relationship mentions | 312 (corrected from paper) | | Canonical concepts | 172 | | Typed relationships | 282 | | Endpoint coverage | 90.38% | | Retrieval top-1 accuracy | 100% | | Retrieval top-5 recall | 100% | ## Significance for Educational AI The auditable construction method is the key contribution — unlike black-box knowledge extraction, every concept and relationship in the graph is traceable to specific evidence (transcript segment, OCR text, or visual element). This provenance tracking is critical for educational applications where: - **Accuracy matters:** Incorrect concept relationships can mislead [[student-modeling|student models]] - **[[adaptive-learning]] systems** need trustworthy prerequisite relationships - **Lecture review tools** benefit from structured, queryable knowledge representations - **Multi-lecture reasoning** requires connecting concepts across different instructional sessions ## Connected Concepts - [[knowledge-tracing]] - [[ai-education]] - [[student-modeling]] - [[adaptive-learning]] ## Connected Articles - [[proprl-prerequisite-relation-learning]] — ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[leveraging-complex-systems-leading-for-transformative-change]] — Leveraging complex systems: Leading for transformative change ## Citation Al Farib, S., Meem, M. A., Islam, S. R., & Raihan, M. T. (2026). [*Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning*](https://arxiv.org/abs/2608.03161v1). arXiv:2608.03161v1. --- ## [Multimodal Learning with Generative AI](https://edtechdev.github.io/aied/articles/multimodal-learning-genai/) > The guide adopts a middle way between "techno-fixing" and rejecting AI as an existential threat. It argues that: > A comprehensive educator's guide to integrating Generative AI into multimodal teaching, learning, and assessment across higher education. Built on Kress's social semiotic theory of multimodality, the guide positions GenAI as a 'cyber-social' partner that complements—but cannot replace—human meaning-making. It proposes the **MMLD-AI** unifying model (UDL + ABC Learning Design) and the **Dual-Track Cyber-Social Learning Model** for designing effective human-AI collaboration.^varga-atkins-educators-guide-multimodal-learning-genai-2025 ## Core Position: Pragmatic, Not Uncritical The guide adopts a middle way between "techno-fixing" and rejecting AI as an existential threat. It argues that: - GenAI is already embedded in daily life; ignoring it does students a disservice - GenAI is **not intelligent** (no consciousness, understanding, or ethical judgement) - Human educators and learners bring vision, purpose, nuanced critique, and meaning-making that AI cannot replicate - Effective use requires **cyber-social partnership**: humans and machines with complementary strengths ## The Four Costs of GenAI | Cost Domain | Key Concern | Educational Response | |-------------|-------------|---------------------| | **Individual** | Privacy, data protection, equity of access, mental health, over-reliance | Transparent authorship; approved tools list; scaffolded critical engagement | | **Environment** | Image/video/audio generation uses significantly more energy than text | Mindful use; limit iterations; group demonstrations; digital decluttering | | **Knowledge** | Removes sourcing process integral to retention; short-term gains may displace deep learning | Students clarify own understanding *before* consulting AI; metacognitive scaffolding | | **Future Jobs** | Entry-level white-collar roles vulnerable to automation | Focus on human strengths: contextual reasoning, ethical judgement, craftsmanship | ## AI Literacy in Multimodal Contexts ### Three Levels 1. **Basic literacy** — Awareness of multimodal GenAI platforms, capabilities, and appropriate uses (creating prompts, generating visual outputs) 2. **Intermediate literacy** — Co-create multimodal content, critically evaluate AI outputs, scaffold uses (transform lecture notes into visuals or podcasts) 3. **Advanced literacy** — Design activities/assessments incorporating multimodal GenAI; lead ethical and philosophical discussions ### Four Implementation Scales | Scale | Strategies | |-------|-----------| | **Individual** | Workshops on creative multimodal tasks; prompt crafting practice; reflective assignments documenting AI use | | **Module** | Embed GenAI literacy into learning outcomes; optional multimodal tasks with clear rubrics; creative/reflective critique components | | **Programme** | Cross-module policies; consistency and transparency via workshops and discussion; alignment with graduate attributes (criticality, creativity, digital fluency) | | **Institutional** | Clear policies with checklists; vetted tools; data privacy protocols enforced; avoid rigid mandates in favor of flexible guidance | ## The MMLD-AI Unifying Model The **Multimodal Learning Design with GenAI** model merges: - **Universal Design for Learning (UDL)**: multiple means of engagement, representation, and action/expression - **ABC Learning Design**: storyboarding the student journey through learning types ### Six Multimodal Engagement Types (adapted from ABC) 1. **Acquisition** of information 2. **Investigation** and/or research 3. **Collaboration** with others 4. **Production** of artefacts (learning, teaching, or assessment) 5. **Practice** of approaches/theories/principles/skills 6. **Discussion/discourse**, including critique/evaluation For each engagement type, educators decide on multimodal affordances of GenAI and explore respective **cyber-social strengths**. ## The Dual-Track Cyber-Social Learning Model (Galla et al., 2025) This complementary model maps human vs. AI strengths across Bloom's taxonomy processes: | Process | Human Strengths | AI Strengths | Cyber-Social Approach | |---------|----------------|--------------|---------------------| | Knowledge & framing | Contextual understanding; embodied knowledge; critical verification | Rapid data retrieval; pattern recognition; broad topical coverage | Humans define purpose and frame problems; AI generates background data and inspiration; humans filter and verify | | Interpretation & analysis | Causal reasoning; cultural/ethical awareness; implicit meaning | Correlation analysis; theme identification; feature extraction at scale | AI identifies statistical patterns; humans determine causality, relevance, deeper significance | | Application & prototyping | Situated judgement; adaptive problem-solving; ethical decision-making; craftsmanship | Rapid simulation; consistent rule application; code/digital artefact generation | AI generates digital prototypes; humans adapt for real-world complexity, apply physical craft, ensure ethics | | Synthesis & creation | Novel conceptual blending; purpose-driven integration | Cross-domain pattern integration; combinatorial exploration | AI explores possible combinations; humans evaluate, refine, and integrate meaningfully | ## Practical Integration: Three Strands ### Teaching (Educator-Created Content) - Generating visuals, diagrams, and infographics from text prompts - Creating podcast scripts and video summaries - Building interactive simulations and virtual scenarios - Using AI to get feedback on marking rubrics and assessment briefs ### Learning (Student-Created Content) - Students transform lecture notes into multimodal artefacts (visuals, podcasts, videos) - Collaborative group projects using GenAI for brainstorming and prototyping - Critical evaluation: students annotate AI-generated outputs for accuracy, bias, coherence - Ethical protocols establishing clear boundaries (e.g., "do not use AI to write reflections; do use it for brainstorming visuals") ### Assessment and Feedback - Multimodal assessment: students submit artefacts combining text, image, audio, video - AI-assisted peer and self-assessment with structured rubrics - Educators use GenAI to generate formative feedback at scale, then verify and personalize - Transparent: assessment briefs explicitly state when and how GenAI may be used ## Relationship to Existing Research | Guide Principle | Wiki Connection | |-----------------|----------------| | Cyber-social partnership (complementary strengths) | [[principled-ai-education]] — "AI must augment, not displace" aligns perfectly | | Four costs framework (individual, environment, knowledge, jobs) | [[ai-tutor-safety-harms]] — Costs to knowledge overlap with cognitive offloading; environmental costs are a new dimension | | AI literacy levels and scales | [[ai-literacy]] — ICAP framework; collaborative learning; this guide adds institutional scaling and multimodal specificity | | MMLD-AI model (UDL + ABC + six engagement types) | [[adaptive-learning]] — Multi-resolution personalization; [[agentic-workflows-education]] — Planning and reflection paradigms | | Dual-Track Cyber-Social Model | [[pedagogical-llm-training]] — Reward "guiding" over "answering"; [[human-in-the-loop-ai]] — Human verification of AI outputs | | Multimodal assessment redesign | [[authentic-assessment]] — Six-dimensional framework; [[formative-assessment]] — AI-generated feedback with human validation | | Scaffolding and metacognition | [[self-regulated-learning]] — UDL's emphasis on student agency; [[metacognition]] — Cyber-social metacognitive awareness | | Faculty development across four scales | [[faculty-development]] — CTL pragmatic transition model; this guide adds module-level and programme-level strategies | ## Case Study Themes from the Guide The guide includes 15+ educator case studies spanning: - **Healthcare**: AI avatars for patient communication training (H5P interactive scenarios) - **Bioscience**: Multimodal groupwork designing organisms for future Earth scenarios - **Business/HR**: Peer conflict resolution with AI-generated scenarios and video avatars - **Education/Teacher training**: AI visual metaphors for reflective practice - **Chemistry**: AI-generated molecular visualizations and 3D models - **Languages**: Text-to-speech and avatar creation for pronunciation practice - **General**: Explainer videos, digital posters, podcast scripts, interactive quizzes ## Open Questions 1. **Environmental cost awareness**: How can educators and students make informed trade-offs between the pedagogical value of multimodal GenAI artefacts and their energy costs? 2. **Transfer across modalities**: Does competence in AI-assisted multimodal creation in one domain (e.g., visual design) transfer to another (e.g., audio production)? 3. **Assessment validity**: When students use GenAI to create multimodal assessment artefacts, how can assessors distinguish genuine human meaning-making from AI-generated polish? 4. **Scaling the MMLD-AI model**: Can the six engagement types be operationalized as automatic learning design recommendations, or does human pedagogical judgment remain essential? ## Connected Concepts - [[adaptive-learning]] - [[ai-literacy]] - [[dot-framework-survey]] - [[faculty-development]] - [[formative-assessment]] - [[human-in-the-loop-ai]] - [[metacognition]] - [[pedagogical-llm-training]] - [[self-regulated-learning]] - [[socratic-ai-dialogue]] - varga-atkins-educators-guide-multimodal-learning-genai-2025 - [[ai-education]] - [[generative-ai]] - [[higher-ed]] - [[scaffolding]] ## Connected Articles - [[agentic-workflows-education]] — Agentic Workflows in Education - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[authentic-assessment]] — Authentic Assessment - [[collaborative-ai-tutoring]] — Collaborative AI Tutoring - [[educational-llm-alignment]] — Educational LLM Alignment - [[multimodal-ai-feedback-learning]] — LLM-based Multimodal AI Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback - [[multimodal-ai-tutoring]] — Multimodal AI Tutoring in STEM - [[principled-ai-education]] — Principled AI in Education - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform ## Citation AI, G., Original, T., Multimodal, A.E.G.T., AI, L.A.G., Investigators:, P., Varga-Atkins, T., Saunders, S., & Hallam, S.B.S. (2026). [*Multimodal Learning with Generative AI*](https://livrepository.liverpool.ac.uk/3194252/1/Output1_an-educators-guide-to-multimodal-learning-and-generative-ai.pdf) --- ## [Students' multimodal prompting practices as epistemic work in AI literacy development](https://edtechdev.github.io/aied/articles/multimodal-prompting-ai-literacy/) > **Synthesis:** Students' multimodal prompting practices as epistemic work in AI literacy development ## Key Findings - Prompting strategies vary along a continuum from basic input-output use to strategic, iterative, and dialogic practices. Across the eight student groups, the authors mapped four prompting methods — Input-Output, Role-Play, Chain-of-Thought, and Generated Knowledge — with one group (G5) interrogating the system about the components of a good story before generating and another (G3) instructing it step-by-step through the assignment. - Prompting emerges as a central epistemic practice through which students critically interpret, refine, and negotiate AI-generated outputs — for example, one group's 16-step prompting history moved from exploring genre conventions to challenging the system's character design choices and requesting explanations of its revisions. - Multimodal engagement exposes challenges in translating abstract meaning into machine-readable prompts, fostering awareness of system limitations and bias: students concluded that "prompt literacy is different between prompting for text than it is for pictures," and that image outputs became increasingly divergent from their intended vision without a precise visual vocabulary ("We realized we don't have as much experience prompting for images"). - Students develop the need to actively construct coherence across modalities when producing text and image outputs, and learn tool-specific constraints — Copilot's image generation was judged less effective than ChatGPT or DALL·E, and groups switched tools after copyright-related refusals, evaluating systems on performance and output quality. - While students demonstrate developing competence in evaluation and creation, the ethical dimensions of [[ai-literacy]] remain underdeveloped: reflections focused on functionality and alignment rather than fairness, transparency, privacy, or responsible use. ## Study Design & Method The study was conducted as an exploratory workshop with 28 postgraduate students engaged in collaborative multimodal prompting tasks, including the creation of short stories or poems and corresponding images using a university-provided GenAI tool. The participants (aged 23–44) came from diverse cultural, linguistic, and academic backgrounds spanning Europe, Asia, Africa, and the Americas, with prior GenAI experience ranging from information retrieval and summarization to image generation and coding tasks. The workshop lasted 1h 45min across two sessions of four groups each; all four groups chose short stories in the first session and the majority chose poems in the second, yielding a balanced exploration of narrative and poetic formats. The corpus comprised eight self-documented group reports (6–22 pages each, 109 pages in total) containing prompting histories, motivations, and reflections, analyzed qualitatively using reflexive thematic analysis, guided by frameworks for prompting methods and AI literacy. The design aimed to provide empirical insight into two questions: which prompting strategies students develop when interacting with open-ended GenAI tools, and how engagement in prompt engineering activities shapes their understanding of GenAI and [[ai-literacy]] more broadly. ## Implications for AI in Education The findings position prompting as an epistemic practice — a way of learning to interpret, negotiate, and guide AI-generated outputs — rather than a merely technical skill. Because students' strategies ranged from simple input-output use to iterative, dialogic refinement, the study suggests that [[prompt-engineering]] competence develops through hands-on engagement rather than through rule-based instruction alone, with effective prompting depending on balancing specificity and openness rather than simply adding detail. Multimodal tasks proved valuable for surfacing the interpretive work behind prompt construction: text prompting yielded satisfactory results with fewer attempts than image prompting, and students discovered that coherence across modalities must be actively constructed, requiring a more precise visual vocabulary. Yet the comparatively weak development of ethical reasoning points to a gap that standard evaluation-and-creation activities do not close on their own — students evaluated tools primarily on performance and output quality, with limited engagement with fairness, transparency, or responsible use. For [[higher-ed]] practitioners, the results highlight the value of iterative, reflective, and multimodal learning designs that foster critical, strategic, and responsible engagement with AI, and they suggest that ethical dimensions of literacy — including platform governance, copyright, bias, and responsible AI use — need to be taught explicitly across the curriculum rather than assumed to follow from technical proficiency. The findings also connect prompting to [[multimodal]] literacy more broadly, positioning it as a situated practice requiring distinct forms of interpretation and meaning-making across text and image generation. ## Limitations The author notes that the small sample size (28 postgraduate students) limits the generalizability of the findings: although participants represented diverse disciplines, professions, and geographic regions, they were self-selected and enrolled in a course module that may have attracted students with a pre-existing interest in technology or digital innovation. The study was confined to a single workshop within a specific course context, which restricts the scope of the intervention and the depth of longitudinal insight into students' evolving AI literacy. The findings should therefore be interpreted as exploratory and context-specific, pointing to the need for broader, more sustained studies across varied educational settings and learner populations. ## Connected Concepts - [[prompt-engineering]] - [[ai-literacy]] - [[higher-ed]] - [[multimodal]] - [[automated-essay-scoring]] - [[reducing-ai-misuse]] - [[affective-tutoring]] - [[cs-education]] ## Connected Articles - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation - [[same-ai-different-pathways]] — Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts - [[pchl-he-framework-genai-content-creation-2026]] — From Prompts to Verified Loops: The PCHL-HE Framework for Generative AI-Assisted Educational and Research Content Creation in Higher Education - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[choi-anchor-aes-prompting-2025]] — Anchor Is the Key: Toward Accessible Automated Essay Scoring with Large Language Models Through Prompting - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration: A Longitudinal Analysis of Prompting Strategies ## Citation Sofkova Hashemi, S. (2026). [*Students' multimodal prompting practices as epistemic work in AI literacy development*](https://doi.org/10.1016/j.caeai.2026.100635). --- ## [Challenges for Musical Education in the Age of AI and Digital Transformation](https://edtechdev.github.io/aied/articles/musical-education-ai-digital-transformation-2026/) > **Synthesis:** This paper maps the challenges that generative AI, streaming algorithms, and digital audio workstations pose for music education. Three converging transformations are examined: the changing nature of music creation and consumption, shifts in the public for music shaped by algorithmic curation, and the democratization of music production through digital tools. The paper surveys implications for both curricular content and pedagogical methods, arguing that music education must adapt to a landscape where AI can produce complete, stylistically coherent pieces from text prompts. ## Key Findings This paper maps the challenges that generative AI, streaming algorithms, and digital audio workstations pose for music education. Three converging transformations are examined: the changing nature of music creation and consumption, shifts in the public for music shaped by algorithmic curation, and the democratization of music production through digital tools. The paper surveys implications for both curricular content and pedagogical methods, arguing that music education must adapt to a landscape where AI can produce complete, stylistically coherent pieces from text prompts. The work contributes to understanding of [[ai-literacy]] in educational contexts, with implications for [[curriculum-design]], [[generative-ai]]. ## Connected Concepts - [[ai-literacy]] - [[curriculum-design]] - [[generative-ai]] - [[ai-education]] ## Connected Articles - [[ai-enabled-serious-games]] - [[embodied-string-learning-blindness-low-vision-musicians]] ## Citation Briot, J.-P. (2026). [*Challenges for Musical Education in the Age of AI and Digital Transformation*](https://arxiv.org/abs/2608.05176). arXiv:2608.05176. --- ## [Neural-Symbolic Knowledge Tracing](https://edtechdev.github.io/aied/articles/neural-symbolic-knowledge-tracing/) > Key limitations exist in both LLM-based tutoring and conventional Deep Knowledge Tracing (DKT): > Combining neural networks with symbolic educational knowledge for interpretable, data-efficient, and temporally reliable learner modeling in intelligent tutoring systems. ## The Problem with Pure Neural Approaches **Hooshyar et al. (2026)** identify key limitations in both LLM-based tutoring and conventional Deep Knowledge Tracing (DKT): - **Limited adaptivity:** LLMs struggle to model learners' *evolving knowledge over time* — they respond to individual turns rather than tracking skill mastery across sessions. - **Black-box opacity:** Deep learning models (LSTM/Transformer-based DKT) lack interpretable learner state representations, making it impossible to audit *why* a student is flagged as struggling. - **Spurious correlations:** Purely data-driven models can latch onto surface patterns (session length, interaction frequency) rather than genuine learning signals. - **Temporal instability:** Conventional DKT produces **prediction inconsistency rates of 0.075–0.138**, meaning predictions frequently "flip-flop" between correct/incorrect as new responses arrive — undermining trust. ## The Responsible-DKT Model Hooshyar et al. propose **Responsible-DKT**, a neural-symbolic architecture built on the **Lifted Relational Neural Networks (LRNN)** paradigm using **PyNeuraLogic**. The key innovation: symbolic educational knowledge is injected as *learnable weighted rules* that **modulate** (not override) neural predictions. ### Architecture | Component | Detail | |-----------|--------| | **Embeddings** | Learnable vectors for quiz, skill, correctness (dim=16) | | **Combined rep** | Weighted sum of embeddings + sigmoid activation | | **Recurrent core** | Two-layer RNN with temporal shift (prediction at *t* uses only past interactions) | | **Symbolic rules** | Three rules with learnable weights, wired into the computation graph | | **Output** | Sigmoid over hidden state + target embedding → ŷ_{t+1} | ### Injected Educational Knowledge (Symbolic Rules) 1. **mastered rule:** Two consecutive correct responses on the same skill/quiz → increases prediction confidence. Encodes the pedagogical assumption that sustained correctness signals mastery. 2. **not_mastered rule:** Three incorrect responses on the same skill/quiz → decreases prediction confidence. Encodes the assumption that repeated failure is a stronger signal — the paper found this rule **dominates** prediction updates. 3. **avg_embed rule:** Historical aggregation of past embeddings for the same skill/quiz — provides a long-run average representation of the student's relationship to a knowledge component. ### Baselines - **BaseNS-DKT:** Identical neural-symbolic architecture but *without* the educational knowledge rules — isolates the value of symbolic injection. - **PyTorch DKT:** Fully data-driven DKT — represents the conventional black-box approach. ## Key Findings ### Data Efficiency (RQ1) - **>0.80 AUC with only 10%** of training data — practically useful for cold-start and sparse-data settings. - Up to **0.90 AUC** with full training data. - **Up to 13% improvement** over both baselines. ### Temporal Reliability (RQ2) - **Lowest prediction inconsistency rates** across all sequence lengths (10, 50, 100, 475): - Responsible-DKT: **0.013–0.039** - PyTorch DKT: **0.075–0.138** - Lower early- and mid-sequence prediction errors. - Prediction updates remain **directionally aligned** with observed responses — the model doesn't contradict itself as new data arrives. ### Interpretability (RQ3) - **Grounded computation graph** exposes the logic behind each prediction — inherently interpretable, not post-hoc explained. - Supports both **local** (per-prediction) and **global** explanations. - Empirical evaluation of pedagogical assumptions revealed that **non-mastery rules** (repeated incorrect responses) have a stronger influence on prediction updates than mastery rules — an insight that would be invisible in black-box models. ### Dataset Real-world 6th-grade Maths from Opiq: 167 students, 21,471 interactions, 13 skills, 1,058 quizzes. Scores binarized at first-quartile threshold (37/100). ## Connection to Knowledge Tracing Landscape Traditional **Knowledge Tracing (KT)** methods: - **IRT-based** (Item Response Theory): [[knowledge-tracing-irt]] — interpretable difficulty parameters but limited sequential modeling. - **BKT** (Bayesian Knowledge Tracing): Explicit prerequisite structures; interpretable but rigid. - **DeepKT** (LSTM/Transformer): High predictive capacity but opaque. - **Neural-symbolic KT** bridges the gap: retains interpretability of symbolic approaches while leveraging neural networks' representational power. ## Implications for Responsible AI in Education The paper positions this under **responsible AI** (Goellner et al., 2024: "human-centred approach fostering user trust through ethical decision-making, explainable outcomes, and privacy-preserving implementation"): 1. **Transparency:** Symbolic components make learner state *auditable* — critical for [[ai-tutor-safety-harms|safety auditing]]. 2. **Pedagogical validity:** Updates follow educational theory (mastery learning, repeated-failure signals), not just data patterns. 3. **Data efficiency:** Practical in real educational settings where large labeled datasets are rare. 4. **Temporal trustworthiness:** Predictions don't oscillate wildly — a key requirement for [[student-experience|student trust]] and instructor decision-making. This contrasts with opaque LLM-based approaches in [[pedagogical-llm-training|tutoring alignment]] and demonstrates a path toward [[principled-ai-education|principled AI in education]] — augment, don't displace, pedagogical reasoning. ## Open Questions - How does Responsible-DKT scale to larger skill graphs (hundreds of knowledge components)? - Can the symbolic rule set be extended with domain-specific pedagogical knowledge (e.g., common misconceptions)? - How does this compare to [[llm-student-modeling-memory|LLM-based longitudinal student modeling]] in both accuracy and interpretability? - Is the 13% improvement maintained with more diverse student populations and subject domains? ## Connected Concepts - [[student-experience]] - [[pedagogical-llm-training]] ## Connected Articles - [[knowledge-tracing-irt]] - [[ai-tutor-safety-harms]] - [[principled-ai-education]] - [[llm-student-modeling-memory]] ## Citation Hooshyar, D., Šír, G., Yang, Y., Kärkkäinen, T., Hämäläinen, R., Krivich, E., Cukurova, M., Gašević, D., & Azevedo, R. (2026). [*Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling*](https://arxiv.org/abs/2604.08263). arXiv:2604.08263. --- ## [I can''t read your mind": A Study of Neurodivergent Computing Students'' Experiences with Collaborative Active Learning](https://edtechdev.github.io/aied/articles/neurodivergent-computing-students/) This study surveyed 24 neurodivergent computing students (autistic and/or ADHD) and 20 neurotypical peers, supplemented by 4 in-depth interviews, to understand how collaborative active learning structures affect comfort and accessibility. Three key findings emerge: **(1)** Neurodivergent students experience significant discomfort with assignments that lack clear structure or have ambiguous expectations — the title quote reflects a common frustration with unspoken social norms in teamwork. **(2)** They strongly prefer **smaller teams that work together consistently**, with **explicitly defined roles**, minimizing the cognitive load of social negotiation. **(3)** Common coping strategies include self-selecting roles and strategic self-disclosure of neurodivergence. The findings connect to broader concerns in [[equity-in-ai-education]] and [[equity]]: as AI tutors and collaborative AI tools enter computing classrooms, their interaction models may inadvertently replicate the same structural ambiguities that disadvantage neurodivergent learners. The preference for defined roles and predictable structures mirrors themes in [[special-education]] about explicit scaffolding. For [[cs-education]], the study provides actionable design recommendations: instructors should provide structured assignments, use smaller consistent teams, and allow role self-selection. While preliminary (n=24), this is among the first studies to center neurodivergent voices in computing education research, contributing to more [[student-experience|student-centered]] approaches to [[collaborative-learning]]. ## Connected Concepts - [[equity-in-ai-education]] - [[equity]] - [[special-education]] - [[cs-education]] - [[student-experience]] - [[collaborative-learning]] ## Connected Articles - [[suacode-african-students-motivations]] — Why SuaCode?": Understanding African Students'' Motivations for Taking a Smartphone-Based Online Coding Course - [[genai-minoritized-knowledges-disability]] — Generative AI and the marginalization of minoritized knowledges in higher education: the case of disability - [[embodied-string-learning-blindness-low-vision-musicians]] — Designing for What Cannot Be Seen: Supporting Embodied String Learning for Musicians with Blindness and Low-Vision - [[dyslexlens-dyslexic-learners-ai]] — DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[ai-literacy-career-adaptability-business-2026]] — AI literacy alone is not enough: Student AI readiness and career adaptability in business and management education ## Citation Cynthia Zastudil, Srishty Muthusekaran, Rayhona Nasimova, Stephen MacNeil (2026). ["I can't read your mind": A Study of Neurodivergent Computing Students' Experiences with Collaborative Active Learning](https://arxiv.org/abs/2605.23823). arXiv:2605.23823. arXiv preprint (cs.HC). --- ## [New systems of learning for distance learning institutions? A six-study review of implementing AIDA](https://edtechdev.github.io/aied/articles/new-systems-of-learning-for-distance-learning-institutions-a-six-study-review-of/) > **Synthesis:** Rienties et al. (2026) examine how the Open University (UK) — a large-scale distance learning institution teaching 200K+ learners across 50+ countries — designed, implemented, and evaluated an AI digital assistant (AIDA) using Sharples' embedded systems approach. Through six iterative Design-Based Research (DBR) studies over 18 months involving 498 students and 20 staff, they found that purpose-built GenAI tools embedded within the learning environment can enhance engagement in distance education, provided development is participatory, governance is robust, and integration aligns with institutional strategy. In an exploratory randomized controlled trial, students using AIDA spent twice as long and visited more pages in the course relative to the control group. ## Key Findings - Early studies (1–3) gathered student preferences for AIDA's design and identified the benefits of 24/7, context-specific support embedded within the learning environment, alongside concerns over ethics, academic integrity, and data privacy; around 20% of students were initially highly sceptical about whether the OU should develop and use AIDA. - Later studies (4–6) explored actual hands-on use. Acceptance increased post-use, especially among initially sceptical participants, with **96% of students expressing interest in having AIDA available in their formal studies** — hands-on use of AIDA's quiz, explain, and chat functions shifted perceptions from abstract potential to tangible usefulness. - In an exploratory randomized controlled trial (Study 6; 115 experimental, 48 control students in an OpenLearn Create course), students supported by AIDA had **twice the usage time** and visited more pages relative to the control group, though no significant differences were found on overall time and other learning process data — possibly due to the exploratory task design and the sandbox environment differing from the main teaching environment. - Mapping the six studies against Sharples' embedded systems framework identified **enabling factors** — senior leadership sponsorship, cross-unit collaboration (between KMi and IET R&D units, Digital Services, and Faculties), iterative DBR refinement, and data-informed decision-making — while also revealing **gaps** in building systems-thinking capacity and institutionalising systemic change (early AIDA work operated in "pilot project" mode). - The paper warns against naive GenAI adoption, citing the demise of KhanMigo (learners were not actually engaging with the chatbot, with limited evidence of positive gains), underscoring that technical capability must be coupled with organisational readiness and governance. ## Study Design & Method This is a six-study, 18-month Design-Based Research (DBR) programme conducted at the Open University, UK, involving 498 students and 20 staff. The research is framed by Sharples' (2025) embedded systems approach, which adapts Bronfenbrenner's (1979) Ecological Systems Theory to distance learning — recognising that how learners engage with AI is influenced by multiple nested layers (peers, materials, tutors, student services, IT support, quality assurance, government regulation, tech companies). It is also informed by the Diffusion of Innovations Theory (Rogers; Jin et al., 2025). Studies 1–3 gathered preferences and perceptions of AIDA's design; Studies 4–6 explored actual hands-on use by students and staff, culminating in Study 6's exploratory randomized controlled trial in an OpenLearn Create course. Findings were mapped against the nine key actions of Sharples' embedded systems framework. ## Implications for AI in Education The study offers practical guidance for higher education (especially distance learning) institutions seeking to adopt GenAI ethically, transparently, and at scale. It demonstrates that purpose-built, contextually tuned AI assistants embedded within the learning environment (as opposed to generic external chatbots) can enhance engagement and perceived value, provided development is participatory (co-designed with students and staff) and integration aligns with institutional strategy. It highlights the unique position of distance learning institutions — where 70% of "students" are already in/at work and where in-person invigilation for academic integrity is unfeasible — making responsible, human-centred GenAI design a particular priority. It connects to [[higher-ed]], [[generative-ai]], [[privacy]], [[human-in-the-loop-ai]], [[learning-analytics]], [[adult-learning]], and [[accessible-learning]], and cautions against over-reliance on tools without organisational readiness and [[governance]]. ## Limitations The RCT (Study 6) was exploratory: students were asked to explore the OpenLearn Create course at their leisure with no credits or external reward, and the sandbox environment differed from the main teaching environment, which may explain the lack of significant differences in process/outcome data. The study focuses on one institution (the Open University), and the AIDA application is institution-specific. The authors acknowledge that early AIDA work operated in "pilot project" mode with limited systems-thinking capacity, and the mapping to Sharples' framework involves interpretive judgment. ## Connected Concepts - [[higher-ed]] - [[generative-ai]] - [[adult-learning]] - [[privacy]] - [[human-in-the-loop-ai]] - [[learning-analytics]] - [[rag]] - [[accessible-learning]] - [[intelligent-tutoring]] ## Connected Articles - [[tzirides-thinking-through-ai-2025]] — Thinking Through AI - [[genai-higher-education-systematic-review-2026]] — GenAI in Higher Education: A Systematic Review - [[ai-adult-learning-guidelines-dis2026]] — AI and Adult Learning Guidelines - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] — Teaching Intro AI Course Redesign - [[taklif-ai-interest-based-personalized-assignments]] — Taklif: AI Interest-Based Personalized Assignments - [[test-driven-ai-assisted-learning]] — Test-Driven AI-Assisted Learning ## Citation Rienties, B., Coughlan, T., Domingue, J., & Herodotou, C. (2026). [*New systems of learning for distance learning institutions? A six-study review of implementing AIDA*](https://doi.org/10.1016/j.caeai.2026.100607). *Computers and Education: Artificial Intelligence*. --- ## [PersonaVLM: Long-Term Personalization for AI Tutors](https://edtechdev.github.io/aied/articles/nie-personavlm-long-term-personalization-2026/) > **PersonaVLM** introduces an agent framework for long-term personalization of multimodal LLMs, enabling AI tutors to remember, reason about, and align with a learner's evolving preferences across hundreds of interaction turns. Tested on 2,000+ curated cases across 200 personas in the Persona-MME benchmark, the framework outperforms GPT-4o by 5.2% in personalization accuracy while operating entirely without proprietary API dependencies — preserving user privacy. **Nie et al. (Nanjing University & ByteDance), CVPR 2026** · arXiv: 2604.13074 · [Project Page](https://personavlm.github.io/) ## Key Findings 1. **The three-capability architecture addresses the core limitation of prior personalization.** Early approaches to [[ai-tutoring]] personalization were static — a one-shot tuning of outputs to a snapshot of user preferences. PersonaVLM introduces a dynamic cycle: *Remembering* (extracting and summarizing chronological multimodal memories), *Reasoning* (multi-turn retrieval across a personalized memory database), and *Response Alignment* (continuously inferring evolving personality from interaction patterns). This moves personalization from a configuration step to an autonomous agent loop. 2. **Memory is structured across four types, mirroring cognitive architectures.** The framework maintains *Core Memory* (foundational user attributes), *Semantic Memory* (event-independent knowledge updated every turn), *Episodic Memory* (timestamped interaction summaries with keywords), and *Procedural Memory* (plans, goals, and habits updated per session). Text embeddings via `all-MiniLM-L6-v2` indexed in FAISS, combined with Grounding DINO for visual concept cropping, enable retrieval at scale within a 128k context window. This approach draws on principles familiar to [[personalized-learning]] systems but extends them to the multimodal, long-horizon setting. 3. **Personality evolves continuously through an Exponential Moving Average mechanism.** The Personality Evolving Mechanism (PEM) infers a per-turn Big Five (OCEAN) personality vector and updates the long-term profile with a dynamic smoothing factor: λ_m = 0.7 − 0.2 · cos(π · min(50, m) / 50). This makes early interactions highly sensitive (capturing initial signals quickly) and stabilizes over time — a design choice that balances responsiveness with robustness. Updates are suppressed when inferred vectors are fully neutral (score 3 on all dimensions), avoiding drift from non-informative turns. 4. **Performance gains are substantial and privacy-preserving by design.** At 128k context, PersonaVLM improves the Qwen2.5-VL-7B baseline by 22.4% on Persona-MME and 9.8% on PERSONAMEM, and outperforms GPT-4o. Critically, the entire training pipeline — 78k SFT samples plus 5.6k GRPO reinforcement learning samples — is built from a self-contained synthesis pipeline generating 30k+ interactions across 500 unique personas (>15% multimodal). No proprietary API calls are needed, eliminating the privacy concerns that shadow many [[conversational-ai-tutors-framework]] deployments in sensitive educational settings. 5. **The Persona-MME benchmark provides the first comprehensive evaluation framework for long-term personalization.** Spanning seven aspects (Memory, Intent, Preference, Behavior, Relationship, Growth, Alignment) and 14 fine-grained tasks at both 32k and 128k context lengths, the 2,034-case benchmark reveals that performance degrades significantly at shorter context windows — long-term memory infrastructure is not a luxury but a necessity for effective personalization. ## Implications for AI in Education PersonaVLM matters for AIEd because it tackles the problem that makes most [[ai-tutor-effectiveness-review]] findings equivocal: personalization that doesn't persist across sessions cannot build the relationship that drives learning gains. A tutor that forgets a student's misconceptions between Monday and Wednesday is barely better than a static problem bank. The PEM mechanism, in particular, offers a path toward [[affective-tutoring]] — systems that adapt not just to what a student knows but to who they are becoming as a learner. The privacy-preserving design is also significant. Schools and districts operating under FERPA, GDPR, or local data protection regimes have been rightly cautious about sending student interaction data to commercial API endpoints. PersonaVLM's fully local inference pipeline — training data synthesized, model run locally — removes that barrier without sacrificing the performance gains that come from long-horizon personalization. This aligns with the growing interest in [[ecnuclaw-k12-personalized-companion]] approaches that prioritize data sovereignty. However, educators and designers should be cautious about the [[correct-answer-trap-ai-tutor]]: even a well-personalized tutor can prioritize affinity over accuracy if alignment is tuned too aggressively. Personalization that mirrors a student's preferences without challenging misconceptions risks reinforcing errors. Future work integrating PersonaVLM-style memory with deliberate [[taklif-ai-interest-based-personalized-assignments]] frameworks — where personalization serves pedagogical goals, not just user satisfaction — would be a productive direction. ## Connected Concepts - [[affective-tutoring]] - [[ai-tutoring]] - [[personalized-learning]] - [[k-12]] - [[llm]] - [[rag]] ## Connected Articles - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalab... - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[ecnuclaw-k12-personalized-companion]] — ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education - [[taklif-ai-interest-based-personalized-assignments]] — Taklif.AI: LLM-Powered Platform for Interest-Based Personalized College Assignments - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning ## Citation Nie, C., Fu, C., Zhang, Y., Yang, H., & Shan, C. (2026). [*PersonaVLM: Long-Term Personalized Multimodal LLMs*](https://arxiv.org/abs/2604.13074). arXiv:2604.13074. --- ## [Not a universal benefit: Examining the differential effects of emotional AI on L2 pre-service teachers' language learning](https://edtechdev.github.io/aied/articles/not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2/) > **Synthesis:** This study challenges the assumption that emotional design in educational AI provides universal benefits, investigating when, for whom and how it impacts L2 vocabulary learning. A quasi-experiment with 147 pre-service teachers found no overall difference in vocabulary acquisition, but a nuanced pattern emerged: the regular agent preserved significantly better learning attitudes and intrinsic motivation for learners with lower baseline proficiency. Qualitative analysis identified four distinct learner archetypes, suggesting emotional AI efficacy may be moderated by these profiles. ## Key Findings This study challenges the assumption that emotional design in educational AI provides universal benefits, investigating when, for whom and how it impacts L2 vocabulary learning. A quasi-experiment with 147 pre-service teachers found no overall difference in vocabulary acquisition, but a nuanced pattern emerged: the regular agent preserved significantly better learning attitudes and intrinsic motivation for learners with lower baseline proficiency. Qualitative analysis identified four distinct learner archetypes, suggesting emotional AI efficacy may be moderated by these profiles. ## Connected Concepts - [[affective-computing]] - [[agentic-ai]] ## Connected Articles - [[ai-in-the-wild-college]] - [[ai-adult-learning-design]] - [[test-driven-ai-assisted-learning]] - [[teaching-intro-ai-course-redesign-bill-of-rights-2026]] ## Citation## Citation Wang, Z., & Pang, H. (2026). [*Not a universal benefit: Examining the differential effects of emotional AI on L2 pre-service teachers' language learning*](https://doi.org/10.1111/bjet.70078). *British Journal of Educational Technology*. --- ## [Not all collaboration benefits from competition: Collaboration modes in a computational thinking game](https://edtechdev.github.io/aied/articles/not-all-collaboration-benefits-from-competition-collaboration-modes-in-a-computa/) > **Synthesis:** This study investigated different collaboration modes and how they interact with competition to influence computational thinking learning, group metacognition and in-game behaviours. In a quasi-experimental 3x2 factorial design with 148 seventh-grade students, results revealed significant main effects of collaboration mode and an interaction effect between collaboration and competition. The cooperative group with competition achieved the highest CT outcomes, while competition undermined the collaborative turn-taking mode. ## Key Findings This study investigated different collaboration modes and how they interact with competition to influence computational thinking learning, group metacognition and in-game behaviours. In a quasi-experimental 3x2 factorial design with 148 seventh-grade students, results revealed significant main effects of collaboration mode and an interaction effect between collaboration and competition. The cooperative group with competition achieved the highest CT outcomes, while competition undermined the collaborative turn-taking mode. ## Connected Concepts - [[collaborative-learning]] - [[computational-thinking]] ## Connected Articles - [[llm-computational-thinking-physics-2026]] - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] - [[computational-thinking-ai-agent-creation]] - [[ai-in-the-wild-college]] ## Citation## Citation Chen, C.-H., & Huang, K. (2026). [*Not all collaboration benefits from competition: Collaboration modes in a computational thinking game*](https://doi.org/10.1111/bjet.70081). *British Journal of Educational Technology*. --- ## [NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models](https://edtechdev.github.io/aied/articles/nsmq-riddles-science-math-benchmark/) > Boateng et al. (2026) introduce **NSMQ Riddles**, a benchmark of 1.8K scientific and mathematical riddles drawn from 11 years of Ghana's **National Science and Maths Quiz** — a live TV competition for senior secondary school students. This is one of the first AI benchmarks originating from the **Global South** for educational evaluation. # NSMQ Riddles: Educational Benchmark from Ghana ## Core Contribution Boateng et al. (2026) introduce **NSMQ Riddles**, a benchmark of 1.8K scientific and mathematical riddles drawn from 11 years of Ghana's **National Science and Maths Quiz** — a live TV competition for senior secondary school students. This is one of the first AI benchmarks originating from the **Global South** for educational evaluation. ## Why It's Distinctive Unlike standard benchmark datasets (MMLU, GSM8K), NSMQ Riddles: - Features **progressive clue revelation** — early clues are vague (worth more points), testing incremental reasoning - Covers **biology, chemistry, physics, and math** at the high school level - Evaluates models against **human student performance** in a competitive format - Represents African educational content, addressing geographic bias in [[stanford-evidence-base-ai-k12-2026]] The benchmark found that even state-of-the-art models (GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.6) **underperform the best student contestants**, highlighting gaps in LLM scientific reasoning. ## Connections to Wiki This benchmark connects to [[teachbench-llm-teaching-evaluation]] as another syllabus-grounded evaluation framework, but from a Global South perspective. It complements the [[educational-vlm-evaluation]] work on DrawEduMath by providing a text-based STEM reasoning benchmark. The focus on competitive quizzing connects to [[automated-question-generation]] research and [[civic-education-ai-lesson-plans]] concerns about AI-generated educational content quality. The finding that LLMs lag behind top human students on these riddles reinforces [[tutoring-specific-vs-general-ai]] concerns — general LLMs may not match specialized educational needs, especially in non-Western contexts. ## Open Questions - How well do [[pedagogical-llm-training]] approaches like EduQwen perform on NSMQ compared to general LLMs? - Can the benchmark be extended to other African and Global South educational systems? - What does the clue-progression format reveal about LLM reasoning vs. retrieval? ## Connected Concepts - [[automated-question-generation]] - [[pedagogical-llm-training]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[teachbench-llm-teaching-evaluation]] - [[educational-vlm-evaluation]] - [[civic-education-ai-lesson-plans]] - [[tutoring-specific-vs-general-ai]] ## Citation al, A.G.B.N.I.S.J.E., and, N.R.A.B.O.S., Large, M.R.F.Q., Models, L., Yeboah3,4, P.A.J.A.M.K.T., and, W.E.A.K.M.N.S.Y., Kumbol2,3, V., & Zurich, E. (2026). [*NSMQ Riddles: A Benchmark of Scientific and Mathematical Riddles for Quizzing Large Language Models*](https://arxiv.org/abs/2605.07051) --- ## [NuclearDiffusion: Text-to-Image Foundation Models for Learning Nuclear Energy Concepts](https://edtechdev.github.io/aied/articles/nuclear-diffusion-text-to-image-learning-2026/) > **Synthesis:** Systematic study of domain-adapted text-to-image models for nuclear engineering education. Fine-tunes Stable Diffusion on nuclear domain images; fine-tuned model achieves 78% domain accuracy vs 12% for base model. Proposes NuclearDiffusion as an educational tool where instructors generate accurate visualizations of nuclear concepts (reactor components, fuel cycles, safety systems). Demonstrates that domain-specific fine-tuning dramatically improves visual correctness for specialized STEM concept illustration. [[generative-ai]], [[generative-ai]], [[stem-education]], content-quality, and [[multimodal]]. Systematic study of domain-adapted text-to-image models for nuclear engineering education. Fine-tunes Stable Diffusion on nuclear domain images; fine-tuned model achieves 78% domain accuracy vs 12% for base model. Proposes NuclearDiffusion as an educational tool where instructors generate accurate visualizations of nuclear concepts (reactor components, fuel cycles, safety systems). Demonstrates that domain-specific fine-tuning dramatically improves visual correctness for specialized STEM concept illustration. ## Connected Concepts - [[generative-ai]] - [[stem-education]] - [[multimodal]] ## Connected Articles - [[mllm-scientific-visualization-literacy]] — Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[genai-runaway-object-math-higher-ed]] — GenAI as a runaway object in higher education: A socio-cultural view on AI-influenced academic practice in mathematics - [[curriculum-as-code-instructional-design-2026]] — Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[prompt-based-programming-lesson]] — Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers ## Citation Mohammed I. Radaideh, Jeremy Moon, Andre Gala-Garza, Emma Son, Yug Shah, & Majdi I. Radaideh (2026). [NuclearDiffusion: Text-to-Image Foundation Models for Learning Nuclear Energy Concepts](https://arxiv.org/abs/2608.04030). arXiv:2608.04030. arXiv preprint (cross-listed cs.GR/cs.CY). --- ## [OATutor: An Open-source Adaptive Tutoring System and Curated Content Library for Learning Sciences Research](https://edtechdev.github.io/aied/articles/oatutor-open-source-adaptive-tutor-2023/) > OATutor (Open Adaptive Tutor) is the first open-source adaptive tutoring system built on Intelligent Tutoring System (ITS) principles, developed at UC Berkeley's CAHL Lab. It combines an MIT-licensed, fully engineered codebase with a Creative Commons (CC BY) algebra content library, knowledge tracing, A/B testing infrastructure, and LTI support — designed to democratize adaptive learning research by removing the barrier to replicating and extending experiments that proprietary platforms create. Despite decades of validated tutoring principles, no adaptive tutoring system had been both fully featured and open-sourced to the research community. This absence forced adaptive learning research onto a small number of proprietary platforms, raising the barrier to entry for researchers wanting to replicate, extend, or field adaptive-learning experiments. OATutor addresses this gap, delivering a system that others can fork, experiment with, and A/B test component-by-component. ## The system OATutor was iteratively developed over three years (from late 2019) with field trials in classrooms and feedback from students, teachers, and researchers. Its core architecture implements established ITS design principles: - **Hints & worked examples** — on-demand declarative help or full worked solutions, shown one at a time to reduce cognitive load. - **Scaffolding / tutored problem solving** — problems broken into steps with sub-questions that communicate the underlying goal structure. - **Mastery estimation** — Bayesian Knowledge Tracing (BKT), a Hidden Markov Model that estimates cognitive mastery per skill, so the tutor withholds problems the student is not ready for and advances them when appropriate. - **Immediate feedback** — real-time correctness feedback upon answer submission. - **Knowledge Component (KC) model** — a centralized skill model mapping each step to skills, defined to promote abstract understanding while remaining fine-grained enough to represent competence. - **Bottom-out hints** — a final hint revealing the answer to prevent dwelling. - **Variabilization** — template variables generate many problem instances, reducing content-author burden. - **A/B testing + data logging** — built-in randomized experimentation and a row-per-interaction logging format familiar to EDM researchers. ### Content and authoring OATutor ships with three Creative Commons (CC BY) OpenStax algebra textbooks (Elementary, Intermediate, College Algebra) curated into the system, with tutoring support authored by the OATutor project. Content follows a hierarchy of **Course → Lessons → Problems → Steps → Tutoring Pathway (hints/scaffolds)**. Authors do not need to know JSON: they author via Google Spreadsheets, and an automated script converts spreadsheets to the JSON content pool, running nightly quality checks. For the College Algebra course, 16 creators completed content in ~6 months with an average of 2.27 hours of training. ### Mastery-based adaptive selection Problems are tagged with skills at the step level via a centralized skill model. Lessons define skills and a target mastery threshold; after each problem, a configurable heuristic selects the next problem the student has not yet mastered — by default, the one with the lowest average posterior mastery across its skills. Learning continues until all lesson skills are mastered. ## Significance for the research community OATutor's design goal is replicability: researchers can run experiments on OATutor, then publish the entire end-to-end framework, content, and platform as a GitHub link for others to replicate, compare, and build on. The open-source, transparent nature also fosters trust and wider adoption. The work is positioned within broader open movements in education — [[open-source]] educational resources, open datasets, and open algorithms (e.g., pyBKT) — and argues it is "long past due" for an ITS-based platform to join them. ## Connection to the wiki OATutor is a foundational example of [[adaptive-learning]] and [[intelligent-tutoring]] research infrastructure, and a companion system for evaluating [[generative-ai]] tutoring capabilities (see the companion learning-gain study). It connects to [[knowledge-tracing]] (BKT mastery estimation), [[personalized-learning]], and [[math-education]], and its open platform model is directly relevant to the [[ai-tutoring]] evidence base and to the automated authoring challenges explored in [[ai-tutor-authoring-promptdecipher]]. ## Connected Concepts - [[intelligent-tutoring]] - [[adaptive-learning]] - [[knowledge-tracing]] - [[open-source]] - [[ai-tutoring]] - [[personalized-learning]] - [[math-education]] - [[edtech-platform]] - [[automated-question-generation]] - [[scaffolding]] ## Connected Articles - [[chatgpt-hints-human-tutor-learning-gains-2024]] — ChatGPT-Generated Help vs. Human Tutor Help - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[access-not-enough-ai-tutoring-2026]] — Access Is Not Enough: AI Tutoring - [[cold-start-knowledge-tracing-safeinsights]] — Cold-Start in Knowledge Tracing - [[adaptive-pretesting-retention]] — Adaptive Pretesting and Retention - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors - [[ai-tutor-authoring-promptdecipher]] — PromptDecipher: AI Tutor Authoring ## Citation Pardos, Z. A., Tang, M., Anastasopoulos, I., Sheel, S. K., & Zhang, E. (2023). [*OATutor: An open-source adaptive tutoring system and curated content library for learning sciences research*](https://doi.org/10.1145/3544548.3581574). In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems*. ACM. https://doi.org/10.1145/3544548.3581574 --- ## [OECD Digital Education Outlook 2026](https://edtechdev.github.io/aied/articles/oecd-digital-education-outlook-2026/) > **OECD flagship report** synthesising empirical evidence and expert insights on generative AI in education. Central finding: general-purpose AI chatbots improve task performance but produce no durable learning gains; purpose-built educational GenAI, co-designed with teachers, is the path to sustained improvement. ## The Core Finding: Performance Is Not Learning The report's most consequential finding is that general-purpose GenAI tools (ChatGPT, Gemini, Claude) improve the *quality of student outputs* on assignments but this advantage **disappears and sometimes reverses** in exams when AI access is removed. This misalignment between task performance and genuine learning is the report's central argument for why purpose-built educational AI is necessary. When students offload cognitive tasks to chatbots, metacognitive engagement drops — the mental processes that turn answers into understanding are short-circuited. See [[genai-performance-vs-learning]] and [[over-reliance]]. ## Educational GenAI: What Works Hybrid systems that combine GenAI with explicit pedagogical models show more promise than general-purpose chatbots. Examples cited include: - **Socratic Playground (SPL)**: Uses Socratic questioning to develop subject knowledge, critical thinking and reflection rather than providing direct answers - **Khanmigo**: Withholds answers and guides reasoning through questioning - **JeepyTA**: AI teaching assistant in university contexts rated comparable to human TAs in clarity and accuracy - **Tutor Copilot**: Mobilises less-qualified tutors effectively through AI support The report draws a sharp line: GenAI tools "designed or used with an intentional pedagogical purpose" produce sustained learning improvements; tools used as answer-dispensing shortcuts do not. See [[ai-tutoring]], [[intelligent-tutoring]], [[codify-socratic-tutoring-programming]]. ## Tutoring: The 9-Percentage-Point Effect A 9-percentage-point increase in student pass rates when low-experience tutors used AI support, with smaller gains for more experienced tutors. Secondary science teachers in England saw a **31% reduction** in time spent on lesson and resource planning. This supports an augmentation model where AI boosts the least experienced practitioners the most. See [[hybrid-human-ai-tutoring-differentiated]]. ## Teacher Agency: Three Paradigms The report's conceptual framework (Ch.7) proposes three paradigms for teacher-AI interaction: 1. **Replacement** — AI takes over tasks; risk of teacher deskilling 2. **Complementarity** — Human judgment paired with machine efficiency 3. **Augmentation** — Teachers and AI work in tandem, critiquing and refining each other's outputs (recommended) The augmentation paradigm preserves professional judgment while achieving the greatest instructional quality gains. See [[genai-can-harm-teaching-rct-2026]]. ## Purpose-Built vs General-Purpose Chapter 8 makes the case for purpose-built educational GenAI systems co-created with teachers and students. These tools would give teachers control over AI behaviour — including setting the level of "hallucinations" — and enable monitoring of student-AI interactions. The tools should align with specific curricula rather than being generic, and maintain teacher autonomy over course design and enactment. ## Collaborative Learning and Creativity GenAI supports collaborative learning in four roles: information hub, personalised material generator, teacher feedback provider, and peer contributor. Studies find small-to-medium improvements in subject learning and large ones in critical thinking and teamwork (Ch.4). For creativity, GenAI works best when used "slowly" for iterative exploration and reflection, not for instant content generation (Ch.5). ## System-Level and Assessment Applications At the institutional level, GenAI enables: curriculum mapping between courses/programs, admissions and career guidance analytics, standardised assessment item generation, interactive writing and speaking assessments, and synthetic datasets for education research (Chs. 11–13). ## Policy Recommendations Four pillars: (1) human-centred teaching and learning with GenAI; (2) investment in educational GenAI R&D grounded in learning science; (3) enabling policy environment for trustworthy GenAI (privacy, safety, bias testing, transparency); (4) equitable digital infrastructure including offline small language models for low-connectivity settings. ## Equity: AI Unplugged A large-scale experiment in rural Brazil (Ch.6) demonstrated that even with intermittent connectivity and minimal equipment, AI could provide feedback and guidance. Small language models running offline on mobile devices are identified as a promising avenue for bridging digital divides. ## Connected Concepts - [[ai-tutoring]] - [[ai-literacy]] - [[generative-ai]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[codify-socratic-tutoring-programming]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[genai-performance-vs-learning]] — Distinguishing performance gains from learning when using generative AI - [[hybrid-human-ai-tutoring-differentiated]] — Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis ## Citation OECD (2026). [*OECD Digital Education Outlook 2026: Exploring Effective Uses of Generative AI in Education*](https://doi.org/10.1787/062a7394-en). OECD Publishing, Paris. --- ## [Peer and AI Review + Reflection (PAIRR): A Human-Centered Approach to Formative Assessment](https://edtechdev.github.io/aied/articles/pairr-ai-peer-review-2025/) > **Synthesis:** Sperber et al. (2025) present the Peer and AI Review + Reflection (PAIRR) model, a human-centered approach to formative assessment that combines peer review best practices with AI review while emphasizing student agency and reflection. In the largest study of college students' use of AI feedback to date (N = 654 across 10 writing courses and three writing-intensive STEM courses at UC Davis), they found that AI feedback is most useful when combined with peer review: the majority of students preferred combined feedback, found the similarity between AI and peer feedback reassuring, valued their complementary perspectives, and developed AI literacy by critically assessing AI outputs. ## Key Findings - The majority of students (58%) preferred to receive feedback from both ChatGPT and peers, while 36% preferred peer feedback alone and only 6% preferred AI feedback alone — AI utility was experienced in the context of human feedback. - AI and peer feedback were often similar and mutually reinforcing: 75% of students reported similarities between peer and ChatGPT feedback, which they described as confirming and strengthening each other, increasing confidence in peer response. - When AI and peer feedback differed, they were complementary: AI feedback was often described as "overly general" (31%) but provided actionable revision strategies and rubric-driven feedback on organization, focus, and structure, while peer feedback was more specific and detailed (28%), drew on contextual knowledge of the course and assignment, and provided an authentic audience and emotional support. - By evaluating AI outputs rather than taking them at face value, students developed [[ai-literacy]] — including learning ethical ways to use AI — while many asserted writerly agency in deciding whether to accept or reject feedback (only 5.3% of students demonstrated overconfidence in AI feedback). - Continued feedback conversations increased perceived utility: 35% of students continued a ChatGPT conversation, and 71% of those preferred combined feedback (vs. 50% who did not). Self-efficacy did not predict continuation, and preferences did not differ statistically between small writing classes and large WI courses. ## Study Design & Method This mixed-methods study implemented PAIRR in 10 distinct writing courses plus three large writing-intensive (WI) courses at a large R1 public university in the western US during winter and spring 2024. The 654 participating students were diverse (37% first-generation, 13% international, 68% multilingual). The intervention sequence: students read and reflected on articles about AI; drafted, provided/received peer review, then prompted ChatGPT for rubric-driven feedback; critically assessed both kinds of feedback and made revision plans; and revised and reflected on the process. Data included pre-/post-surveys, interviews with 4 faculty and 11 students, focus groups with 8 TAs and 1 reader, and 654 students' reflections and feedback assessments. Survey data were analyzed with descriptive/inferential statistics (binary logistic regression and chi-squared test in R); qualitative data were thematically coded in MaxQDA using a stratified sub-sample of 131 students (20%) with open, axial, and selective coding and inter-rater reliability. ## Implications for AI in Education The study offers a research-informed, human-centered model for integrating AI into [[formative-assessment]] in [[writing-education]]. It directly addresses the concern that AI feedback might remove humans from the assessment loop, showing that AI feedback only gains full utility when considered alongside the contextual knowledge and human connection that peers bring. The PAIRR model demonstrates how students' [[metacognition|critical assessment]] of AI outputs builds practical and critical [[ai-literacy]] while exercising writerly agency and discouraging [[over-reliance|overreliance]]. It positions AI as a complement to — not a replacement for — peer and instructor feedback, relevant to [[ai-feedback-quality]], [[self-regulated-learning]], and efforts to bridge equity gaps in writing support. The authors recommend centering humans in AI-integrated feedback, emphasizing metacognitive reflection and agency, and cultivating critical AI literacies that highlight AI's limitations and biases. ## Limitations The study's focus was on student perceptions of AI feedback utility, so it did not directly evaluate AI outputs for bias or quality. Differences in course mode, writing support, instructor experience, rubrics, assignment prompts, and genre across courses may have affected perceptions; the large online PLA course was overrepresented, and its students received only one peer reviewer (vs. two elsewhere) yet got comprehensive TA feedback. ChatGPT 4.0 was released mid-study, and version access was variable. The authors did not distinguish one-shot vs. continued-conversation feedback in the main findings. ## Connected Concepts - [[ai-feedback-quality]] - [[writing-education]] - [[formative-assessment]] - [[ai-literacy]] - [[student-experience]] - [[self-regulated-learning]] - [[metacognition]] - [[academic-integrity]] - [[peer-review]] ## Connected Articles - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and Teacher Feedback - [[multimodal-ai-feedback-learning]] — LLM-Based Multimodal AI Feedback Produces Equivalent Learning - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[learner-centered-feedback-ai]] — Enhancing Learner-Centered Feedback With AI - [[ai-internal-feedback-evaluative-judgments]] — Unravelling Undergraduates' Development of Evaluative Judgments - [[student-rationalization-ai-writing]] — "It's OK Because...": The Wild West of Student Rationalization of AI Writing - [[ai-generated-feedback-higher-ed]] — AI-Generated Feedback in Higher Education ## Citation Sperber, L., MacArthur, M., Minnillo, S., Stillman, N., & Whithaus, C. (2025). [*Peer and AI review + reflection (PAIRR): A human-centered approach to formative assessment*](https://doi.org/10.1016/j.compcom.2025.102921). --- ## [ParaTutor: LLM Mediated Parent Child Tutoring through Role Separated Scaffolding Interface in Real Time](https://edtechdev.github.io/aied/articles/paratutor-parent-child-tutoring/) > **Lan Luo, Anqi Wang, Muzhi Zhou, Junhua Zhu, Jie Cai, Ao Yu, Hui Pan** (2026). arXiv cs.HC ## Key Findings - Effective parent–child tutoring depends on preserving distinct, asymmetric roles: parents guide the learning process while children remain actively engaged in understanding and reasoning. - Recurring challenges include parents struggling to understand problem structure, lacking sufficient knowledge to provide support, and encountering communication difficulties that disrupt shared understanding. - Generic LLM assistance tends to reduce the parent's tutoring role; ParaTutor better preserves parent-led support and sustains children's participation in reasoning through role-separated [[scaffolding]]. - In multi-user learning, the value of LLM support depends not only on model capability but on how support is distributed across users with different roles. - Emotionally supportive language — gentle reminders, positive reinforcement, neutral phrasing such as "let's try a different way" — enhances persistence, reduces frustration, and fosters learner confidence. ## Study Design & Method The work began with a formative study in the Chinese home tutoring context that identified locally common math word problem-solving procedures, common challenges in parental tutoring, and seven strategies commonly adopted by parents. ParaTutor was then developed using multi-agent chatbots, providing parents with guidance for tutoring and children with visual grounding for problem solving. The system was evaluated with 23 parent–child dyads (children aged 10–12) under four tutoring conditions that varied how LLM assistance was delivered. Design features for emotional buffering include sentence starters such as "You're on the right track," "let's think it through together," and "That's okay, let's try another way," which help parents maintain a constructive tone when children struggle or make mistakes. ## Implications for AI in Education ParaTutor illustrates how LLM support for multi-user learning must be distributed by user roles rather than offered as a single generic interface. Compared with general conversational LLM support, ParaTutor kept parents centrally involved by making guidance something they delivered and adapted in interaction rather than merely retrieved from the model. Phase-gated support helped dyads progress without jumping to answers: parents of higher-performing children more frequently chose approaches that encourage independent thinking, reflecting a belief that their role is to support rather than control the learning process. For [[intelligent-tutoring]] and [[human-ai-collaboration]] research, the study contributes design implications for LLM systems that support family learning, particularly in [[math-education]] contexts where LLM accuracy is limited and children should not interact with models independently. ## Connected Concepts - [[math-education]] - [[pedagogical-agent]] - [[reinforcement-learning]] - [[intelligent-tutoring]] - [[lifelong-learning]] - [[personalized-learning]] - [[adaptive-learning]] - [[affective-tutoring]] ## Connected Articles - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges - [[special-r1-rl-special-education]] — Special-R1: Reinforcement Learning for Special Education — Aligning LLM Tutors to Diverse Learners through Disability-Adaptive Training - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions ## Citation Lan Luo, Anqi Wang, Muzhi Zhou, Junhua Zhu, Jie Cai, Ao Yu, Hui Pan (2026). [ParaTutor: LLM Mediated Parent Child Tutoring through Role Separated Scaffolding Interface in Real Time](https://arxiv.org/abs/2606.18030). arXiv:2606.18030. arXiv cs.HC. --- ## [The Paternalistic Filter: Epistemic Injustice and Differential Refusal in LLM-Mediated History Education for Marginalized Romanian Students](https://edtechdev.github.io/aied/articles/paternalistic-filter-llm-history-education/) A systematic API audit of four LLMs acting as history tutors evaluates 1,800 responses about the 1989 Romanian Revolution, exposing a 'paternalistic filter': models differentially refuse or soften answers for marginalized students, reproducing epistemic injustice. The audit reveals that guardrails and refusals are not uniform but patterned by student identity and topic sensitivity. Centers on [[llm]] tutors in [[student-experience]] and the harms catalogued under [[ai-tutor-safety-harms]], with strong links to [[bias-mitigation]] and [[equity]] in [[higher-ed]]. It extends [[over-reliance]] concerns by showing even refusal behavior can entrench inequality, and calls for fairness evaluations that account for differential treatment across student groups. ## Connected Concepts - [[llm]] - [[student-experience]] - [[bias-mitigation]] - [[equity]] - [[higher-ed]] - [[over-reliance]] ## Connected Articles - [[ai-tutor-safety-harms]] ## Citation Alexis Popovici, Andrei Ionascu, Adrian-Marius Dumitran (2026). [The Paternalistic Filter: Epistemic Injustice and Differential Refusal in LLM-Mediated History Education for Marginalized Romanian Students](https://arxiv.org/abs/2607.11292). arXiv:2607.11292. arXiv preprint. --- ## [Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components](https://edtechdev.github.io/aied/articles/pattern-kc-programming-recommendation/) Introductory programming instruction relies on hands-on practice and short learning activities to support mastery of foundational concepts. Although many such learning resources exist, organizing and linking these items in instructionally meaningful ways is challenging without time-intensive expert curation. This study investigates the use of pattern-based Knowledge Components (KCs) to automatically identify code-based learning resources targeting similar concepts. In our approach, pattern-based KCs are extracted from each code sample, and related activities are identified by measuring similarity between the KC sets associated with each activity. By leveraging alignment at the level of semantically important programming patterns, this method supports contextually appropriate and pedagogically useful recommendations. We evaluate our approach on an expert-organized corpus of introductory Python materials in which instructors grouped items into bundles based on conceptual similarity. Results show that our pattern-based KC approach retrieves resources that align with this expert organization, and outperformed representative KC- and embedding-based baselines across standard ranking evaluations. Overall, the framework supports targeted, concept-oriented guidance for programming learners and can help instructors organize, bundle, and recommend instructional content at scale. Pattern-based Knowledge Components extracted from code samples enable automatic, instructionally-meaningful recommendation of programming practice activities; on an expert-bundled Python corpus the approach beats KC- and embedding baselines, helping instructors organize and recommend content at scale. This work connects to core wiki themes: [[knowledge-tracing]] [[personalized-learning]] [[adaptive-learning]] [[intelligent-tutoring]] [[programming-its]]. It highlights how generative-AI tooling is reshaping both what learners do and how educators structure support, reinforcing the need for design that preserves authentic engagement rather than enabling shallow bypass. ## Connected Concepts - [[knowledge-tracing]] - [[personalized-learning]] - [[adaptive-learning]] - [[intelligent-tutoring]] ## Connected Articles - [[programming-its]] ## Citation Hoq, Pitts, Duan, Narayanan, Hassany, Lan, Brusilovsky & Akram (2026). [Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components](https://arxiv.org/abs/2607.05409). arXiv:2607.05409. CSEDM Workshop 2026 (arXiv preprint). --- ## [From Prompts to Verified Loops: The PCHL-HE Framework for Generative AI-Assisted Educational and Research Content Creation in Higher Education](https://edtechdev.github.io/aied/articles/pchl-he-framework-genai-content-creation-2026/) > **Synthesis:** This conceptual preprint develops the Prompt-Context-Harness-Loop Framework for Higher Education (PCHL-HE), a pedagogically grounded vocabulary that differentiates four increasingly complex configurations of generative-AI interaction — prompt, context, harness, and verified loop — across eight dimensions of control, grounding, orchestration, and oversight. ## Key Findings 1. **Higher education lacks a vocabulary for complex AI interaction.** GenAI is moving beyond isolated prompt-response exchanges toward systems that curate information, call tools, retain state, verify intermediate products, and repeat actions under explicit control, yet HE lacks a pedagogically grounded way to select among these increasingly complex forms. 2. **Four task configurations.** PCHL-HE differentiates four configurations: *prompt* engineering designs a bounded instruction; *context* engineering designs the information environment available to the model; *harness* engineering designs a complete, multi-step, tool-mediated working pass; *loop* engineering closes a verified, repeatable loop. 3. **Eight differentiating dimensions.** The framework spans unit of control, information grounding, workflow complexity, temporal horizon, tool orchestration, feedback and iteration, verification, and human oversight. 4. **Integrative construction.** The framework is built through synthesis of research on prompting, retrieval and context management, agentic systems, human interaction with automation, AI in HE, assessment, feedback, academic integrity, and design science. ## Implications The PCHL-HE framework addresses a real pedagogical gap: as AI moves toward [[agentic-ai]] and tool-orchestrated workflows, teachers and students need language to reason about what level of control, grounding, and oversight an AI interaction requires. It extends [[prompt-engineering]] into a broader design space that includes [[rag]]-style context engineering and verified agentic loops. For [[higher-ed]] and [[instructional-design]], the framework offers a taxonomy for [[ai-literacy]] curricula and for designing assignments that match task complexity to appropriate AI configurations — for example, using verified loops only where intermediate products and oversight matter, which bears on [[academic-integrity]] and [[assessment]]. The explicit attention to verification and human oversight connects to [[human-in-the-loop-ai]] and to debates about when AI-generated content is ready for use. The framework is conceptual and would benefit from empirical validation, but it provides a useful shared vocabulary for [[faculty-development]] and for institutional [[ai-governance-education]] conversations about tool selection and policy. ## Connected Concepts - [[academic-integrity]] - [[agentic-ai]] - [[ai-governance-education]] - [[ai-literacy]] - [[generative-ai]] - [[faculty-development]] - [[generative-ai]] - [[higher-ed]] - [[human-in-the-loop-ai]] - [[instructional-design]] - [[prompt-engineering]] - [[rag]] - [[hallucination-risk]] ## Connected Articles - [[learnity-graphs-lifelong-learning-framework-2026]] — Learnity graphs framework - [[genai-higher-education-systematic-review-2026]] — GenAI in higher education review - [[ai-uk-higher-education-policy-2026]] — AI in UK higher education policy - [[xie-hillm-cd-2026]] — HILLM curriculum design ## Citation Nalyvaiko, O. (2026). [*From Prompts to Verified Loops: The PCHL-HE Framework for Generative AI-Assisted Educational and Research Content Creation in Higher Education*](https://osf.io/wudk7). EdArXiv preprint. --- ## [Pedagogical Safety in Educational Reinforcement Learning](https://edtechdev.github.io/aied/articles/pedagogical-safety-rl/) > As reinforcement learning personalizes instruction in intelligent tutoring systems, there is no formal framework for pedagogical safety — a critical gap. > First formal framework for defining and detecting *pedagogical safety* violations in RL-based AI tutoring systems. ## The Problem: Reward Hacking in Educational RL **Olukola & Rahimi (2026)** identify a critical gap: as reinforcement learning personalizes instruction in ITS, **there is no formal framework for pedagogical safety**. ### What is "Reward Hacking" in Education? In RL, agents optimize for a reward signal. In education, this can lead to: - **Test score inflation:** Optimizing for quiz performance without deep learning - **Engagement gaming:** Keeping students "engaged" via entertainment rather than learning - **Short-term gains:** Sacrificing long-term retention for immediate performance metrics ## Four-Layer Model of Pedagogical Safety | Layer | Focus | Safety Criterion | |-------|-------|------------------| | **Structural** | System architecture | No single point of pedagogical failure | | **Progress** | Learning trajectory | Demonstrated progress toward learning objectives | | **Engagement** | Motivation quality | Engagement that supports (not replaces) learning | | **Outcome** | Transfer & retention | Learning persists beyond tutored context | ## Connection to SafeTutors Taxonomy This complements the [[ai-tutor-safety-harms|SafeTutors taxonomy]] which identifies 11 harm dimensions: - **Over-reliance** (learner dependency on AI) - **Misinformation** (incorrect content delivery) - **Bias** (demographic performance gaps) Pedagogical Safety in RL adds: **system-level safety** (how the *learning algorithm itself* can cause harm through optimization failures). ## Detection Methods The paper proposes detecting reward hacking via: 1. **Discrepancy auditing:** Compare RL-optimized metrics vs. independent learning measures 2. **Policy inversion:** Analyze what behaviors the RL policy actually encourages 3. **Long-term tracking:** Monitor retention and transfer (not just immediate performance) ## Implications - **RL in education requires safety frameworks** beyond general AI safety (pedagogical validity is domain-specific) - **Reward design matters:** Poorly specified educational rewards can optimize for *appearing* to teach - **Audit infrastructure:** ITS using RL need interpretable policy inspection tools This parallels concerns in [[educational-llm-alignment]] where benchmark misalignment with teaching quality reveals similar optimization gaps. ## Connected Concepts - [[reinforcement-learning]] - [[affective-tutoring]] - [[knowledge-tracing]] - [[help-seeking]] - [[pedagogical-llm-training]] - [[affective-computing]] - [[lifelong-learning]] - [[socratic-method]] ## Connected Articles - [[ai-tutor-safety-harms]] - [[educational-llm-alignment]] ## Citation Rahimi, A.O.O.N. (2026). [*Pedagogical Safety in Educational Reinforcement Learning*](https://arxiv.org/abs/2604.04237) --- ## [The Pedagogy of AI Mistakes: Fostering Higher-Order Thinking](https://edtechdev.github.io/aied/articles/pedagogy-ai-mistakes/) > An instructional approach that deliberately leverages AI errors, hallucinations, and limitations as teaching tools to foster higher-order thinking. Rather than viewing AI mistakes as failures to be avoided, this pedagogy treats them as cognitive provocations that demand analysis, evaluation, and reflection from students. Proposed by Hosseini (2026) in a database design course context. # Pedagogy of AI Mistakes ## Definition An instructional approach that deliberately leverages AI errors, hallucinations, and limitations as teaching tools to foster higher-order thinking. Rather than viewing AI mistakes as failures to be avoided, this pedagogy treats them as cognitive provocations that demand analysis, evaluation, and reflection from students. Proposed by Hosseini (2026) in a database design course context. ## Mechanism Students interact with AI-generated outputs that contain intentional or known errors. They must: 1. **Analyze** the output for correctness against disciplinary standards 2. **Evaluate** where and why the AI went wrong 3. **Reflect** on what the error reveals about both the domain and AI limitations This maps directly to the upper levels of Bloom's taxonomy (Analyze, Evaluate, Create) and engages metacognitive processes central to [[metacognition]]. ## Relationship to Existing Approaches - Complements [[socratic-ai-dialogue]]: while Socratic approaches use questions to guide reasoning, mistake pedagogy uses erroneous outputs as the provocation - Extends [[ai-literacy]]: students learn not just to use AI but to critically evaluate its outputs - Addresses the [[llm-fallacy-misattribution]] problem by making AI's limitations visible and discussable - Contrasts with [[tutoring-specific-vs-general-ai]]: here the AI's imperfection is the feature, not the bug ## Open Questions - Does mistake-based pedagogy transfer across disciplines beyond STEM? - What is the optimal error difficulty — too obvious vs. too subtle? - How does this approach affect trust in AI tools long-term? ## Connected Concepts - [[metacognition]] - [[socratic-ai-dialogue]] - [[ai-literacy]] ## Connected Articles - [[llm-fallacy-misattribution]] - [[tutoring-specific-vs-general-ai]] ## Citation Hosseini, A.H. (2026). [*The Pedagogy of AI Mistakes: Fostering Higher-Order Thinking*](https://arxiv.org/abs/2605.05472) --- ## [Using the Pepper Robot to Support Sign Language Communication](https://edtechdev.github.io/aied/articles/pepper-robot-sign-language-lis-2025/) > **Synthesis:** Bolla et al. (2025) investigate whether the commercial Pepper social robot can produce intelligible Italian Sign Language (LIS) signs and short signed sentences, addressing the underexplored accessibility of social robots for Deaf users. With the help of a Deaf student and an expert interpreter, they co-designed and implemented 52 LIS signs on Pepper using either manual animation techniques or a MATLAB-based inverse kinematics solver. An exploratory user study with 12 participants proficient in LIS (both Deaf and hearing) assessed the intelligibility of the robot's signs. ## Key Findings - Social robots are increasingly used in public and assistive settings, but their **accessibility for Deaf users remains underexplored**. - Italian Sign Language (LIS) is a fully-fledged natural language with complex **manual and non-manual components**, posing a challenge for robot communication. - The team **co-designed and implemented 52 LIS signs on Pepper** using manual animation techniques or a MATLAB-based inverse kinematics solver. - An exploratory user study with **12 participants proficient in LIS (Deaf and hearing)** assessed sign intelligibility, with the aim of fostering more inclusive human-robot interaction in social environments (hospitals, airports, educational settings). ## Study Design & Method This is an **exploratory design and user-study** in human-robot interaction. The researchers co-designed 52 Italian Sign Language signs with a Deaf student and an expert interpreter, implementing them on the Pepper platform using two techniques (manual animation and MATLAB-based inverse kinematics). An exploratory user study with 12 LIS-proficient participants (both Deaf and hearing) evaluated the intelligibility of the produced signs and short signed sentences, examining the feasibility of robot-mediated sign-language communication. ## Implications for AI in Education The work extends [[social-robots]] into [[equity-in-ai-education]] by exploring robots' capacity to communicate in [[special-education]], which could foster more inclusive human-robot interaction in educational settings and support Deaf learners. It connects to [[human-robot-interaction]], [[accessible-learning]], and [[special-education]], highlighting both the potential and the technical challenges of enabling robots to produce intelligible sign language. For educators, it points toward socially assistive robots as tools for inclusive classrooms, while underscoring the need for co-design with Deaf communities. ## Limitations The user study was exploratory with a small sample (12 participants), and the intelligibility of robot-produced LIS was partial and context-dependent. The technical implementation (52 signs) covers a limited vocabulary, and the non-manual components of LIS (facial expression, body posture) that are crucial to meaning were challenging for the robot to reproduce. Findings are specific to the Pepper platform and Italian Sign Language. ## Connected Concepts - [[social-robots]] - [[human-robot-interaction]] - [[special-education]] - [[equity-in-ai-education]] - [[special-education]] - [[accessible-learning]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[social-robot-study-companions]] — Social Robots as Study Companions - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo: Digital Storytelling Robots and Motivation ## Citation Bolla, G., Bolla, M., Gena, C., Mazzei, A., Donini, M., & Lillo, A. (2025). *Using the Pepper robot to support sign language communication*. arXiv:2509.09889. --- ## [Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study](https://edtechdev.github.io/aied/articles/persistent-ai-agents-academic-research/) ## Overview This is the first empirical study of what happens when AI agents are embedded **persistently** in a real academic research environment — with durable memory, local files, external tools, scheduled routines, delegated roles, and explicit safety protocols. Over 96 active days (January 31 to May 25, 2026), the researcher-agent ecosystem generated 75,671 de-duplicated telemetry records, 23,710 assistant messages, and 73.95 million tokens (82.9% cache reads). The study introduces **PARE-M** (Persistent Agentic Research Environment Measurement), a framework covering architecture, utilization, artifact production, resource use, reproducibility, and governance. ## Key Findings The workflow was overwhelmingly **cache-dominant** (82.9% cache reads), suggesting that persistent agentic environments shift the economic unit from **cost per token** to **cost per completed artifact**. With 17 configured agents, 502 memory-related files, and 57 skill files, the ecosystem resembles the [[agentic-ai]] vision but at the individual-investigator scale. The study also recorded 889 failure, verification, correction, or protocol-proxy events — roughly one intervention every 1.5 hours of active system time. This aligns with findings from [[ai-productivity-moderation]] research showing that AI productivity gains require active human involvement rather than passive delegation. ## Implications for AI in Education Research This study is directly relevant to [[agentic-workflows-education]] research. The PARE-M framework provides vocabulary for measuring and comparing persistent agent deployments in educational contexts — whether for faculty research, [[faculty-development]], or student-facing [[intelligent-tutoring]] systems. The cache-dominance finding challenges current pricing models and suggests that institutional AI deployments should optimize for artifact throughput rather than token costs. The 17-agent configuration demonstrates how [[ai-changing-teaching-workflows]] might scale within academic institutions. If a single investigator can productively orchestrate 17 specialized agents, the same could apply to a course with multiple AI teaching assistants, each with distinct roles (grader, discussion moderator, content curator, etc.). ## Methodological Contribution: PARE-M PARE-M provides six measurement dimensions that could be adapted for [[learning-analytics]] in AI-augmented classrooms: architecture mapping, utilization tracking, artifact production metrics, resource consumption, reproducibility assessment, and governance event logging. This structured approach to measuring human-AI ecosystems addresses the [[ai-higher-ed-bridge-gap]] between technological capability and institutional adoption. ## Connected Concepts - [[agentic-ai]] - [[faculty-development]] - [[intelligent-tutoring]] - [[learning-analytics]] ## Connected Articles - [[ai-productivity-moderation]] - [[agentic-workflows-education]] - [[ai-changing-teaching-workflows]] - [[ai-higher-ed-bridge-gap]] ## Citation Alzahrani, A. H. (2026). [*Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study*](https://arxiv.org/abs/2605.26870). arXiv:2605.26870. --- ## [Students' Epistemological Beliefs and their Chatbot Preferences in AI-mediated Physics Learning](https://edtechdev.github.io/aied/articles/physics-chatbot-epistemological-beliefs-2026/) > **Synthesis:** Sirnoorkar & Mamidpalliwar (2026) investigate the association between introductory physics students' preferences for chatbot behavior and their epistemological beliefs, using a custom online waves module with simulations integrated with a chatbot. Preferences were captured through three options (guided-inquiry, direct answer, and a combination); beliefs via the EBAPS survey. Students who preferred chatbots that initially engage in guided-inquiry but provide answers when explicitly sought ('Combination') demonstrated more sophisticated epistemological beliefs than those preferring answer-providing chatbots, though no association was found between EBAPS total scores for guided-inquiry vs answer-oriented preferences, and differences did not survive Bonferroni adjustment. The work connects to [[physics-education]], [[pedagogical-agent]], and [[socratic-method]] research. ## Chatbot Preference and Epistemological Beliefs Generative AI promises to transform higher education. This study asks whether introductory students' preferences for chatbot behavior are associated with their epistemological beliefs surrounding physics, in a custom-built online module on waves containing simulations integrated with a chatbot. ## Design Chatbot preferences were captured through three provided options — guided-inquiry, direct answer, and a combination of inquiry and answer. Epistemological beliefs were captured through the standardized Epistemological Beliefs Assessment for Physical Sciences (EBAPS) survey. ## Findings Students who preferred chatbots that initially engage them in guided-inquiry but provide answers when explicitly sought ('Combination') demonstrated more sophisticated epistemological beliefs than those who preferred answer-providing chatbots. No association was observed between EBAPS total scores among guided-inquiry vs answer-oriented preference groups, and differences did not remain statistically significant after Bonferroni adjustment. Proceedings of the 2026 Physics Education Research Conference. ## Connected Concepts - [[physics-education]] - [[pedagogical-agent]] - [[socratic-method]] - [[ai-literacy]] - [[student-experience]] - [[generative-ai]] - [[stem-education]] - [[higher-ed]] - [[socratic-ai-dialogue]] - [[metacognition]] ## Connected Articles - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[becker-chatgpt-typology-physics-2026]] - [[genai-tutor-engagement-patterns]] - [[chatgpt-feedback-engagement-genai]] ## Citation Sirnoorkar, A., & Mamidpalliwar, O. (2026). [*Students' epistemological beliefs and their chatbot preferences in AI-mediated physics learning*](https://arxiv.org/abs/2607.29385). Proceedings of the 2026 Physics Education Research Conference. arXiv:2607.29385. --- ## [Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning](https://edtechdev.github.io/aied/articles/physiological-signals-exam-outcomes-ml/) > Investigates ML models to predict exam outcomes from physiological data (electrodermal activity, heart rate, skin temperature) collected during exams. Evaluates logistic regression, random forest, SVM, transformers, LSTM, and GRU. Random forest often outperformed deep learning models while offering interpretability. Highlights value of physiological data for understanding student stress and real-time intervention potential. ## Key Findings - Random forest predicts exam outcomes from physiological signals as well as deep learning models, with better interpretability and computational efficiency. - Random forest achieved the highest mean performance (0.96), effectively modeling non-linear relationships in the data. - GRU and LSTM models yielded reliable results with mean performance metrics of 0.92 and 0.93, demonstrating excellent generalization to unseen data. - Transformers showed notable versatility on tabular data with a slightly lower mean metric of approximately 0.90, comparable to the LSTM and GRU models. - SVM exhibited moderate performance (mean metrics around 0.74), while logistic regression performed lowest with mean metrics below 0.42, making it ill-suited to the dataset's complexity. ## Study Design & Method Physiological stress indicators — electrodermal activity, heart rate, and skin temperature — were collected during examination sessions and analyzed to uncover their association with academic performance. A diverse set of machine learning approaches was employed, ranging from standard models (logistic regression, random forest, support vector machines) to advanced architectures (transformers, long short-term memory, and gated recurrent unit models), with a key focus on assessing the adaptability of transformers in processing numerical data. Standard performance metrics — accuracy, precision, recall, and F1-score — were used to compare model efficacy, and trend analysis was conducted across 30 replicates to examine the mean performance and consistency of the models. ## Implications for AI in Education By elucidating the relationships between physiological signals and academic performance, this study contributes to understanding the stressors that affect students' mental health and promotes leveraging physiological data to enhance student well-being and academic outcomes. The results highlight the value of physiological data for real-time intervention: stress indicators gathered unobtrusively during exams could feed [[learning-analytics]] and [[affective-computing]] pipelines that flag struggling students before outcomes are finalized. The study also underscores the importance of experimenting with a broad class of models aligned with the objectives of the problem at hand, balancing precision, efficiency, and interpretability — a practical lesson for [[student-modeling]] in educational settings where model transparency matters for trust and adoption. ## Connected Concepts - [[affective-computing]] - [[learning-analytics]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[administrator]] - [[teacher-ai-competency]] - [[help-seeking]] - [[pedagogical-agent]] ## Connected Articles - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[dynamic-skill-matching-capstone-teams]] — Improving Capstone Team Outcomes through Dynamic Skill Matching and Preference Alignment - [[hypergamification-game-engine-lms]] — Hypergamigication Through Integrating Game Engines and Learning Management Systems: Ender's Game - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments ## Citation Lala Yamazaki, Ramchandra Rimal (2026). [Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning](https://arxiv.org/abs/2606.14960). arXiv:2606.14960. arXiv preprint. --- ## [Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction](https://edtechdev.github.io/aied/articles/polished-artifacts-fragile-engagement-2026/) > **Synthesis:** Drawing on CSCL research traditions, this paper conceptualizes the risk of reduced epistemic effort when learners use generative AI to produce knowledge artifacts. It identifies two strands of risk: a social-cognitive strand grounded in automation bias (attributing greater epistemic competence to AI) and an artifact-oriented strand focused on polished external artifacts inducing epistemic closure. The paper appeals to structure AI participation as an argumentative partner or challenger to preserve conflict and iterative refinement without diminishing human epistemic effort. ## Key Findings Drawing on CSCL research traditions, this paper conceptualizes the risk of reduced epistemic effort when learners use generative AI to produce knowledge artifacts. It identifies two strands of risk: a social-cognitive strand grounded in automation bias (attributing greater epistemic competence to AI) and an artifact-oriented strand focused on polished external artifacts inducing epistemic closure. The paper appeals to structure AI participation as an argumentative partner or challenger to preserve conflict and iterative refinement without diminishing human epistemic effort. The work contributes to understanding of [[collaborative-learning]] in educational contexts, with implications for [[cognitive-offloading]], [[metacognition]]. ## Connected Concepts - [[collaborative-learning]] - [[cognitive-offloading]] - [[metacognition]] - [[critical-thinking]] - [[generative-ai]] ## Connected Articles - [[cognitive-offloading-speedup-illusion]] - [[efficiency-gain-illusion-ai-overreliance]] - [[critical-genai-use-predictors]] ## Citation## Citation Kimmerle, J. (2026). [*Polished Artifacts, Fragile Engagement? Tackling the Challenge of Reduced Epistemic Effort in Human-AI Knowledge Construction*](https://doi.org/10.35542/osf.io/b8zjk_v1). EdArXiv preprint. --- ## [The (im)possibility of AI literacy](https://edtechdev.github.io/aied/articles/possibility-ai-literacy-critical-editorial/) > **Synthesis:** Pangrazio (2026) offers a critical editorial questioning whether AI literacy is a meaningful or even achievable goal. Tracing the history of literacy from its elite origins through mass institutionalization, she argues that AI literacy has been positioned as a "cure-all" — a solutionist, normative response to the complex and evolving phenomenon of AI. The analysis asks what "text" AI literacy actually refers to when AI is embedded, opaque, self-evolving, and deeply entangled in platforms and power structures, and whether current AI literacy discourse can move beyond instrumental skill-based framings to address the epistemological and political dimensions of AI — proposing that literacy, rights, and sovereignty responses must be developed in tandem. ## Key Findings - **Literacy is never neutral:** tracing literacy from sixteenth-century elite domains through the Protestant Reformation, mass institutionalization, and Freirean critical literacy, Pangrazio argues literacy has always been tied to power, access, cultural authority, and social order — yet this contested history is forgotten in the rush to make AI literacy a "cure-all." - **What is the "text" of AI?** Conventional literacy applies to a text (a book, movie, article) that can be decoded and comprehended. For AI, the "text" is unclear — the algorithm? the LLM? the data? This makes a critical-literacy deconstruction (identifying assumptions, contradictions, intertextual references) very difficult. - **AI is embedded, opaque, and evolving:** AI is often not an isolated tool but part of an ecosystem (e.g., Copilot inside Microsoft 365), can be self-evolving/autonomous, is deliberately opaque (its construction is the domain of computer scientists), and most literacy approaches equate AI with generative AI, overlooking content-moderation and other instantiations. - **Programmatic practices:** AI reshapes practices, values, and behaviours in ways that make critique harder — it curates and shapes public discourse while also being an object to discuss and regulate (Ananny's public object/public agent distinction). Criticality requires independent thought not shaped by the very technology being scrutinized. - **Digital literacies are declining and commercially captured:** despite ~30 years of digital-literacies research, standardized tests (ACER 2023: only 46% of year-10 students proficient; ICILS 2023 declining) show digital literacy is falling. Meanwhile commercial tech companies (Microsoft, Google) have moved in to define literacies around their products, and AI literacy has been "cleaved off" from the broader field of digital literacies. - **Literacy may not be the right response:** following Eynon, literacy "responsibilises the end user" rather than exploring regulation. Pangrazio proposes that literacy, rights (safe/ethical, unbiased, privacy-protecting AI), and sovereignty (countries/institutions developing their own AI and controlling data) responses should be developed in tandem, alongside speculative and interpretive responses. ## Study Design & Method This is a **critical editorial** (not an empirical study) in *Learning, Media and Technology*. It draws on a short history of literacy (Graff; Freire; Green; Janks; Luke), scholarship on critical digital literacies and algorithmic accountability (Ananny & Crawford; Seaver; Suchman; Nichols et al.; Pangrazio & Sefton-Green), and international assessments of digital literacy (ACER 2023; ICILS 2023). The argument proceeds through conceptual analysis of three challenges to AI literacy — the embedded/opaque/evolving nature of AI, the "programmatic practices" of AI, and the history of digital literacies — before proposing alternative (rights and sovereignty) responses. ## Implications for AI in Education The editorial offers a critical corrective to the proliferation of AI literacy frameworks in education, arguing that [[ai-literacy]] is often deployed as a solutionist, instrumental "cure-all" that overlooks questions of power, agency, and ideology — the cornerstones of [[critical-thinking|critical]] and [[equity|equitable]] approaches. It cautions that AI literacy programmes from commercial tech companies (Google, Microsoft) can only produce "partial forms" of the knowledge and skills required, and that AI literacy should be located within the broader field of critical digital literacies rather than treated as a separate, rarefied programme. For educators, it argues institutions must be reconfigured to find time and space for critical engagement with AI, supporting students to write with, redesign, or even resist AI. It connects to [[ai-education]], [[ethics]], and [[generative-ai]], and resonates with critical work on AI literacy and power ([[ai-literacy-power-knowledge|AI literacy as power-knowledge]]). ## Limitations As an editorial, the article presents an argumentative and conceptual position rather than empirical evidence; it does not offer a practical AI literacy framework or curriculum. Its historical and theoretical framing draws on Western (particularly Anglophone and Freirean) literacy traditions, and the proposal for rights- and sovereignty-based responses is programmatic rather than operationalized. The critique of commercial capture is illustrative rather than systematically evidenced. ## Connected Concepts - [[ai-literacy]] - [[critical-thinking]] - [[ethics]] - [[equity]] - [[ai-education]] - [[generative-ai]] ## Connected Articles - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] — Metacognitive AI Literacy: Beyond the Skills Gap - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy Interaction Patterns - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy ## Citation Pangrazio, L. (2026). [*The (im)possibility of AI literacy*](https://doi.org/10.1080/17439884.2026.2615553). *Learning, Media and Technology*, 51(1), 1–7. --- ## [Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills?](https://edtechdev.github.io/aied/articles/post-covid-ict-career-aspirations/) **Post-COVID ICT Career Aspirations** uses PISA 2018 and 2022 country-level data to investigate whether students entering the generative AI era have adequate educational foundations. Using a mixed-methods approach including Variational Autoencoders for latent representation learning, the study finds that ICT career aspirations have increased globally but unevenly. Digital skills are the strongest and most consistent predictor of ICT aspirations, while teacher support plays a complementary role and student autonomy shows only weak, context-dependent effects. The findings challenge simplistic narratives about [[ai-literacy]] — rising interest in ICT careers does not automatically mean students have the foundational skills to succeed in AI-driven labor markets. This connects to [[digital-literacy-illusion]] findings about student overconfidence, and to [[teacher-ai-adoption-confidence]] research on the complementary role of teacher support. The multidimensional nature of educational readiness aligns with [[teacher-ai-competency]] frameworks, and the uneven global distribution of digital skills raises [[equity]] concerns previously documented in [[ai-education-global-capacity]]. ## Connected Concepts - [[ai-literacy]] - [[teacher-ai-competency]] - [[equity]] ## Connected Articles - [[digital-literacy-illusion]] - [[teacher-ai-adoption-confidence]] - [[ai-education-global-capacity]] ## Citation Diana Maria Popa, Simona-Vasilica Oprea, Adela Bâra (2026). [Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills?](https://arxiv.org/abs/2605.27391). arXiv:2605.27391. arXiv preprint. --- ## [When AI Does the Work, What Is Learning For? Post-Instrumental Learning and the Risk of Capacity Dissolution](https://edtechdev.github.io/aied/articles/post-instrumental-learning-capacity-dissolution/) Argues that as AI systems become capable of producing the artifacts through which institutions recognize competence, existing ethical frameworks centered on AI failures become insufficient. Develops the concept of "post-instrumental learning" and warns that each technical improvement appears to weaken the case for human learning itself, risking "capacity dissolution." ## Key Findings - If the case for learning rests only on current AI failures — bias, opacity, hallucination, labor extraction, privacy risk, and weak accountability — then each technical improvement appears to weaken it. - The article develops an idealization of AI that executes specified tasks flawlessly while lacking authority over purposes, legitimacy, and responsibility, and argues for post-instrumental learning that preserves the capacities people and institutions need when many useful outputs can be delegated. - Five capacities are analyzed — end-setting, reason-giving, contestability, refusal/revision, and participation — and their erosion is named capacity dissolution. - The central case is [[assessment]] under generative AI: when a polished artifact no longer reliably evidences understanding, institutions must assess the learner's accountable relation to AI-mediated work rather than the artifact alone. - The takeaway is practical: AI governance should evaluate not only whether systems perform well, but also whether their deployment leaves people able to understand, challenge, revise, and share responsibility for the practices those systems mediate. ## The Central Case: Assessment under Generative AI The argument opens with a recognizable scene: a student submits a fluent essay and cannot explain its central claim; a manager approves an elegant market analysis without knowing which assumptions make it plausible. In each case the output looks successful, but the relation between the person, the institution, and the reasons behind the work has become thin. Existing criticisms of large-scale AI systems — that they reproduce inequality, hide responsibility, depend on invisible labor, and deepen relational injustice — remain indispensable, but they do not fully answer the educational question. Treating learning merely as preparation for producing outputs makes AI improvement a reason to produce those outputs with less human effort, a conclusion the author argues is too quick. ## Implications for AI Governance The article translates the argument into a deployment-review question: what will people stop learning if this system works? Losing obsolete routines may be harmless, but losing the ability to set goals, give reasons, contest, and participate is not. The practical right of contestability is not a demand that everyone master every technical detail: a student need not reproduce every step of a model's generation to defend a claim made in a paper. Rather, each person needs a path from a problem they experience to reasons they can understand and, where appropriate, to a process that can revise the decision — a path that depends on trained intermediaries, accessible records, meaningful appeal, and public settings where reasons can be tested. The article warns that when privileged users can contest AI while marginalized users receive automated completion or automated suspicion, the institution has changed the distribution of expertise while calling the result access. For [[ai-governance-education]] and [[over-reliance]] debates, this reframes the goal of schooling: preserving capacities such as [[ethics]]-inflected judgment and accountable participation rather than maximizing efficient output generation. ## Connected Concepts - [[over-reliance]] - [[assessment]] - [[ethics]] - [[plagiarism-detection]] - [[student-misconceptions-ai]] - [[cognitive-offloading]] - [[prompt-engineering]] - [[affective-tutoring]] ## Connected Articles - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[efficiency-gain-illusion-ai-overreliance]] — The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback Across a Semester of Writing - [[buggy-genai-code-student-responses]] — When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code ## Citation Kai Yao (2026). [When AI Does the Work, What Is Learning For? Post-Instrumental Learning and the Risk of Capacity Dissolution](https://arxiv.org/abs/2607.28041). arXiv:2607.28041. cs.CY. --- ## [A Posthumanist Approach to AI Literacy](https://edtechdev.github.io/aied/articles/posthumanist-ai-literacy-2025/) > **Synthesis:** Wang and Wang (2025) argue for a posthumanist reframing of AI literacy, moving beyond the humanistic view of AI as a discrete "tool" used by autonomous human agents toward understanding AI literacy as an understanding of how meaning emerges through the entanglement of human and AI agencies. Through a case study of two multilingual undergraduate students (Zhimo and Asuka) in US writing courses, they document a productive tension between students' experiments with posthumanist literacy and their entrenched humanistic assumptions, showing how posthumanism offers a relational approach to cultivating AI literacy in language and literacy education. ## Key Findings - The study contrasts two distinct approaches: Zhimo embodies a **critical-pragmatic humanistic AI literacy**, conceptualizing ChatGPT as a sophisticated "tool" (调教 / "tiao jiao", to train and manipulate) that he directs, while Asuka adopts an **anthropomorphic posthumanist stance**, treating AI as an agentic collaborator whose authorial agency is entangled with her own. - A central tension: Zhimo consistently asserts his human authorial agency while his writing process is nonetheless deeply entangled with AI — a posthumanist reading reveals that his "fix my language but not my meaning" approach still co-produces meaning through human-AI intra-actions. - Posthumanism de-centers humans as the sole autonomous meaning-making agents; agency is "enacted" through intra-actions between bodies, ideas, materials, language, technologies, and space (drawing on Barad, Latour, Bennett, Deleuze & Guattari's assemblage and rhizomaticity). - The framework repudiates both extremes: uncritical anthropomorphization of AI (which encourages overreliance and a transactional view of literacy) and dismissing AI as a mere tool (which leads to punitive measures that undermine students' exploratory meaning-making). - The article proposes that a posthumanist approach to AI literacy is essentially **relational rather than transactional work**, questioning celebratory corporate anthropomorphism and the anthropocentric discourse of human subjects leveraging AI "tools." ## Study Design & Method The study is a qualitative case study of two multilingual undergraduate students — Zhimo (a first-year student from China in an academic writing course) and Asuka (a Japanese student in an elective writing-intensive course on generative AI and writing) — at a private US college. Data were collected in Spring 2024 through 1.5–2 hour semi-structured interviews, students' AI-assisted writing artifacts, and guided reflections. A thematic analysis examined the students' AI-mediated literacy practices through both Wang and Wang's (2025) critical AI literacy model (awareness, positionality, human-AI interactions, evaluation of AI affordances) and posthumanist theory, triangulating data sources to understand how the students conceptualized, positioned themselves with, intra-acted with, and evaluated AI. ## Implications for AI in Education The article reframes [[ai-literacy]] in [[writing-education]] away from tool-competence toward relational understanding of human-AI entanglement. For educators, it suggests three practical applications: (1) encouraging students to critically interrogate AI-generated texts as co-constructed, contingent artifacts rather than static outputs; (2) recognizing AI literacy extends beyond textual engagement to multimodal, algorithmic, and linguistic dimensions of meaning-making; and (3) creating learning environments that allow both engagement with and ethical refusal of AI. The case studies show how distinct cultural-linguistic backgrounds and rhetorical objectives shape students' approaches to AI, and how posthumanism complements (rather than replaces) humanistic perspectives by deconstructing and decentering them. It connects to [[student-experience]], [[agency-gap-ai-writing|agency gap]], and debates about [[academic-integrity]] and authorship in the age of [[generative-ai]]. ## Limitations The study is a small qualitative case study of two students in one institutional context, bounding generalizability. The posthumanist theoretical framing is interpretive and does not offer measurable learning outcomes. The authors acknowledge a posthumanist approach is not an all-encompassing framework and must be situated within broader unresolved debates about AI's societal and ethical implications. The case-study design emphasizes depth of understanding over breadth. ## Connected Concepts - [[ai-literacy]] - [[writing-education]] - [[student-experience]] - [[generative-ai]] - [[language-learning]] - [[academic-integrity]] - [[metacognition]] ## Connected Articles - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[agency-gap-ai-writing]] — The Agency Gap in AI-Supported Writing - [[ai-writing-support-stage-ownership-2026]] — From Planning to Revision: AI Writing Support at Different Stages - [[student-rationalization-ai-writing]] — "It's OK Because...": The Wild West of Student Rationalization of AI Writing - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] — Metacognitive AI Literacy: Going Beyond the Skills Gap Agenda ## Citation Wang, Z., & Wang, C. (2025). [*A posthumanist approach to AI literacy*](https://doi.org/10.1016/j.compcom.2025.102933). --- ## [From Precision Medicine to Precision Education: A Vision for AI-Powered Student Digital Twins, Preventive Student Success, and Career-Aligned Academic Pathways](https://edtechdev.github.io/aied/articles/precision-education-student-digital-twins-2026/) > **Synthesis:** This paper proposes a precision education framework that adapts precision medicine's predictive, preventive approach to higher education. It envisions AI-powered student digital twins — computational models that integrate academic, behavioural, and career trajectory data to forecast risk, personalise interventions, and align course pathways with employment outcomes. The paper argues that traditional reactive models (responding after students fail or drop out) can be replaced with continuous risk stratification, early-warning nudges, and dynamic pathway optimisation. Key architectural components include federated data integration across institutional silos, explainable AI for advisor trust, and longitudinal models that evolve with the student. Higher education remains largely reactive in its approach to student success. Institutions frequently identify academic problems only after students have failed courses, fallen behind in degree progression, accumulated excessive debt, or departed without a credential. Healthcare faced a similar challenge decades ago. It responded by shifting from reactive treatment to preventive care powered by predictive models, risk stratification, electronic health records, and artificial intelligence (AI). This paper argues that higher education stands at an analogous inflection point. Drawing on advances in learning analytics, educational data mining, machine learning, workforce analytics, and digital twin technologies, we propose a paradigm we call Precision Education. Under this framework, AI contin proposes a precision education framework that adapts precision medicine's predictive, preventive approach to higher education. It envisions AI-powered student digital twins — computational models that integrate academic, behavioural, and career trajectory data to forecast risk, personalise interventions, and align course pathways with employment outcomes. The paper argues that traditional reactive models (responding after students fail or drop out) can be replaced with continuous risk stratification, early-warning nudges, and dynamic pathway optimisation. Key architectural components include federated data integration across institutional silos, explainable AI for advisor trust, and longitudinal models that evolve with the student. ## Connected Concepts - [[knowledge-tracing]] - [[affective-computing]] - [[student-modeling]] - [[adaptive-learning]] - [[reinforcement-learning]] - [[intelligent-tutoring]] - [[lifelong-learning]] - [[personalized-learning]] ## Connected Articles - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[kt4eqg-personalized-question-generation]] — KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing - [[cross-subject-validity-delayed-start]] — Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs ## Citation Dutta, Kaushik (2026). [From Precision Medicine to Precision Education: A Vision for AI-Powered Student Digital Twins, Preventive Student Success, and Career-Aligned Academic Pathways](https://arxiv.org/abs/2608.06322). arXiv:2608.06322. --- ## [Principled AI in Education](https://edtechdev.github.io/aied/articles/principled-ai-education/) > The framework rests on three interconnected anchors that must be addressed *before* selecting tools: > Rejecting the binary promise-vs-peril discourse and the rush to immediate implementation, Finkelstein (2025) proposes a principled framework anchored in three questions: What are our educational goals? What do we know about human learning? How can technologies serve those goals and models?^[[finkelstein-principled-ai-education-2025]] > **v2 update (Jul 2026).** A new preamble contextualizes the paper one year after initial sharing. Finkelstein argues that the critical question is not whether machines will match humans at every task, but *which skills, habits of mind, and practices must remain within the human purview to preserve our humanity and societies*. Three objectives for education stand out in the modern era: **discernment** (framing questions, validating and contextualizing answers, acting appropriately on results), **empathy** (understanding and sharing others' perspectives, the basis of communication and collaboration), and **sense of self** (understanding one's role, purpose, and belonging within a course, field, or culture). The original arguments and principles of action, he notes, remain relevant one year later. ## The Three Anchors The framework rests on three interconnected anchors that must be addressed *before* selecting tools: 1. **Goals** — What should education accomplish? Meaningful learning, democratic participation, preparation for dynamic futures. 2. **Models of human learning** — How do people actually learn? Drawing on decades of learning sciences scholarship: active construction, social mediation, transfer, metacognition, motivation. 3. **Use of technologies** — How can AI serve the goals and models, rather than dictating them? ## Roles Clarified The framework defines distinct responsibilities across four domains: | Domain | Educators | Learners | Technologies | |---|---|---|---| | **Shaping curricula** | Define learning goals, sequence concepts, assess alignment | Express interests, co-design pathways | Recommend resources, flag gaps | | **Designing instruction** | Craft activities, scaffold progressively, adjust in real time | Engage actively, seek help strategically | Generate variations, personalize pacing | | **Assessing learning** | Design authentic tasks, interpret patterns, give feedback | Self-assess, reflect, revise | Score at scale, track patterns, suggest interventions | | **Cultivating community** | Build norms, facilitate dialogue, mediate conflict | Collaborate, support peers, participate | Connect learners, moderate asynchronously | ## Core Mandate: Augment, Don't Displace > "AI must augment, not displace, human capacities." This principle is the through-line of the framework. Technology use must be aligned with enduring educational values: - Advancing **meaningful learning** (not just efficient content delivery) - Supporting **democratic societies** (not just individual skill acquisition) - Preparing students for **dynamic futures** (not just today's job market) ## Relationship to Existing Research | Finkelstein principle | Supporting wiki evidence | Tension | |---|---|---| | Goals before tools | [[stanford-evidence-base-ai-k12-2026]] (few causal studies because goals were ill-defined) | Industry pressure to deploy fast | | Augment, not displace | [[ai-tutor-safety-harms]] (displacement harms catalogued) | [[llm-fallacy-misattribution]] (users willingly displace themselves) | | Meaningful learning | [[self-regulated-learning]] (SRL as meaningful engagement) | Convenience tempts toward shortcut design | | Democratic societies | [[ai-literacy]] (distributed AI literacy for citizenship) | Platform concentration concentrates power | | Assess authentically | [[authentic-assessment]] (six-dimensional framework) | Automated grading scales cheaply | ## Using the Framework The framework can serve as a diagnostic for any AI-in-education initiative: 1. **State the goal clearly** — Not "use AI in my course" but "improve students' ability to evaluate evidence" 2. **Identify the learning model** — What research on learning supports this goal? (e.g., productive failure, deliberate practice, social learning) 3. **Match technology to model** — Which AI affordance serves this model? (e.g., Socratic dialogue for productive failure, peer matching for social learning) 4. **Evaluate against displacement** — Does the tool do something the learner should do themselves? If yes, redesign. ## Connected Concepts - [[ai-literacy]] - [[faculty-development]] - [[human-in-the-loop-ai]] - [[metacognition]] - [[self-regulated-learning]] - [[ai-education]] - [[higher-ed]] - [[scaffolding]] ## Connected Articles - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-workflows-education]] — Agentic Workflows in Education - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-higher-ed-bridge-gap]] — Higher Education Must Bridge the AI Gap - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[transfer-of-learning]] — AI Learning Transfer - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[aied-carbon-footprint-reporting]] — The Environmental Cost of LLMs in AIED: Reporting and Practices - [[authentic-assessment]] — Authentic Assessment - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI - [[educational-llm-alignment]] — Educational LLM Alignment - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[institutional-change-framework-ai]] — A Framework for Institutional Change in the Age of AI - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[multi-agent-instructional-design]] — Multi-Agent Systems for Instructional Design - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[teacher-control-ai-generation-math-visuals]] — When Should Teachers Control AI Generation for Mathematics Visuals? - [[universities-ai-era-rethinking]] — The University AI Didn't Replace: Rethinking Universities in the AI Era - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning ## Citation Finkelstein, N. (2025). [*A principled way to think about AI in education: guidance for educators and policy makers on action based on goals, models of human learning, and use of technologies*](https://arxiv.org/abs/2510.01467). arXiv:2510.01467. --- ## [Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition](https://edtechdev.github.io/aied/articles/privacy-aware-classroom-incident-recognition-2026/) > **Synthesis:** Pilot study on privacy-aware computer vision for classroom incident detection. Introduces a hybrid benchmark combining generative CCTV-style videos with real classroom pose data. Proposes a lightweight motion reasoning model that achieves strong incident recognition while preserving student privacy (no facial recognition). Demonstrates that efficient motion-based features can generalize across classroom environments without collecting identifiable student data. [[privacy]], [[k-12]], [[multimodal]], [[edtech-platform]], and benchmark. Pilot study on privacy-aware computer vision for classroom incident detection. Introduces a hybrid benchmark combining generative CCTV-style videos with real classroom pose data. Proposes a lightweight motion reasoning model that achieves strong incident recognition while preserving student privacy (no facial recognition). Demonstrates that efficient motion-based features can generalize across classroom environments without collecting identifiable student data. ## Connected Concepts - [[privacy]] - [[k-12]] - [[multimodal]] - [[edtech-platform]] ## Connected Articles - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[llm-unlearning-math-privacy]] — Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education - [[turtleai-visual-programming-benchmark]] — TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[antiskillbench-persona-skills-privacy-2026]] — When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills ## Citation Paritosh Parmar, Landy Lan, Hong Yang, Chen Yi, & Chiat Pin Tay (2026). [Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition](https://arxiv.org/abs/2608.05115). arXiv:2608.05115. arXiv preprint (cross-listed cs.CV/cs.HC). --- ## [Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing](https://edtechdev.github.io/aied/articles/prober-ai-inquiry-writing/) > A web-based writing environment that inverts the AI-tutoring paradigm: rather than generating improved text for students, Prober.ai constrains an LLM to ask only targeted inquiry-based questions about argumentative weaknesses. Students must reflect before receiving revision suggestions. Developed by Bi et al. (2026), awarded second place at NY EdTech Hackathon. # Prober.ai: Gated Inquiry Writing Feedback ## Definition A web-based writing environment that inverts the AI-tutoring paradigm: rather than generating improved text for students, Prober.ai constrains an LLM to ask only targeted inquiry-based questions about argumentative weaknesses. Students must reflect before receiving revision suggestions. Developed by Bi et al. (2026), awarded second place at NY EdTech Hackathon. ## Core Innovation: Pedagogical Friction The system implements a **Challenge → Unlock** architecture: 1. **Challenge Phase:** AI delivers inquiry-based questions targeting specific argumentative weaknesses (e.g., "What evidence would convince a skeptic of this claim?") 2. **Unlock Phase:** Only after the student responds to those questions does the system reveal concrete revision suggestions This gating mechanism deliberately creates friction — students cannot bypass critical thinking to access help. The approach is grounded in Toulmin's argumentation theory and research on peer feedforward questioning. ## Why This Matters Conventional AI writing tools that generate or rewrite text risk creating cognitive debt — students outsource thinking rather than developing it. Prober.ai's approach connects to [[metacognition]] by forcing reflective engagement and to [[socratic-ai-dialogue]] through inquiry-based interaction. It represents a design pattern for "cognition-preserving AI" that could extend beyond writing to other domains. ## Connected Concepts - [[metacognition]] - [[socratic-ai-dialogue]] ## Connected Articles - [[pedagogy-ai-mistakes]] — The Pedagogy of AI Mistakes: Fostering Higher-Order Thinking - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[moodle-ai-tutoring-deep-learning]] — From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle - [[guided-llm-scaffolding-independent-learning]] — Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics - [[constructing-epistemic-ai-literacy-student-ai-co-programming]] — Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming ## Citation Zhou, A.R.B.S.W.Y. & place), V.N.E.H. (2026). [*Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing*](https://arxiv.org/abs/2605.05598) --- ## [Programming Intelligent Tutoring Systems](https://edtechdev.github.io/aied/articles/programming-its/) > **SCRIPT** (Deriyeva, Dannath, Paassen, 2026) implements an intelligent tutoring system for **Python programming** in a German university context, filling a gap in prior ITS which rarely supported Python. > ITS for programming education with individualized hints, addressing the scarcity of human tutors for practice-intensive coding courses. ## SCRIPT: Python Programming Tutor **SCRIPT** (Deriyeva, Dannath, Paassen, 2026) implements an intelligent tutoring system for **Python programming** in a German university context, filling a gap in prior ITS which rarely supported Python. ### System Design - **Context:** Large undergraduate programming courses where individual tutor feedback doesn't scale - **Language:** Python (vs. prior ITS focused on Java, C++) - **Pedagogy:** Individualized hints and advice during coding exercises - **Deployment:** Real classroom integration (not just lab prototype) ### Key Features | Feature | Description | |---------|-------------| | **Hint generation** | Context-aware hints based on code state and error type | | **Exercise adaptation** | Difficulty adjustment based on learner performance | | **Scaffolding** | Graduated support (from syntax to algorithm design) | ## Connection to Broader Programming Education SCRIPT addresses a core challenge: **practice and extensive exercises are essential in programming education**, but human tutors cannot scale to large cohorts. This connects to: - **[[collaborative-ai-tutoring]]** — ProPACT for pair programming (collaborative variant) - **[[formative-assessment]]** — AI-generated coding exercises with human-in-the-loop validation - **[[agentic-workflows-education]]** — Agentic approaches to coding education (e.g., Claude Code tutor) ## Research Gap Addressed Prior ITS for programming focused on: - Java (most common in early CS education research) - C/C++ (systems programming contexts) - Block-based languages (K-12) SCRIPT's contribution: **Python-specific** tutoring with German-language context (university-level). Python's dynamic typing and REPL-based workflow require different hint strategies than statically-typed languages. ## Implications - **Language-specific scaffolding:** ITS must adapt to language idioms (Pythonic vs. Java-esque solutions) - **Classroom integration:** Real deployment reveals usability barriers invisible in lab prototypes - **Scalability:** ITS enables personalized feedback without proportional instructor hiring ## Connected Concepts - [[formative-assessment]] ## Connected Articles - [[collaborative-ai-tutoring]] - [[agentic-workflows-education]] ## Citation Paassen, A.A.D.J.D.B. (2026). [*Programming Intelligent Tutoring Systems*](https://arxiv.org/abs/2604.16117) --- ## [Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers](https://edtechdev.github.io/aied/articles/prompt-based-programming-lesson/) This study by Tran, Marwan & Price (2026) introduces and evaluates a 45-minute structured lesson on prompt-based programming, a new modality enabled by LLMs where users express computational goals through natural language. The lesson design incorporates guided practice principles and targets end-user programmers with limited formal training. Results show significant pre-to-post gains in prompt quality, code correctness, and self-efficacy, supporting the case that [[ai-literacy]] interventions need not be lengthy to be effective. The work connects to broader conversations about how [[llm]] tools change the skills required for programming — shifting emphasis from syntax to prompt engineering. By focusing on end-user programmers rather than CS students, the study expands the scope of [[stem-education]] research to include lifelong and professional learning contexts. It also raises questions about the [[teacher-role]] in an AI-mediated classroom, where instructors must now teach prompt design alongside or instead of traditional coding concepts. Future work could explore how such lessons scale across [[higher-ed]] institutions and whether the gains translate to real-world programming tasks. ## Connected Concepts - [[ai-literacy]] - [[llm]] - [[stem-education]] - [[teacher-role]] - [[higher-ed]] ## Connected Articles - [[bridging-instructional-design-framework-math]] - [[llm-misconception-difficulty-easy-trap]] - [[bloom-aligned-educational-control-llms]] - [[youtube-frames-chatgpt-education]] - [[anvil-ai-educational-animations]] - [[prompt-problems-nl-programming-mistakes]] ## Citation Keith Tran, Samiha Marwan, Thomas Price (2026). [Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers](https://arxiv.org/abs/2606.30547). arXiv:2606.30547. cs.CY. - [[voice-text-prompt-problems-computing-education]] — Modality choice in prompt construction --- ## [Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development](https://edtechdev.github.io/aied/articles/prompt-coach-agentic-tutor-prompt-engineering/) Prompt engineering is a critical yet undertaught skill for software developers, poorly served by traditional instruction because of its evolving, interactive, context-dependent nature. The authors introduce **Prompt Coach (PC)**, an agentic tutor embedded in-flow within a developer's IDE that teaches prompt crafting through Socratic guidance. PC scores prompt quality across multiple dimensions and surfaces targeted questions that guide self-correction, grounded in the developer's codebase and the target LLM's behavior. An early empirical study with 15 professional developers showed statistically significant improvements after a single 60-minute session, with the largest gains on dimensions developers commonly overlook, plus strong trust and adoption readiness. - **In-flow Socratic tutoring:** PC delivers [[scaffolding]] and [[feedback-loop]] support directly inside the IDE, contextualized to the developer's code and the target model. - **Empirical gains:** Statistically significant prompt-quality improvement after one 60-minute session, aligning with findings in [[prompt-based-programming-lesson]] about prompt fluency as a learnable skill. - **Self-correction emphasis:** Targeted Socratic questions steer developers to repair their own prompts rather than receive answers, consistent with [[formative-assessment]] practice. - **Professional context:** Extends [[cs-education]] and [[professional-training]] beyond novices to working developers. - **Trust and adoption:** Participants reported high trust and unanimous belief PC improved their skills, contributing to [[ai-literacy]] for practitioners. ## Connected Concepts - [[scaffolding]] - [[feedback-loop]] - [[formative-assessment]] - [[cs-education]] - [[professional-training]] - [[ai-literacy]] ## Connected Articles - [[prompt-based-programming-lesson]] ## Citation Mehra, R., Singi, K., Kaulgud, V., Sharma, V. S., & Choudhury, S. G. (2026). [Prompt Coach: An Empirical Evaluation of an Agentic Tutor for Learning Prompt Engineering in Software Development](https://arxiv.org/abs/2607.06074). arXiv:2607.06074. --- ## [Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs](https://edtechdev.github.io/aied/articles/prompt-injection-defenses-educational-llm-tutors/) > Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs **Maiorano (2026)** — arXiv cs.CR/cs.AI. Evaluating Prompt Injection Defenses for Educational LLM Tutors: Security-Usability-Latency Trade-offs ## Summary This paper addresses a fundamental tension in educational LLM deployment: tutors must be both **secure** (resist prompt injection attacks) and **usable** (not block legitimate educational interactions). The author presents a systematic evaluation methodology using a **480-query benchmark** (369 injection, 111 benign) with statistically rigorous comparison. **Defense methods compared:** | Method | Bypass Rate | False Positive Rate | Latency | |--------|-------------|---------------------|---------| | Proposed Multi-Layer Pipeline | 46.34% | **0.00%** | **2.50ms** | | Prompt Guard (Meta) | 38.48% | 3.60% | — | | NeMo Guardrails (NVIDIA) | **0.0%** | 16.22% | 1.3s | **The proposed pipeline** combines deterministic pattern filters, structural validation, contextual sandboxing, and session-level behavioral checks. Its design prioritizes **pedagogical usability** — zero false positives means no legitimate student queries get blocked, an essential requirement for [[intelligent-tutoring]] systems where interruptions harm learning. **NeMo Guardrails** blocks all attacks but incorrectly flags ~16% of benign requests — a rate that would seriously degrade the [[student-experience]] in real tutoring sessions. **Prompt Guard** provides middle-ground performance. The framework enables **evidence-based guardrail selection** under institutional risk and usability requirements. This directly connects to [[ai-tutor-safety-harms]] concerns and the emerging field of [[pedagogical-safety-rl]]. The latency dimension is particularly important for real-time [[conversational-ai-tutors-framework]] where response delays degrade engagement. The paper highlights that **educational settings have unique requirements**: false positives are more costly than in general-purpose chatbots, because blocking a student's learning interaction carries pedagogical harm. This aligns with findings in [[eduframetrap-llm-sycophancy-educational-safety]] that educational safety requires domain-specific benchmarks. ## Connected Concepts - [[intelligent-tutoring]] - [[student-experience]] ## Connected Articles - [[ai-tutor-safety-harms]] - [[pedagogical-safety-rl]] - [[conversational-ai-tutors-framework]] - [[eduframetrap-llm-sycophancy-educational-safety]] ## Citation Maiorano, A. C. (2026). [Evaluating prompt injection defenses for educational LLM tutors: Security-usability-latency trade-offs](https://arxiv.org/abs/2605.06669). arXiv:2605.06669. --- ## [Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks](https://edtechdev.github.io/aied/articles/prompt-problems-nl-programming-mistakes/) Learning to communicate with code-generating AI is an emerging skill for novice programmers. 'Prompt Problems' — having students solve computational tasks by writing natural-language prompts for code-generating models — is a recent pedagogical approach, yet little was known about the specific prompt-level mistakes novices make, the computational details they fail to communicate, and how they recover when generated code is wrong. Padurean et al. (2026) studied attempts by more than 900 students to solve dialogue-based Prompt Problems in a CS1 course, analyzing the misconceptions and repair strategies that surface when learners must specify intent in English rather than code. The study extends the [[prompt-based-programming-lesson]] lineage and the broader [[reshaping-cs-education-genai]] movement, situating prompt-writing as a core [[ai-literacy]] competency within [[cs-education]]. It also connects to [[programming-its]] (where natural-language specification has long been a goal) and highlights the need for [[scaffolding]] that helps novices articulate computational detail. Findings on [[student-experience]] and recovery behavior inform [[higher-ed]] course design as LLM pair-programming becomes routine. ## Connected Concepts - [[ai-literacy]] - [[cs-education]] - [[scaffolding]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[prompt-based-programming-lesson]] - [[reshaping-cs-education-genai]] - [[programming-its]] ## Citation Victor-Alexandru Padurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake, Paul Denny, Adish Singla (2026). [Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks](https://arxiv.org/abs/2607.05034). arXiv:2607.05034. arXiv:2607.05034 [cs.CY]. - [[voice-text-prompt-problems-computing-education]] — Text vs voice modality on Prompt Problems --- ## [ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs](https://edtechdev.github.io/aied/articles/proprl-prerequisite-relation-learning/) > **Synthesis:** ProPRL advances [[adaptive-learning|prerequisite relation learning]] by going beyond conventional link prediction to adaptively integrate complementary educational evidence from concept-resource hypergraphs and directed learning-behavior graphs. The Irreversibility Constraint — an anti-symmetry regularizer that penalizes contradictory bidirectional predictions — addresses a fundamental issue in [[knowledge-tracing|educational knowledge graphs]]: the asymmetry of prerequisite relationships. Experiments on multiple real-world educational datasets demonstrate state-of-the-art performance. ## Framework Components ProPRL introduces three core innovations: **1. Complementary Concept Representations:** - Learns from a concept-resource hypergraph (which resources teach which concepts) - Simultaneously learns from a directed learning-behavior graph (how students traverse concepts) - Direction-preserving personalized propagation aggregates multi-hop behavioral evidence **2. Pair-Conditioned Gate:** - Adaptively weights and fuses the two representation views for each candidate ordered concept pair - Different pairs may benefit from different evidence sources — the gate learns this balance **3. Irreversibility Constraint:** - Anti-symmetry regularizer that penalizes high confidence in both directions of a concept pair - Enforces the fundamental property that prerequisites are directional (A → B, not B → A) - Addresses a limitation of prior methods that treated prerequisite learning as symmetric link prediction ## Key Results - **State-of-the-art performance** on prerequisite relation learning across multiple real-world educational datasets - The Irreversibility Constraint significantly reduces contradictory bidirectional predictions - Pair-conditioned fusion outperforms uniform weighting of evidence sources - Multi-hop behavioral propagation captures richer learning trajectories than direct co-occurrence ## Implications for Adaptive Learning Accurate prerequisite modeling is foundational to: - **[[personalized-learning]]:** Sequencing content appropriately for each learner - **[[knowledge-tracing]]:** Understanding which concepts a student is ready to learn - **[[student-modeling]]:** Building accurate representations of student knowledge states - **Curriculum design:** Identifying optimal learning pathways through complex knowledge domains ProPRL's property-aware approach ensures that these systems respect the asymmetric nature of learning dependencies. ## Connected Concepts - [[adaptive-learning]] - [[knowledge-tracing]] - [[personalized-learning]] - [[student-modeling]] ## Connected Articles - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[multimodal-knowledge-graph-educational-reasoning]] — Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components ## Citation Cheng, X., Wang, J., He, C., Dong, R., & Guan, Q. (2026). [*ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs*](https://arxiv.org/abs/2608.03006v1). arXiv:2608.03006v1. --- ## [PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback](https://edtechdev.github.io/aied/articles/psyscore-essay-scoring-zpd-feedback/) > **Wei Xia, Jin Wu, Haoran Shi, Xiangyu Wang, Chanjin Zheng** (2026). East China Normal University / arXiv cs.CL preprint **Wei Xia, Jin Wu, Haoran Shi, Xiangyu Wang, Chanjin Zheng** (2026). East China Normal University / arXiv cs.CL preprint ## Overview PsyScore integrates diagnostic assessment with instructional scaffolding through a shared latent ability representation. Three modules: (1) Trait-Adaptive Neural IRT Scorer incorporating GPCM for precise ability estimation (QWK 0.747, besting prior SOTA 0.722); (2) ZPD-Scaffolded Feedback Generator conditioning multi-agent feedback on diagnosed ability; (3) Multi-Perspective Feedback Evaluation via pairwise preferences and simulated revisions. Demonstrates that psychometrically-aware AES can transform from summative scoring into formative diagnosis. ## Key Contributions ### Paper 1: Code as Anchor, Memory and Metaphor as Support: Learner Experiences with Multi-View Visualizations - **Problem:** Students often ignore well-designed program visualizations; existing cognitive design principles don't explain learner engagement/disengagement. - **Method:** Within-subjects study (N=19 undergraduates, post-CS1/CS2) using think-aloud, interviews, and webcam gaze tracking with a multi-representational probe. - **Gaze finding:** ~47% of time on code despite visual scaffolds; students without prior experience anchored more in code and ignored metaphor views. - **Three engagement themes:** Agency (control over cognitive effort), Representational Fit (wide individual variation), Legitimacy (metaphors seen as childish at university level). - **Implication:** multi-representational-tools need attention to affective and social factors, not just cognitive design. ### Paper 2: Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring - **Problem:** Static-prompt LLM tutoring systems fail to adapt across diverse academic disciplines. - **Solution:** Subject-aware prompt routing using 14 pedagogical features extracted from transcripts; contextual bandit formulation with 20 pedagogical prompts. - **Simulation:** Router achieves 0.694 vs 0.647/0.64 static baselines (p<0.001). - **Real-world A/B test:** N=656 conversations, 359 Dutch high-school students. Stochastic router achieves 28.1% exercise conversion rate vs 19.6% baseline. - **Implication:** [[adaptive-prompt-routing]] with stochastic sampling improves both efficiency and engagement in real-world tutoring. ### Paper 3: Confidence-Aware Automated Assessment of Student-Drawn Scientific Models - **Problem:** Automated scoring of student-drawn scientific models lacks reliability indicators, leaving teachers unable to decide when to trust scores. - **Method:** Vision Transformer (ViT) with LoRA + confidence-aware framework using test-time perturbations. - **Dataset:** Six NGSS-aligned middle school assessment items (477-816 responses each, scored Beginning/Developing/Proficient). - **Key innovation:** Response-level confidence enables selective automation — high-confidence auto-scored, uncertain cases deferred for human review. - **Implication:** [[confidence-aware-ai-assessment]] enables practical triage between automation and human oversight in educational assessment. ### Paper 4: PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback - **Problem:** AES systems treat scoring and feedback as separate; neural scoring lacks interpretability; LLM feedback is ability-agnostic. - **Solution:** Unified psychometric latent space (θ) via Neural GPCM Trait-Adaptive Scorer. - **Scoring performance:** QWK 0.747 (besting prior SOTA 0.722); 1st in 6/8 ASAP++ prompts and 10/11 trait dimensions. - **Feedback quality:** ZPD-based strategy mapping (Explicit Correction for θ<-1, Scaffolding for -1<θ<1, Socratic for θ>1). Multi-agent fusion from Llama-4-Scout, Qwen3-235B, GPT-4o. - **Evaluation:** >90% win rate in Actionability vs GPT-4o, Llama-4-Scout, Qwen3. - **Implication:** psychometrically-aware-ai can unify assessment and instruction, transforming automated essay scoring from summative to formative. ## Connected Concepts - [[adaptive-prompt-routing]] - [[confidence-aware-ai-assessment]] ## Connected Articles - [[icle-plus-plus-essay-scoring]] ## Citation Wei Xia, Jin Wu, Haoran Shi, Xiangyu Wang, Chanjin Zheng (2026). [PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback](https://arxiv.org/abs/2606.20287). arXiv:2606.20287. East China Normal University / arXiv cs.CL preprint. --- ## [Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis](https://edtechdev.github.io/aied/articles/q-learning-lab-rl-teaching/) > Presents Q-Learning Lab, a single-file tool that makes the Bellman update concrete by letting undergraduates inspect how each value is computed and why actions are chosen, through learner-generated trace analysis. It addresses the abstraction gap where students watch policy convergence without understanding mechanism. A [[reinforcement-learning]] artifact grounded in [[active-learning]] and [[stem-education]] for [[higher-ed]]. It connects to [[self-regulated-learning]] by making the learning process observable, and to [[scaffolding]] of difficult quantitative concepts, offering a reproducible teaching tool for ML/RL courses. ## Key Findings - Q-Learning Lab is a single-file, browser-based, bilingual (Thai/English) tool for teaching tabular Q-learning that requires no installation and no network connection. - Beyond the usual gridworld visualization — color-coded Q-values and policy arrows on a 5×5 world — a live Bellman-substitution panel shows the numeric update at every step. - Each transition is logged, including the full pre-action Q-row, the greedy-versus-random decision under ε-greedy exploration, and wall-collision events, into an exportable trace. - The central contribution is a learn–export–analyze loop: learners run their own agent, export the complete trace as CSV, and analyze it themselves, producing learning curves, value heatmaps, and visitation maps that turn a passive demonstration into learner-generated data for reflective inquiry. - A reward-editing study separates two behaviorally identical but diagnostically opposite failure modes — an exploration failure versus genuine reward misspecification — that a single edited reward can produce. ## Study Design & Method The tool was validated without human-subject data through three complementary evaluations: (i) correctness of the learned values and policy against a value-iteration ground truth on the identical MDP; (ii) hyperparameter sweeps over α, γ, and ε demonstrating that every pedagogical claim the tool makes is reproducible; and (iii) a reward-editing study in which editing the pit reward produced distinct failure modes: with a pit reward of +5 the optimal policy still reaches the goal (V*(s0)=6.38), yet Q-learning converges to the pit in 10/10 seeds — an exploration failure — whereas with a pit reward of +20 (above the 7.774 threshold) the optimal policy genuinely prefers the pit (V*(s0)=16.86) and Q-learning agrees in 10/10 seeds — true reward misspecification. The paper also situates the tool against existing gridworld visualizers via a feature comparison, grounds it in learning-by-doing pedagogy, and includes a 50-minute lesson plan; the tool and all experiment code are openly available. ## Implications for AI in Education Reinforcement learning is usually introduced through the Bellman update, yet the equation often remains abstract to undergraduates: they watch policy arrows converge but rarely observe how each value is computed or why an action is chosen. By making the numeric update and every decision visible and exportable, Q-Learning Lab supports [[self-regulated-learning]] and reflective inquiry in quantitative courses, and its learn–export–analyze loop gives instructors a reproducible way to diagnose common student confusions — such as conflating exploration failure with reward misspecification — that surface only when learners analyze their own agent's traces. ## Connected Concepts - [[reinforcement-learning]] - [[active-learning]] - [[stem-education]] - [[higher-ed]] - [[self-regulated-learning]] - [[scaffolding]] ## Connected Articles - [[codify-socratic-programming-tutor]] — Codify: An Intelligent Socratic Tutoring System for Programming Education - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[llm-automated-assessment-student-self-explanations]] — Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education - [[forap-pjbl-computing-education]] — Adoption-Ready Project-Based Learning for Computing Education: The FORAP Framework and a Multi-Scale Project Portfolio - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning ## Citation Ekkachai Jueng (2026). [Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis](https://arxiv.org/abs/2607.10802). arXiv:2607.10802. arXiv preprint. --- ## [Quantum Education Intelligent Tutoring](https://edtechdev.github.io/aied/articles/quantum-education-its/) > **From Prototype to Classroom** (Elhaimeur & Chrisochoides, 2026) describes a tutoring system for quantum computing that bridges the gap between dense mathematical formalism and limited qualified instructors. > Knowledge-graph-augmented ITS for quantum computing education, addressing instructor scarcity and concept counterintuitiveness. ## System Architecture **From Prototype to Classroom** (Elhaimeur & Chrisochoides, 2026) describes a tutoring system for quantum computing that bridges the gap between dense mathematical formalism and limited qualified instructors. ### Knowledge Graph Foundation - **Structure:** Concept nodes with prerequisite relationships mapped explicitly - **Pedagogical use:** Tutor traverses graph to select next topic based on learner state - **Advantage:** Explicit prerequisite mapping aids navigation of counterintuitive quantum concepts (superposition, entanglement, measurement) ### Adaptive Components | Component | Function | |-----------|----------| | **Learner Model** | Tracks mastery per concept node in knowledge graph | | **Pedagogical Module** | Selects optimal next concept/scaffold based on zone of proximal development | | **Interface** | Visualizes quantum states (Bloch spheres, circuit diagrams) with stepwise guidance | ## Key Challenges Addressed 1. **Concept counterintuitiveness:** Quantum mechanics violates classical intuition — requires specialized scaffolding beyond generic ITS 2. **Mathematical density:** Formalism (Dirac notation, unitary evolution) creates barrier for beginners 3. **Instructor scarcity:** Few qualified faculty outside well-resourced institutions The system's knowledge-graph approach allows *structured progression* through these barriers rather than open-ended dialogue (which can confuse novices in quantum topics). ## Connection to Broader ITS Trends Unlike general-purpose ITS (e.g., [[adaptive-learning|adaptive systems]] for math or programming), quantum education requires: - **Domain-specific visualizations** (quantum circuits, state spaces) - **Specialized misconception handling** (classical intuition interference) - **Formalism scaffolding** (gradual introduction of mathematical tools) This aligns with the [[tutoring-specific-vs-general-ai|tutoring-specific design]] principle: domain adaptation matters more than general conversational ability. ## Implications for AI in Education - **Niche STEM domains:** Knowledge-graph augmentation enables ITS deployment in specialized fields with scarce human expertise - **Visualization integration:** Quantum tutoring shows the importance of domain-aligned visual scaffolds (cf. [[multimodal-ai-tutoring]] which also emphasizes multimodal errors in STEM) - **Scalability:** Addresses equity gaps between well-resourced and under-resourced institutions ## Connected Concepts - [[adaptive-learning]] ## Connected Articles - [[tutoring-specific-vs-general-ai]] - [[multimodal-ai-tutoring]] ## Citation Chrisochoides, A.I.E.N. (2026). [*Quantum Education Intelligent Tutoring*](https://arxiv.org/abs/2604.24807) --- ## [Effects of an AI-supported inquiry model on AI literacy and authentic performance: A quasi-experimental study with preservice teachers](https://edtechdev.github.io/aied/articles/quest-ai-inquiry-preservice-teachers/) > **Synthesis:** Effects of an AI-supported inquiry model on AI literacy and authentic performance: A quasi-experimental study with preservice teachers ## Key Findings - A 10-week quasi-experimental study with 95 preservice teachers in an educational research methods course compared two intact classes (experimental n = 52; comparison n = 43). The groups were equivalent at baseline, with no statistically significant pre-intervention differences in age (M = 20.38 vs 20.05), gender (12:40 vs 5:38 male:female), grade level, major, or AI-literacy pretest scores (M = 79.46 vs 80.33). - The QUEST+AI model structures AI-supported inquiry in five phases: Question, Understand, Engage, Solve, and Teach. - The experimental group showed higher overall [[ai-literacy]], with small but meaningful gains concentrated in applying AI, AI-supported problem solving, and emotion regulation during AI use (partial η² ≈ .05–.09). Within-group gains ran from M = 79.46 (SD = 12.22) at pretest to M = 84.94 (SD = 8.27) at posttest (t(51) = 3.66, p < .001), whereas the comparison group showed no statistically reliable change. - No clear group differences were found for more concept-focused or evaluative dimensions of AI literacy (concepts, detection, ethics, creation, and persuasion). - The experimental group earned higher scores on the final research proposal (partial η² = .137), indicating a moderate advantage in authentic performance: after covariate adjustment, adjusted means were 84.14 vs 80.72 (F(1, 90) = 14.24, p < .001), a difference of 3.42 points (95% CI [1.62, 5.22]). ## Study Design & Method Grounded in Deweyan Inquiry and the Practical Inquiry model, the study examined the effects of QUEST+AI, an AI-supported inquiry model built around five phases: Question, Understand, Engage, Solve, and Teach. Both groups received the same in-class instruction, but the experimental group completed two QUEST+AI cycles with coached generative AI use, whereas the comparison group completed conventional homework. Outcomes included a multidimensional AI literacy measure and a capstone research proposal scored with a common rubric. Within-group change was assessed with paired-samples t-tests, and group differences with ANCOVA controlling for pretest AI literacy, gender, and grade. The coached cycles embedded prompt logs and verification routines that made AI-supported decisions visible, and the same inquiry routines mapped directly onto the rubric dimensions used to score the proposals (problem framing, synthesis quality, methodological coherence, and argumentation). ## Implications for AI in Education The findings suggest that structured AI-supported inquiry can strengthen applied and self-regulatory aspects of AI literacy while improving discipline-relevant performance — evidence that coached GenAI use embedded in authentic academic work outperforms conventional homework for developing practical AI competence. The moderate effect on research proposal quality indicates that AI-supported inquiry does not trade away disciplinary learning for tool proficiency. Notably, these gains were achieved without expanding lecture time: the experimental group received no additional AI-literacy-focused direct instruction, which the authors suggest explains the absence of change on concept-heavy and evaluative scales. For [[teacher-ai-competency]] and [[faculty-development]] efforts, the model offers a concrete, phase-structured template for integrating GenAI into coursework, with brief targeted activities (concept refreshers with retrieval checks, verify-and-trace exercises, rubric-scored ethics cases) recommended to address the dimensions the model alone did not shift. The pattern of results marks this as an [[efficacy-study]] whose design logic favors pedagogical specificity over mere tool availability. ## Limitations The authors acknowledge several limitations: nonrandom group assignment within a single institution, brief exposure, a modest sample size, several short subscales with modest reliability, self-reported literacy outcomes, and rubric-based scoring for performance. These features increase ecological validity but limit causal inference and may reduce sensitivity to change; stronger controls and objective measures — such as randomized or crossover designs, longer interventions with follow-up measures, and performance-based assessments triangulated with protocol-adherence data — would improve robustness. ## Connected Concepts - [[teacher-ai-competency]] - [[ai-literacy]] - [[efficacy-study]] - [[faculty-development]] - [[professional-training]] - [[k-12-ai-education]] - [[instructional-design]] - [[higher-ed]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[amponsah-ai-acceptance-science-teachers-2026]] — Perceptions And Acceptance of Artificial Intelligence in Science Education Programmes: Voices of Pre-Service Science Teachers - [[genai-skill-bypass-literacy]] — The GenAI Skill Bypass: Mapping Divergent Pathways of University Students and Staff AI Literacy - [[cognitive-shift-ai-education]] — Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence? - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study - [[teacher-education-ai-literacy-sdt-2026]] — Teacher education for artificial intelligence literacy through a self-determination theory perspective ## Citation Cao, D., Yan, Y., Xiong, A., & Wicks, D. (2026). [*Effects of an AI-supported inquiry model on AI literacy and authentic performance: A quasi-experimental study with preservice teachers*](https://doi.org/10.1016/j.caeai.2026.100630). --- ## [Rethinking Generative AI Literacy: An Integrative, Developmental, and Dialectical Framework for K-12 Teacher Education](https://edtechdev.github.io/aied/articles/rail-ed-genai-literacy-teacher-education/) > **RAIL-Ed is an integrative, developmental, and dialectical framework for generative AI literacy in K-12 teacher education, built from a systematic review of 67 studies and specifying six interdependent pillars with a three-level maturity rubric.** **Shahin Hossain, Sima Ahmadi, Leqi Li, Idowu David Awoyemi, Wei Huang, Chenxi Zhou, Jujia Li, Samaa Haniya, Shapla Khanam, Tasbirun Mashreka Subaha** — arXiv (cs.CY / cs.HC) preprint, 2026 (UMBC, Kent State, Penn State, University of Alabama, etc.). ## Synthesis Developed through a systematic review and qualitative framework analysis of 67 studies (2023-2025), grounded in critical, pragmatist, sociocultural, and human-centered traditions (Freire, Dewey, Vygotsky, Shneiderman). Specifies six interdependent pillars: Technical Fluency, Critical Evaluation, Human-AI Collaboration, Contextual Awareness, Ethical Reasoning, and Empowered Agency — the absence of any pillar produces a characteristic pedagogical failure. Developmental: a three-level rubric (Emerging, Competent, Advanced) describes how each pillar matures across the K-12 teacher-preparation continuum. Dialectical: the same generative affordance can deepen or displace learning depending on teacher literacy, making literacy cultivation — not tool adoption — the object of design. Aligned with the UNESCO AI Competency Framework for Teachers and the OECD/European Commission AILit Framework; advances falsifiable propositions for empirical validation. ## Connected Concepts - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[k-12-ai-education]] - [[ai-education]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[curriculum-design]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[feedback-futures-genai]] — Feedback futures: beyond the limits of human and GenAI capacities - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[lodge-loble-cognitive-offloading-2026]] — Artificial intelligence, cognitive offloading and implications for education - [[ai-tpack-teacher-multi-agent-workflow]] — Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI ## Citation Shahin Hossain, Sima Ahmadi, Leqi Li, Idowu David Awoyemi, Wei Huang, Chenxi Zhou, Jujia Li, Samaa Haniya, Shapla Khanam, Tasbirun Mashreka Subaha (2026). [Rethinking Generative AI Literacy: An Integrative, Developmental, and Dialectical Framework for K-12 Teacher Education](https://arxiv.org/abs/2608.01705). arXiv:2608.01705. arXiv (cs.CY / cs.HC) preprint. --- ## [Review of Artificial Intelligence in Education from 2020 to 2025](https://edtechdev.github.io/aied/articles/raza-farooq-aied-review-2020-2025/) > **Synthesis:** Raza & Farooq (2025) conduct a comprehensive content analysis of AI in education from 2020-2025, examining 100+ peer-reviewed articles through a three-layer framework: the genome layer (infrastructure, algorithms), the cognitive layer (predictive analytics, multimodal sensing, discourse analysis), and the symbiotic layer (learning platforms, smart classrooms, GenAI copilots). Three forward trends identified: human-AI co-orchestration, privacy-preserving edge/federated AI, and authentic multimodal assessment. ## Three-Layer Framework This review organizes AIED research into three interrelated layers: - **Genome Layer:** The foundational elements — instructional customs, evaluation practices, digital tools, and algorithmic schemas that underpin AI in education. - **Cognitive Layer:** Instructional mechanisms including [[personalized-learning|personalized learning]] via measure-model-adapt loops, predictive analytics, [[multimodal]] sensing, discourse and affect analysis. - **Symbiotic Layer:** End-to-end implementations including [[edtech-platform|learning platforms]], smart classrooms, automation of processes, and [[generative-ai|generative AI]] copilot systems. ## Key Findings - **100+ peer-reviewed articles** examined, covering empirical and analytical studies - **Three forward trends:** (1) human-AI co-orchestration as default classroom pattern; (2) privacy-preserving, edge/federated AI; (3) authentic multimodal assessment via [[generative-ai]] simulations - **Effects summarized** on learning outcomes, engagement, teacher workload, and institutional adoption - **Actionable guidance** for researchers, practitioners, and system leaders ## Implications This review provides a structured map of the AIED landscape during the LLM era (2020-2025), useful as a reference point for [[ai-education]]. The three-layer framework connects to [[learning-analytics]], [[intelligent-tutoring]], and [[adaptive-learning]] research, while the forward trends align with discussions in [[ai-governance-education]] and institutional AI readiness. ## Connected Concepts - [[ai-education]] - [[personalized-learning]] - [[generative-ai]] - [[learning-analytics]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[multimodal]] - [[edtech-platform]] - [[ai-governance-education]] ## Connected Articles - [[liang-genai-systematic-review-human-ai-2026]] - [[zerkouk-comprehensive-review-its-2025]] - [[genai-meta-analysis-programming-learning]] ## Citation Raza, S. H., & Farooq, A. (2025). [*Review of Artificial Intelligence in Education from 2020 to 2025*](https://doi.org/10.35542/osf.io/6bnez_v1). EdArXiv. doi:10.35542/osf.io/6bnez_v1. --- ## [Regulating the AI Tutor: SRL and Help-Seeking in Adolescent GenAI Use](https://edtechdev.github.io/aied/articles/regulating-ai-tutor-adolescent-srl/) Examines how 98 Grade-9 students across three German Gymnasium schools regulated their use of a Mistral-Large GenAI tutor while preparing for a math exam. Despite overwhelmingly selecting scaffolded support before the session, students' actual interactions were dominated by instrumental requests (asking for answers) with almost no explicit monitoring or evaluation of their own learning. **Critical finding:** Post-test performance was significantly lower than pre-test, and higher extraneous cognitive load predicted lower post-test scores after controlling for prior knowledge. This reveals an **intention-behavior gap** — students intend to use AI for learning but default to answer-seeking, undermining [[self-regulated-learning|self-regulated learning]] processes. The paper proposes a turn-level codebook combining SRL and [[metacognition|help-seeking constructs]] with LLM-specific codes (agency over AI, epistemic vigilance). Results support the need for [[scaffolding|scaffolds]] that promote more agentic and epistemically proactive GenAI use, and hybrid human-AI analysis of interaction patterns. Contributes directly to the [[over-reliance|over-reliance and cognitive offloading]] literature. ## Connected Concepts - [[self-regulated-learning]] - [[metacognition]] - [[scaffolding]] - [[over-reliance]] ## Connected Articles - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning - [[socraticode-k12-programming-tutor]] — Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participatory Design Study - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[llm-reasoning-traces-metacognition]] — Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning - [[scheu-mobile-chatbot-journaling-motivation-2026]] — Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement ## Citation Abdelghani, R., Kaiser, P., & Murayama, K. (2026). Regulating the AI Tutor: Intentions, Help-Seeking, and Self-Regulated Learning in Adolescent GenAI Use. [arXiv:2606.08568](https://arxiv.org/abs/2606.08568). --- ## [Reimagining feedback through generative AI in engineering education](https://edtechdev.github.io/aied/articles/reimagining-feedback-through-generative-ai-in-engineering-education/) > **Synthesis:** Pecuchova, Benko, and Drlik (2026) investigate the capacity of a large language model (GPTo1) to generate formative feedback for student-created UML diagrams in a university software engineering course. Across two cohorts (N = 262), AI-generated, teacher-generated, and no-feedback conditions were compared, analyzing student perceptions, learning outcomes, and grading reliability. Results show that while students rated GenAI feedback as beneficial and often comparable to human comments, teacher feedback remained more effective in supporting performance gains, especially in complex modeling tasks. Linguistic analysis revealed GenAI feedback was more repetitive and less pedagogically rich than human feedback. ## Key Findings - Students rated both feedback sources highly (GenAI mean = 4.09; teacher mean = 4.38 on a 5-point scale) and perceived GenAI feedback as beneficial; for some assignments (requirements specification, activity diagram) they found no significant difference between GenAI and teacher feedback, while teacher feedback was rated higher for use-case and class diagrams. - **Teacher feedback was more effective for learning**: students receiving teacher feedback gained on average 0.87 points more than those receiving GenAI feedback on score gain, and scored significantly higher on the automatically evaluated theoretical quizzes (p < 0.01) — though feedback source did not significantly influence final theoretical exam scores. - Feedback source significantly influenced performance on the **visual modeling tasks** of the final exam (p < 0.05), where teacher feedback led to better outcomes — teacher feedback was more effective in helping students master abstract modeling tasks. - **Perceived feedback quality** (not source alone) significantly predicted final exam performance; a mediation analysis suggested feedback source influenced perceived quality (β = −0.28, marginally non-significant p = 0.056). - **GenAI feedback was more repetitive and less linguistically rich**: teacher feedback had higher TTR, Hapax, and Entropy values. GenAI feedback skewed heavily toward diagnostic feedback (66% vs. teacher's 48%), was predominantly positive/neutral in early tasks but overly punitive in complex visual modeling, and had reduced capacity for nuance with partial or ambiguous diagrammatic content. - The findings argue for **hybrid feedback architectures** where GenAI operates as a first-pass evaluator and instructors retain formative oversight, given GenAI's "brittle" pedagogical judgment in ambiguity, abstraction, and design variation. ## Study Design & Method This empirical study was conducted across two cohorts (N = 262 students) in a university software engineering course at Constantine the Philosopher University in Nitra, Slovakia, over two academic years (2023/2024, 2024/2025). Students completed four UML modeling assignments (requirements specification, use case diagram, class diagram, activity diagram) and six theoretical quizzes, followed by a final exam with three UML diagram tasks. A structured dataset of over 1800 unique student submissions with 68 variables per student was compiled. The GPTo1 model evaluated diagrams using nine prompt configurations (3 per diagram type), and results were compared with two independent human graders. Feedback was categorized by a human evaluator into four types (diagnostic, design, metacognitive, motivational), analyzed via two-way ANOVA, Mann-Whitney U tests, chi-square tests, Spearman correlation, and mediation analysis. A custom Slovak lexicon-based sentiment analysis and QuitaUp linguistic metrics (TTR, Hapax, Entropy) were used to assess feedback tone and richness. Students rated feedback on a 5-point Likert scale without knowing the source. ## Implications for AI in Education The study provides empirical evidence on GenAI's capacity for formative feedback on complex, open-ended diagrammatic tasks — a domain where feedback is assessment-intensive and hard to scale. It shows GenAI can provide high-volume, well-received feedback but its pedagogical judgment is "brittle" under ambiguity, abstraction, and design variation, making it best suited as a first-pass evaluator within a hybrid human-AI feedback system. It contributes to the [[ai-feedback-quality]] and [[formative-assessment]] literature, connects to [[self-regulated-learning]] (feedback's role in driving learning), and positions GenAI feedback as reshaping assessment models, curriculum design, and faculty roles — democratizing access to formative feedback while raising equity, accountability, and policy challenges. The finding that perceived feedback quality (not source) drives outcomes suggests educators should focus on ensuring feedback is high-quality and actionable regardless of its origin. ## Limitations The study is context-specific (a Slovak university software engineering course; UML diagram tasks in a domain-specific language), and the sentiment analysis relied on a custom heuristic Slovak lexicon rather than validated general-purpose tools. The dataset included a high proportion of international students, but nationality was not used in analysis. The no-feedback control condition's effects and long-term retention are not fully separated from the main comparisons. As a single-institution study, generalizability to other disciplines, languages, and feedback contexts requires replication. ## Connected Concepts - [[generative-ai]] - [[formative-assessment]] - [[ai-feedback-quality]] - [[higher-ed]] - [[curriculum-design]] - [[self-regulated-learning]] - [[cognitive-offloading]] - [[automated-essay-scoring]] ## Connected Articles - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and Teacher Feedback - [[multimodal-ai-feedback-learning]] — LLM-Based Multimodal AI Feedback Produces Equivalent Learning - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[pairr-ai-peer-review-2025]] — Peer and AI Review + Reflection (PAIRR) - [[ai-internal-feedback-evaluative-judgments]] — Unravelling Undergraduates' Development of Evaluative Judgments - [[rail-ed-genai-literacy-teacher-education]] — AI Literacy Training for Teachers - [[liang-genai-systematic-review-human-ai-2026]] — GenAI Systematic Review: Human-AI - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap ## Citation Pecuchova, J., Benko, L., & Drlik, M. (2026). [*Reimagining feedback through generative AI in engineering education*](https://doi.org/10.1016/j.caeai.2026.100574). *Computers and Education: Artificial Intelligence*. --- ## [Reinforcement Learning Measurement Model](https://edtechdev.github.io/aied/articles/reinforcement-learning-measurement-model-assessment/) # Reinforcement Learning Measurement Model for Interactive Assessment ## Key Findings Interactive assessments generate sequential process data that conventional item response models (IRT) cannot adequately handle. This paper proposes a **reinforcement learning measurement model** that links action choices to state-action values, extending beyond existing MDP-based measurement approaches. The model addresses the gap between traditional static assessment models and the dynamic, interactive nature of modern computer-based assessments. It builds on prior work (LaMar, 2018) but improves reliability of estimates for interactive assessments where students' action sequences carry diagnostic information. ## Connections to AIED This work directly extends [[knowledge-tracing-irt]] by replacing static IRT with a dynamic RL-based approach. It connects to [[assessment]] by providing a new framework for scoring interactive assessments that capture student problem-solving processes rather than just final answers. The approach also relates to [[knowledge-tracing]] and [[learning-analytics]] as it extracts diagnostic signals from interaction data that go beyond traditional performance metrics. ## Connected Concepts - [[assessment]] - [[knowledge-tracing]] - [[learning-analytics]] ## Connected Articles - [[knowledge-tracing-irt]] ## Citation Xu, W., & Ji, F. (2026). [Reinforcement learning measurement model](https://arxiv.org/abs/2605.09305). arXiv:2605.09305. --- ## [Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols](https://edtechdev.github.io/aied/articles/reliable-programming-kt/) ## Key Findings - Programming Knowledge Tracing (PKT) has advanced through hybrid attention+RNN architectures, which have become the dominant modeling approach in the subfield. - The re-evaluation identifies critical protocol issues in prior work: attention dimension misconfiguration and temporal causality violations. - Under controlled experiments, the advantage of attention-enhanced models over standard DKT is significantly reduced. - The findings challenge published claims of attention-augmented superiority and underscore the need for reliable experimental protocols in educational data mining. ## Analysis This ITS 2026 paper challenges claims about attention-augmented Programming Knowledge Tracing (PKT) superiority. The authors identify three critical protocol flaws: **attention dimension misconfiguration**, **temporal causality violations** (ignoring ServerTimestamp), and **hyperparameter leakage** across CV folds. When controlled, the gap between attention-enhanced models and standard DKT narrows significantly. This reinforces [[ground-truth-reliability-aied]] by showing that experimental protocol flaws can produce misleading performance claims. It also has implications for benchmark design in educational AI, arguing for standardized evaluation frameworks. The CodeWorkout dataset protocol serves as a template for [[programming-its]] research. ## Implications for Benchmark Design The paper's central message is methodological: performance claims in [[knowledge-tracing]] research are only as trustworthy as the protocols that produce them. Attention dimension misconfiguration can distort what a model learns from interaction sequences, while temporal causality violations risk leaking future information into predictions and inflating apparent accuracy. For the AI-in-education community, the lesson is that model comparisons should be run under controlled, preregistered-style protocols before informing [[student-modeling]] or automated assessment tools. The re-evaluation also points to the value of re-analysis culture in [[benchmark]] development: revisiting established results with rigorous controls can narrow apparent gaps between sophisticated architectures and simpler baselines, redirecting research effort toward modeling choices that genuinely matter. ## Connected Concepts - [[knowledge-tracing]] - [[student-modeling]] - [[benchmark]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[cognitive-diagnosis]] - [[knowledge-graph]] - [[feedback-loop]] ## Connected Articles - [[ground-truth-reliability-aied]] - [[programming-its]] ## Citation Kim & Kim (2026). [Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols](https://arxiv.org/abs/2605.04727). arXiv:2605.04727. ITS 2026 (Springer LNCS). --- ## [Play-Testing REMind: Evaluating an Educational Robot-Mediated Role-Play Game](https://edtechdev.github.io/aied/articles/remind-robot-mediated-roleplay-antibullying-2026/) > **Synthesis:** Sanoubari, Fernandes, Rebello, Pan, Houston, and Dautenhahn (2026) present REMind, an educational robot-mediated role-play game designed to support anti-bullying bystander intervention among children. REMind invites players to observe a bullying scenario enacted by social robots, reflect on the perspectives of the characters, and rehearse defending strategies by puppeteering a robotic avatar. Evaluated through a mixed-methods play-testing study with 18 children aged 9–10, the findings suggest the experience supported learning goals related to self-efficacy, perspective-taking, understanding outcomes of defending, and intervention strategies — highlighting Robot-Mediated Applied Drama (RMAD) as a novel pedagogical framework for social-emotional learning. ## Key Findings - **REMind is an educational robot-mediated role-play game** supporting anti-bullying bystander intervention among children. - The game has players **observe a bullying scenario enacted by social robots, reflect on character perspectives, and rehearse defending strategies by puppeteering a robotic avatar**. - A **mixed-methods play-testing study with 18 children aged 9–10** evaluated the experience. - Findings suggest the experience supported key learning goals: **self-efficacy, perspective-taking, understanding outcomes of defending, and intervention strategies**. - The work highlights **Robot-Mediated Applied Drama (RMAD)** as a novel pedagogical framework to support **social-emotional learning**. ## Study Design & Method This is a **mixed-methods play-testing study**. The researchers designed REMind, an educational robot-mediated role-play game for anti-bullying bystander intervention, and evaluated it with 18 children aged 9–10. In the game, children observed bullying scenarios enacted by social robots, reflected on the perspectives of the characters, and rehearsed defending strategies by puppeteering a robotic avatar. Mixed-methods data collection assessed the experience's support for learning goals including self-efficacy, perspective-taking, understanding outcomes of defending, and intervention strategies. ## Implications for AI in Education REMind demonstrates how [[social-robots]] can support [[social-emotional-learning]] through embodied role-play, introducing **Robot-Mediated Applied Drama (RMAD)** as a pedagogical framework. By letting children rehearse anti-bullying bystander intervention strategies in a safe, embodied context, the approach builds [[self-efficacy]], perspective-taking, and empathy in [[k-12]] settings. It connects to [[educational-robotics]], role-play/game-based learning, and the use of embodied AI to address social and emotional learning outcomes — a distinct application beyond technical/programming robotics education. ## Limitations The evaluation involved a small sample (18 children aged 9–10) in a play-testing context, limiting generalizability and evidence for sustained behavior change. The focus is on learning-goal support during the experience rather than long-term anti-bullying outcomes. The robot-mediated role-play setup requires specific hardware and facilitation. ## Connected Concepts - [[social-robots]] - [[social-emotional-learning]] - [[educational-robotics]] - [[k-12]] - [[self-efficacy]] - [[student-engagement]] ## Connected Articles - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo: Digital Storytelling Robots - [[icub-humanoid-storytelling-llm-hri-2025]] — LLM-Powered Narrative HRI - [[task-context-trust-educational-hri-2026]] — Task Context and Trust in Educational HRI ## Citation Sanoubari, E., Fernandes, N., Rebello, K., Pan, A., Houston, A., & Dautenhahn, K. (2026). *Play-testing REMind: Evaluating an educational robot-mediated role-play game*. arXiv:2604.00300. --- ## [Student Evaluation of Repeated AI Feedback Across a Semester of Writing](https://edtechdev.github.io/aied/articles/repeated-ai-writing-feedback-semester/) This short paper provides rare descriptive classroom evidence on what happens when students repeatedly use generative-AI feedback across a full semester of writing coursework. Drawing on 2,988 reflective essay-feedback-appraisal instances from 283 Estonian bachelor students, the authors find that students rated AI feedback as helpful and actionable more often than not, but a growing minority (about one in ten) found it unhelpful toward the end of the term. The work sits squarely in the [[ai-generated-feedback-higher-ed]] literature and complements prior [[ai-feedback-quality]] studies by tracking feedback appraisal longitudinally rather than in a one-off lab task. The study surfaces the central tension in [[over-reliance]]: generative AI offers a fast, scalable route to immediate writing advice, but it is not a self-contained path to deeper reflection. Using a validated AI-text classifier, the authors estimate the share of essays that look like unaided student writing, linking tool use to the broader question of whether AI assistance erodes learning gains. These findings reinforce concerns echoed in [[generative-ai-reduced-study-time-math]] and [[sequenced-ai-feedback-learning]] about dosage and critical engagement. The paper argues benefits depend on whether students learn to use AI selectively and critically, a skill squarely within [[ai-literacy]] and the [[student-experience]] of writing support in [[writing-education]]. ## Connected Concepts - [[ai-feedback-quality]] - [[over-reliance]] - [[ai-literacy]] - [[student-experience]] - [[writing-education]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] - [[generative-ai-reduced-study-time-math]] - [[sequenced-ai-feedback-learning]] ## Citation Karjus, A., Leoste, J., & Oun, T. (2026). [Student Evaluation of Repeated AI Feedback Across a Semester of Writing](https://arxiv.org/abs/2607.16115). arXiv:2607.16115. --- ## [Representation Robustness under Executable Reasoning Constraints in Large Language Models for Mathematical Problem Solving](https://edtechdev.github.io/aied/articles/representation-robustness-llm-math-problem-solving/) This study probes how sensitive [[llm]] mathematical problem solving is to the surface representation of an item — a question with direct bearing on [[assessment-validity]] when LLMs are used for scoring or tutoring in [[stem-education]]. Systematically varying representationally equivalent formulations (story problems, word-equations, symbolic equations, and isomorphic paraphrases) across 5 contemporary LLMs, the authors find substantial representational sensitivity: models frequently flip correctness across equivalent formulations, and even subtle paraphrase-level changes degrade performance despite preserved mathematical structure. A second, code-augmented condition constraining models to externalize reasoning as executable Python reveals strong latent capability in weak models but does not uniformly improve robustness — instead failures shift from opaque reasoning errors to protocol and execution violations. The work cautions that treating formulations as interchangeable conflates reasoning errors with interface failures, complicating [[ai-tutoring]] and diagnostic uses like [[llm-cognitive-diagnosis-handwritten-math]]. It connects to measurement concerns in [[reinforcement-learning-measurement-model-assessment]] and to reasoning scaffolds in [[epistemic-proactivity-math]]. ## Connected Concepts - [[llm]] - [[assessment-validity]] - [[stem-education]] - [[ai-tutoring]] ## Connected Articles - [[llm-cognitive-diagnosis-handwritten-math]] - [[reinforcement-learning-measurement-model-assessment]] - [[epistemic-proactivity-math]] ## Citation Nath, Graf, Zhang & Zapata-Rivera (2026). [Representation Robustness under Executable Reasoning Constraints in Large Language Models for Mathematical Problem Solving](https://arxiv.org/abs/2607.20520). arXiv:2607.20520. HCI International 2026. --- ## [Reshaping Undergraduate Computer Science Education in the Generative AI Era](https://edtechdev.github.io/aied/articles/reshaping-cs-education-genai/) **Yi-Chieh Lee, Nattapat Boonprakong, Yugin Tan, Harold Soh et al.** — Workshop report from NUS-Google Workshops — cs.CY This white paper synthesizes findings from two international NUS-Google Workshops in Singapore convening faculty, industry practitioners, and students to reshape undergraduate CS education in response to generative AI. The central argument is that as GenAI automates implementation-level programming, debugging, and testing, CS curricula must shift toward understanding and verifying AI-generated artifacts. Critical skills to preserve include system design, abstraction, and critical evaluation; skills becoming less important include low-level implementation details. The paper proposes prerequisites for reform: fostering AI-native competencies, re-centering fundamental education, enhancing advanced pathways, embracing new pedagogies, and shifting institutional support. This connects to broader debates in [[cs-education]] about what constitutes core competence in an era of AI pair programmers, and aligns with [[ai-literacy]] frameworks that emphasize evaluation over generation. ## Connected Concepts - [[cs-education]] - [[ai-literacy]] ## Connected Articles - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education ## Citation Lee, Y.-C., Boonprakong, N., Tan, Y., Soh, H., et al. (2026). [*Reshaping Undergraduate Computer Science Education in the Generative AI Era*](https://arxiv.org/abs/2606.07545). arXiv:2606.07545. --- ## [ResidencyRL: Reinforcement Learning in Simulated Clinical Environments](https://edtechdev.github.io/aied/articles/residencyrl-clinical-rl-training-2026/) > **Synthesis:** Liévin et al. (2026) present **ResidencyRL**, a reinforcement learning method for training clinical AI agents through simulated multi-turn clinical encounters (up to 60 dialogue turns and 8 tool calls per trajectory). It pairs the policy agent with LLM simulators capable of complex, adversarial behaviors, training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety. On held-out evaluation the agent improves diagnostic accuracy by 7.0% under adversarial conditions (88.0% vs 81.0%) and reduces missed red-flag rates by 31%; blinded expert clinicians preferred it in 87.6% of side-by-side comparisons, and competencies transfer to unseen benchmarks. The work connects to [[reinforcement-learning]], [[simulation]], and [[professional-training]] in health education. ## Learning Clinical Decision Sequences In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters. While LLMs excel on static medical benchmarks, methods to optimize the full sequence of clinical decisions remain underdeveloped. ResidencyRL trains clinical AI agents through simulated multi-turn clinical encounters. ## Method ResidencyRL pairs the policy agent with LLM simulators capable of complex, adversarial behaviors, training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety. ## Findings On held-out evaluations the agent improves diagnostic accuracy by 7.0% under adversarial conditions (88.0% vs 81.0%) and reduces missed red-flag rates by 31%, demonstrating rigorous mitigation of premature closure. Blinded expert clinicians preferred the trained agent in 87.6% of side-by-side comparisons. Procedural competencies transfer to unseen benchmarks, outperforming the base model across all six clinical axes of the AMIE multi-visit benchmark with consistent directional improvements on AgentClinic and CRAFT-MD. ## Connected Concepts - [[reinforcement-learning]] - [[simulation]] - [[professional-training]] - [[professional-training]] - [[llm]] - [[ai-tutoring]] - [[trust-calibration]] - [[cognitive-diagnosis]] - [[human-in-the-loop-ai]] - [[pedagogical-safety]] ## Connected Articles - [[tibetcpr-ai-training-feedback]] - [[astra-atco-training-simulator]] - [[medgame-llm-medical-education-gamification]] - [[ai-use-critical-thinking-medical-students-2026]] - [[pedagogical-safety-rl]] - [[hazra-safetutors-pedagogical-safety-2026]] ## Citation Liévin, V., Schmidgall, S., Strother, T., Bijamov, A., Goel, A., Palepu, A., et al. (2026). [*ResidencyRL: Reinforcement learning in simulated clinical environments*](https://arxiv.org/abs/2608.07418). arXiv:2608.07418. --- ## [Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference](https://edtechdev.github.io/aied/articles/responsible-assessment-ai-era-stanford-2026/) > **Responsible assessment in the AI era** — assessment grounded in learners' sociocultural contexts and designed to generate valid, trustworthy, context-specific inferences from accumulated evidence, not one-shot outputs. This Stanford Accelerator for Learning + ETS white paper (McGee, Thille, Choi, Ercikan & Hau, 2026, distilled from a January 2026 convening of ~100 education leaders) argues generative AI has broken the assumption that final products measure human capability: learners can produce high-quality artifacts without the underlying learning, AI scoring introduces construct-irrelevant variance, and the gap between what is measured and what matters is widening. The field's response is a shift from testing events to systems of inference — continuous and [[formative-assessment]], portfolio- and conversation-based evidence ([[socratic-tests-conversational-assessment]]), [[authentic-assessment]] in real tasks, and [[human-in-the-loop-ai]] design — paired with validity infrastructure for [[automated-grading]], shared definitions of emerging constructs like [[ai-literacy]], and sustained attention to [[equity]], transparency, and trust. The report synthesizes a future-focused convening (January 29, 2026) on how assessment should evolve as AI reshapes learning, work, and measurement. It defines **responsible assessment** as assessment that is grounded in individuals' sociocultural contexts and designed to produce valid, trustworthy, context-specific inferences from accumulated evidence — a shift toward continuous, context-rich, developmentally oriented practices that leverage AI responsibly (building on Johnson, 2025). **Why traditional assessment is evolving.** Three interconnected pressures: (1) *Misalignment with how learning actually occurs* — learning is a process, but assessment remains event-based; conditions that support learning (risk-taking, mistake-making) are absent in assessment mode, and assessment misses the collaborative, AI-supported pathways students actually take. (2) *The increasing influence of AI* — GenAI means outputs alone can no longer serve as reliable indicators of human capability, so assessment "risks measuring technological proficiency rather than human skill." AI scoring (the most widely used AI application in assessment) can capture construct-irrelevant features (e.g., rewarding language competency instead of scientific reasoning), which becomes bias when it systematically advantages or disadvantages particular learners. Other validity threats: construct underrepresentation, failures of generalization, training data that don't represent authentic child learning (especially K-12), and calibration differences between what humans and AI attend to in the same rubric. Meanwhile AI literacy and "durable skills" (critical thinking, creativity, curiosity, collaboration, agency, adaptability) rise in importance, and workforce shifts (30% of jobs automated within four years per McKinsey; 39% of core skills changing by 2030 per WEF) demand assessments of the skills workers will actually need. (3) *A growing gap between what we measure and what matters* — measurement signals what systems value; assessment shapes curriculum and public judgments of quality. Advances in data availability (stealth assessment grounded in evidence-centered design, continuous assessment, process/ambient/longitudinal data) expand what can be observed, but more data does not automatically produce better insight, and privacy, learner awareness, and overinterpretation loom. Human-centered skills remain underdefined — AI literacy and adaptability lack shared operational definitions, though the OECD/EC AILit Framework is emerging (feeding the 2029 PISA Media and AI Literacy assessment). Existing/emerging tools: cognitive interviews, live demonstrations, role-playing, scenario-based assessments, portfolios, conversation-based assessments (ECD + AI agents), virtual robotics tasks, and game-based assessment in naturalistic settings (e.g., Guess What?). **What system change requires.** (1) *Testing events → systems of inference*: summative results often arrive too late to help; a development-centered orientation elevates formative assessment and AI agents that generate questions and evidence on demand. (2) *Infrastructure and validity for AI-powered assessment*: data architectures, interoperable platforms, and sustained human investment are prerequisites — "without infrastructure, even the most advanced AI-supported assessment cannot function effectively"; LLM scoring of constructed responses may be less reliable and *more* costly once validity-evidence curation is counted, and requires more extensive validity evidence than traditional scoring. (3) *Expanding what counts as evidence*: relational and social learning (Vygotskian social constructivism, teacher–student relationships, collaborative problem-solving) is poorly captured by traditional systems; a broader evidentiary base — multiple sources, inclusive demonstrations (leadership shown through caring for siblings, not just sports captaincies) — is needed. (4) *Extending formative assessment*: embedded feedback loops, peer formative assessment compared against AI-generated feedback, culturally responsive AI apps (e.g., MOSAIC), and a five-step framework for integrating AI into formative assessment — while acknowledging AI bias, inaccurate information, and the ethics of passive data collection ("the more data we're able to collect passively, the harder those questions are going to become"). **Advancing responsible assessment.** Reflection: responsibility begins with the people assessment serves — humanizing assessment, keeping humans in the loop, co-designing with diverse stakeholders, recognizing learner uniqueness, and contextualizing design. Trust through transparency: trust depends on what a system can reliably do (Brunskill's microwave analogy — users need faith in safeguards, not full inner mechanics), demonstrated value ("give educators what they don't have"), technical documentation (ETS's automated-scoring docs), adversarial testing and efficacy studies, and *limits* on transparency (protecting mechanics to preserve trustworthiness; AI extrapolation to small cultural groups produces nonsensical outputs). Action across four stakeholder groups: education systems should pilot continuous embedded assessment, adopt portfolio/competency-based approaches, reduce reliance on high-stakes one-time testing, build educator capacity, and engage learners/families; researchers should clarify constructs, advance ecological validity, develop validity standards for AI-assisted assessment, and leverage new evidence forms responsibly (e.g., the MAGIC Project measuring curiosity and inquiry); developers should design for interpretability, support learning processes not just outputs, enable human-in-the-loop workflows, and avoid fully automated high-stakes decision-making; funders should invest in new constructs and measurement models, shared infrastructure (longitudinal data systems, synthetic data environments), translation, and coordinated mechanisms. **Conclusion.** Assessment must evolve without losing fairness and validity. High-stakes assessment retains an essential role, but assessment must also show how learners progress over time. "AI can expand what is feasible, but it cannot be held accountable for the consequences of assessment decisions. People can." ## Connections to the Wiki - **Assessment validity & AI scoring** — the report's catalog of validity threats (construct-irrelevant variance, underrepresentation, generalization, calibration) is a policy-level complement to the wiki's empirical evidence on [[automated-grading]] failures (e.g., [[llm-handwritten-math-grading]], [[ai-scoring-language-bias-physics]], [[machines-misread-pedagogical-quality]]). - **Formative assessment** — the call to extend [[formative-assessment]] with AI (peer feedback vs. AI feedback, culturally responsive tools) connects to [[ai-generated-feedback-higher-ed]] and [[feedback-loop]] research. - **Conversation-based assessment** — ECD-based AI-agent dialogue assessment aligns with [[socratic-tests-conversational-assessment]] and [[intelligent-tutoring]] design. - **Construct definitions** — the underdefined AI literacy / durable-skills problem echoes [[ai-literacy]] debates and educational-theory work on what AI-era competencies mean operationally. - **Equity & trust** — sociocultural responsiveness, bias in AI scoring, and human accountability map to [[equity]], [[human-in-the-loop-ai]], and [[ai-ed-evaluation]]. ## Connected Concepts - [[ai-ed-evaluation]] - [[ai-literacy]] - [[assessment-validity]] - [[automated-grading]] - [[formative-assessment]] - [[generative-ai]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[ai-scoring-language-bias-physics]] — AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations... - [[authentic-assessment]] — Authentic Assessment - [[llm-handwritten-math-grading]] — Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs - [[machines-misread-pedagogical-quality]] — Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation - [[socratic-tests-conversational-assessment]] — The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis ## Citation ASSESSMENT, R., ERA, I.T.A., a, K.I.F., Conference, F., McGee, N.J., Thille, C., Choi, I., & Ercikan, K. (2026). [Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference](https://acceleratelearning.stanford.edu/conference/responsible-assessment-in-the-ai-era/) --- ## [Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments](https://edtechdev.github.io/aied/articles/rethinking-scaffolding-llm-tutors/) > **Alexandra Neagu, Jeffrey T. H. Wong, Marcus Messer, Rhodri Nelson, Peter B. Johnson** (2026). Pluralistic Alignment Workshop @ ICML 2026 ## Key Findings - Alignment and evaluation methods for embedding scaffolding behaviour into chatbots rest on an implicit assumption: that students will take up the scaffolding and engage in the conversation. - The paper introduces an evaluation pipeline around two metrics — Chatbot Scaffolding and Student Uptake — applied across nine datasets of 9,490 chats spanning AI tutor benchmarks and real-world deployments of educational chatbots. - While benchmarks assume a high-scaffolding, high-student-uptake environment, students in real-world settings exhibit lower levels of uptake overall, frequently bypassing the chatbot's pedagogical framing to drive the interaction toward their own learning goals at little interpersonal cost. - Bypassing scaffolding is not necessarily detrimental; it frequently highlights a mismatch between a chatbot's pedagogical framing and the student's learning goals. - Future benchmarks must move beyond the assumption that students will simply take up the scaffolding, and instead evaluate how chatbots navigate diverse learning contexts and student-driven interaction patterns. ## Study Design & Method Scaffolding describes how a tutor calibrates support to the learner's current state — guiding through graduated hints, posing questions rather than giving answers, and withdrawing support as the student gains competence. Delivering timely, dialogic, and scaffolded feedback to every student at every moment of struggle is difficult at scale, and LLM-based chatbots have been proposed as a way to approach this challenge. However, deploying LLMs as tutors introduces a tension: they are trained to be helpful by presenting information and answering directly, rather than engaging students in guided discovery — behaviour that is at odds with scaffolding, where a tutor withholds answers to promote reasoning. The evaluation pipeline operationalizes this tension through the Chatbot Scaffolding and Student Uptake metrics, and the corpus spans both benchmark datasets and real-world chatbot deployments. ## Relevance to AI in Education This paper contributes directly to understanding how AI systems interact with learners in authentic educational settings. It challenges benchmark assumptions about student uptake of LLM tutor scaffolding, showing that real-world learners frequently bypass pedagogical framing in favour of their own goals, and that this behaviour is often a rational response to a mismatch rather than a failure of engagement. For [[ai-tutoring]] design, the implication is that scaffolding should be adaptive to student-driven interaction patterns — including [[help-seeking]] styles — rather than presupposed by the interface. The conversational structure of tutoring normally allows students to respond, negotiate, and ask follow-up questions, building understanding iteratively and exercising agency; benchmarks that ignore this dynamic risk overestimating both the value of rigid scaffolding and the quality of LLM tutors. For the [[benchmark]] community, the work argues for evaluation designs that reward chatbots for navigating diverse learning contexts instead of assuming uptake. ## Connected Concepts - [[help-seeking]] - [[ai-tutoring]] - [[pedagogical-llm-training]] - [[benchmark]] - [[affective-tutoring]] - [[socratic-ai-dialogue]] - [[pedagogical-agent]] - [[automated-question-generation]] ## Connected Articles - [[llm-judged-helpfulness-pedagogy-signal]] — Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents - [[zhang-tutormoments-2026]] — When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle - [[didactical-teacher-assistant-dimensional-modeling]] — A didactical-driven teacher assistant for a dimensional modeling course ## Citation Alexandra Neagu, Jeffrey T. H. Wong, Marcus Messer, Rhodri Nelson, Peter B. Johnson (2026). [Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments](https://arxiv.org/abs/2606.15766). arXiv:2606.15766. Pluralistic Alignment Workshop @ ICML 2026. --- ## [Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education](https://edtechdev.github.io/aied/articles/retrieval-augmented-tutoring-algorithm-kite/) ## Synthesis KITE (Knowledge-Informed Tutoring Engine) introduces a [[intelligent-tutoring]] architecture that grounds its responses in course materials through a multimodal [[scaffolding|RAG pipeline]]. Unlike generic LLM tutors that may drift from curriculum content, KITE retrieves relevant material — lecture slides, problem sets, code examples — before generating Socratic hints, guiding questions, and progressive scaffolds tailored to the student's current error. The system's intent-aware Socratic response strategy builds on [[socratic-ai-dialogue]] principles but operationalizes them through retrieval: rather than relying on the model's parametric knowledge of how to tutor, KITE anchors every response in authoritative course content. This design addresses a key gap identified in [[ai-tutor-effectiveness-review]] — that pedagogical quality alone is insufficient without content fidelity. KITE was evaluated through three complementary lenses. RAGAs-based metrics confirmed response grounding and quality. Expert pedagogical review assessed the educational appropriateness of its Socratic scaffolding. Most innovatively, a simulated student pipeline — where a weaker LLM acts as a student across two-turn dialogues — measured whether KITE's [[formative-assessment|feedback]] actually improved follow-up accuracy on procedural and algorithm tracing questions. Results were positive: simulated students produced more accurate responses after receiving KITE feedback, particularly on tracing tasks where step-by-step reasoning is critical. This work contributes to the growing literature on [[llm-student-modeling-memory|LLM-based student modeling]] by demonstrating that retrieval-augmented tutoring can maintain both pedagogical and content fidelity. It also connects to [[programming-its]] research by targeting algorithmic reasoning specifically, and to [[personalized-learning]] through its adaptive response strategies. The simulated student evaluation approach offers a replicable, low-cost methodology that could accelerate tutor development cycles — a theme also explored in [[eduagentbench-agent-teaching-benchmark]] for broader tutor evaluation. ## Connected Concepts - [[intelligent-tutoring]] - [[scaffolding]] - [[socratic-ai-dialogue]] - [[formative-assessment]] - [[personalized-learning]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[llm-student-modeling-memory]] - [[programming-its]] - [[eduagentbench-agent-teaching-benchmark]] ## Citation Jain, M., Bhatt, T., Pitts, G., Pandya, A., Brusilovsky, P., Norouzi, N., Hellas, A., Leinonen, J., & Akram, B. (2026). [Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education](https://arxiv.org/abs/2605.12988). arXiv:2605.12988. BEA 2026. --- ## [RoboBlockly Studio: Conversational Block Programming With Embodied Robot Feedback for Computational Thinking](https://edtechdev.github.io/aied/articles/roboblockly-conversational-block-robotics-ct-2026/) > **Synthesis:** Li, Du, Sun, and colleagues (2026) design and evaluate RoboBlockly Studio, an integrated interactive system that combines block-based programming, a conversational AI teaching agent, and embodied robot execution to support computational thinking. Recognizing that learners and teachers face challenges connecting abstract program logic to meaningful outcomes, the system creates a tight iterative loop of authoring, running, observing, and revising. Informed by interviews with five programming teachers, the system was designed to support four goals: preserving learner agency, making program behavior transparent, grounding programming in embodied classroom-aligned tasks, and scaffolding reflection through AI dialogue. It was deployed with 32 high school students. ## Key Findings - Computational thinking (CT) is increasingly promoted as a core literacy, yet learners and teachers face challenges **connecting abstract program logic to meaningful outcomes**. - **RoboBlockly Studio combines block-based programming, a conversational AI teaching agent, and embodied robot execution**, creating a tight iterative loop of authoring, running, observing, and revising. - Informed by **interviews with five programming teachers**, the system supports four goals: **(1) preserving learner agency**, (2) making program behavior **transparent and interpretable**, (3) grounding programming in **embodied, classroom-aligned tasks**, and (4) **scaffolding reflection** through pedagogically grounded AI dialogue. - The system was **deployed with 32 high school students**, with observation of how robot and AI dialogue affected learning. ## Study Design & Method This is a **design-based development and deployment study**. The researchers designed RoboBlockly Studio through iterative consultation (interviews with five programming teachers informed the four design goals), combining block-based programming with a conversational AI teaching agent and embodied robot execution. The system was deployed with 32 high school students, and the researchers observed how robot execution and AI dialogue supported computational thinking, including learner agency, program-behavior transparency, and reflection. ## Implications for AI in Education RoboBlockly Studio addresses the challenge of making [[computational-thinking]] concrete by grounding abstract block programs in embodied robot execution and scaffolding reflection with a conversational AI agent. It connects to [[programming-education]], [[programming-education|block-based programming]], [[educational-robotics]], and [[llm|LLM-based]] learning assistants. The design emphasis on **preserving learner agency** and **transparency** speaks to responsible AI tutoring design, and the embodied feedback loop supports learners in connecting code to real outcomes in [[k-12]] settings. ## Limitations The deployment involved 32 high school students, and detailed learning-outcome data are not fully reported in the abstract; the focus is on the design and observed use of the system. The system's effectiveness relative to other computational-thinking approaches requires comparative evaluation. Findings are specific to the high-school context and the particular robot/AI configuration. ## Connected Concepts - [[computational-thinking]] - [[programming-education]] - [[educational-robotics]] - [[llm]] - [[embodied-learning]] - [[k-12]] ## Connected Articles - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM: LLMs and Robotic Simulation - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[computational-thinking-educational-robotics-secondary-2026]] — Computational Thinking and Educational Robotics ## Citation Li, L., Du, C., Sun, J., et al. (2026). *RoboBlockly Studio: Conversational block programming with embodied robot feedback for computational thinking*. arXiv:2605.12059. doi:10.1145/3800645.3813071. --- ## [RoboBuddy in the Classroom: Exploring LLM-Powered Social Robots for Storytelling in Learning and Integration Activities](https://edtechdev.github.io/aied/articles/robobuddy-llm-social-robots-classroom-2025/) > **Synthesis:** Tozadore, Ertug, Chaker, and Abderrahim (2025) present RoboBuddy, an intuitive interface that lets teachers create scenario-based storytelling activities from their regular curriculum using LLMs and social robots. The system addresses two practical classroom challenges: the significant planning time required to create improvised scenarios for content delivery (intensified when using complex technologies like social robots), and the need to embed multicultural integration into an already tight curriculum. The authors co-designed activity frameworks with four teachers and deployed the system in a week-long study with 27 students. ## Key Findings - **RoboBuddy enables teachers to create scenario-based activities from their regular curriculum using LLMs and social robots**, reducing the planning burden associated with improvisational content delivery and social-robot use. - The system **addresses multicultural integration** within regular curriculum activities, tackling it as part of everyday learning rather than as a separate add-on. - **Co-design with four teachers** shaped the activity frameworks; the system was deployed in a study with 27 students over one week. - Findings validated the system's efficacy and highlighted the **positive impact of integration policies perceived by children**, and the importance of scenario-based activities for content engagement. ## Study Design & Method This is a **design-based/co-design and deployment study**. The researchers co-designed scenario-based activity frameworks with four teachers using an LLM-powered interface that generates storytelling activities from curriculum content for a social robot. The system was deployed in a classroom study with 27 students over one week, evaluating the system's efficacy in delivering scenario-based learning content and supporting multicultural integration. Data collection involved the classroom deployment and assessment of children's perceptions. ## Implications for AI in Education RoboBuddy demonstrates how [[llm|LLM-powered]] [[social-robots]] can make [[educational-robotics]] practical for teachers by lowering the planning barrier and integrating [[culturally-relevant-pedagogy|multicultural integration]] into regular [[k-12]] curriculum. It highlights the role of the [[teacher-role|teacher]] as orchestrator of scenario-based robotic activities and shows how generative AI can help teachers author content for social robots. This connects to [[human-robot-interaction]], [[storytelling-in-education|narrative-based learning]], and the use of embodied AI to support inclusive, culturally responsive classrooms. ## Limitations The deployment was a single-week study with 27 students, limiting generalizability and insight into long-term effects. The interface and activity frameworks were co-designed with four teachers, so the design reflects their context. The study focused on integration policies and scenario-based efficacy, with learning outcomes not comprehensively measured. ## Connected Concepts - [[social-robots]] - [[human-robot-interaction]] - [[llm]] - [[educational-robotics]] - [[teacher-role]] - [[k-12]] - [[student-engagement]] ## Connected Articles - [[icub-humanoid-storytelling-llm-hri-2025]] — LLM-Powered Narrative HRI With the iCub Humanoid - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Robot-LLM Integration and Embodiment in Creative Writing - [[social-robot-study-companions]] — Social Robots as Study Companions ## Citation Tozadore, D., Ertug, N., Chaker, Y., & Abderrahim, M. (2025). *RoboBuddy in the classroom: Exploring LLM-powered social robots for storytelling in learning and integration activities*. arXiv:2508.16706. --- ## [REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading](https://edtechdev.github.io/aied/articles/rubric-aware-grading-rec-cbm/) **REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models** advances the [[automated-grading]] frontier by solving a fundamental trust problem: even accurate AI graders are unusable if educators cannot verify their reasoning. Standard [[llm]]-based graders operate as black boxes, while earlier Concept Bottleneck Models (CBMs) offer interpretability but fail at modeling rubric dimensions, ordinal score semantics, and noisy human annotations. REC-CBM introduces three innovations: (1) a rubric-aware concept encoder that learns concept-specific representations aligned with actual grading rubrics, (2) an ordinal pairwise calibration objective that preserves score ordering (e.g., 'poor' < 'fair' < 'good'), and (3) a latent error-correction module that denoises concept predictions while maintaining full interpretability. Experiments demonstrate consistent improvements in both grading accuracy and concept-level reasoning faithfulness over baselines. This work directly addresses [[assessment-validity]] concerns raised in [[genai-assessment-governance]] and complements [[automatic-short-answer-grading]] by adding the interpretability dimension. The rubric-aware design aligns with [[formative-assessment]] needs and [[scaffolding]] principles, and the error-correction approach resonates with work on [[ground-truth-reliability-aied]]. ## Connected Concepts - [[automated-grading]] - [[llm]] - [[assessment-validity]] - [[formative-assessment]] - [[scaffolding]] ## Connected Articles - [[genai-assessment-governance]] - [[automatic-short-answer-grading]] - [[ground-truth-reliability-aied]] ## Citation Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu (2026). [REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading](https://arxiv.org/abs/2605.27402). arXiv:2605.27402. arXiv preprint. --- ## [Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts](https://edtechdev.github.io/aied/articles/same-ai-different-pathways/) > **Synthesis:** Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts ## Key Findings - A mixed-methods study of undergraduates across two discipline–institution contexts in Thailand — Design students at Raffles International College Bangkok (n = 221) and Business (BBA) students at Siam University (n = 222), recruited from Years 1–3 — modeled how AI literacy translates into coursework performance through two behavioral routes and two mediating mechanisms. - AI literacy significantly predicted both prompting proficiency and verification behavior (p < .001), confirming the proximal-behavior hypotheses; prompting proficiency was in turn positively associated with AI-supported calibration accuracy, while verification behavior was negatively associated with perceived task-related cognitive load. - Bootstrapped indirect effects of AI literacy on assignment quality were supported in both cohorts (bias-corrected 95% CIs, 5000 resamples): the prompting route (e.g., Design estimate .072, p < .001; Business .051, p = .001) and the verification→lower-load route (e.g., Design .034, p < .001; Business .024, p = .001). - The most performance-proximal route differed by discipline: verification behavior directly predicted task quality in the Design cohort, whereas AI-supported calibration accuracy — plus a direct AI-literacy path (p < .01) — predicted subject-specific assignment quality in the Business cohort, consistent with each discipline's "task ecology" (open-ended ideation vs. audit-like analytic standards). - Measurement invariance across the two cohorts was supported (ΔCFI ≤ .010), and the multigroup models showed acceptable fit (χ2/df below 3; CFI/TLI/IFI above .90; residual-based indices in acceptable ranges). - In 20 semi-structured interviews (thematic saturation reached by the 18th), Design students described using ChatGPT mainly to accelerate ideation and refinement, while Business students used it for structured analysis and to make their work more auditable; across both contexts, verification functioned as a metacognitive safeguard that reduced overload and stabilized accuracy. ## Study Design & Method The quantitative strand surveyed students immediately after course-embedded assignments, measuring AI literacy, prompting proficiency, verification behavior, AI-supported calibration accuracy (CAL), and perceived task-related cognitive load (ECL) via self-report, with rubric-scored performance outcomes (task quality for Design; subject-specific assignment quality for Business). Multigroup structural equation modeling with measurement invariance testing and bootstrapped indirect effects was used; for example, the configural invariance model reported χ2(170) = 356.1, CFI = .958, TLI = .942, RMSEA = .059, SRMR = .048. The qualitative strand consisted of 20 semi-structured interviews with a purposive, maximum-variation subsample, analyzed thematically in NVivo until saturation at the 18th interview; integration used joint displays mapping SEM paths to interview themes, with an audit trail and explicit attention to discrepant cases. ## Implications for AI in Education The findings argue for operationalizing [[ai-literacy]] in coursework as a set of teachable, assessable practices — prompting with intent, verifying with method, and relying with calibrated trust — rather than general tool familiarity. Practical levers suggested by the author include rubric-anchored [[prompt-engineering]] instruction, short "prompt studios" (10–15 minutes) embedded in tutorials, low-friction verification checklists and "evidence traces" (a short appendix of links/DOIs plus a 2–3 sentence rationale) for high-stakes assignments, and assessment policies differentiated by task ecology that reward documented judgment rather than volume of AI use — for example, reflective prompts asking students where they relied on, rejected, or revised AI output. The results also connect AI-mediated learning to [[metacognition]] and to the [[cognitive-offloading]] risks of AI-assisted work, suggesting verification routines as a counterweight. ## Limitations The design is non-experimental, so causal claims remain tentative despite theory-consistent directional modeling. Mediators were assessed via post-task self-report, and residual common-method bias cannot be fully ruled out. CAL captured students' self-assessed assignment quality and rubric fit rather than system-level trust calibration, and ECL may blend extraneous load with intrinsic task difficulty and time pressure. The single-author mixed-methods design creates structural risks that procedural sequencing mitigates but does not eliminate, and the sample draws on two programs within a single national context, bounding generalizability to local curriculum expectations, institutional policies, and discipline-specific assessment norms. ## Connected Concepts - [[ai-literacy]] - [[generative-ai]] - [[higher-ed]] - [[prompt-engineering]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries ## Citation Shen, Q. (2026). [*Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts*](https://doi.org/10.1016/j.caeai.2026.100601). --- ## [Faculty Readiness for AI-Supported Teaching and Scalable Online Program Delivery in Higher Education: The EPIQ-AI Framework for Epistemic Integrity](https://edtechdev.github.io/aied/articles/sangwa-epiq-ai-faculty-readiness-2026/) > **Synthesis:** Sangwa, Ndahayo & Dusengumuremyi (2026) develop the EPIQ-AI Readiness Framework synthesizing data from 2020-2025 to explain how institutions can align faculty capacity, governance, and quality assurance for AI-supported teaching. Key finding: faculty readiness is a sociotechnical alignment problem, not an individual skills deficit. ## Four EPIQ-AI Domains - **Epistemic Readiness:** Faculty confidence and depth of AI understanding — adoption is widespread but pedagogical clarity remains limited. - **Pedagogical Readiness:** Course design support, AI integration strategies, and [[instructional-design]] alignment. - **Institutional Readiness:** Policy maturity, governance structures, and [[faculty-development]] investment. - **Quality and Compliance:** Assessment integrity and the limits of detection-centred [[academic-integrity]] regimes. ## Key Findings - **Faculty AI adoption is widespread** but depth of use and confidence vary dramatically - **Assessment is the central pressure point** — detection tools are unreliable and insufficient for high-stakes decisions - **Readiness reframed:** Not an individual skills gap but a sociotechnical alignment problem shaped by governance, incentives, workload, and [[ai-literacy]] support ## Implications This framework connects directly to [[institutional-change-framework-ai]] and [[teacher-ai-competency]] research. The finding that detection-centred integrity regimes are unreliable echoes [[plagiarism-detection]] and has implications for [[assessment-validity]] and [[academic-integrity]] policies. ## Connected Concepts - [[faculty-development]] - [[ai-literacy]] - [[higher-ed]] - [[academic-integrity]] - [[teacher-ai-competency]] - [[assessment-validity]] - [[instructional-design]] ## Connected Articles - [[teacher-ai-adoption-confidence]] - [[institutional-change-framework-ai]] - [[ai-assessment-scale-reform]] - [[raza-farooq-aied-review-2020-2025]] ## Citation Sangwa, S., Ndahayo, C., & Dusengumuremyi, F. (2026). [*Faculty Readiness for AI-Supported Teaching and Scalable Online Program Delivery in Higher Education: The EPIQ-AI Framework for Epistemic Integrity*](https://doi.org/10.35542/osf.io/kj4vt_v1). EdArXiv. doi:10.35542/osf.io/kj4vt_v1. --- ## [SAVVY: Student Attention Visualization for Video-based Learning Analysis](https://edtechdev.github.io/aied/articles/savvy-student-attention-video-learning/) > **Shixian Zhou, Minghuan Shen, Xiaolin Wen, Zijun Qiu, Yongliang Jiang, Xiangyang Wu, Fei Wu, Yong Wang, Zhiguang Zhou** — arXiv preprint (2026). ## Synthesis SAVVY is an interactive visual analytics system for video-based learning that integrates visual and auditory attention signals from multimodal brain data to support top-down exploration of student attention variation across instructional videos. A novel attention modeling framework based on multimodal brain signals enables stable tracking of student attention in real-world environments, addressing the noise susceptibility of existing attention quantification algorithms. The system supports teachers in analyzing pilot cohorts' attention before releasing videos, reducing the guesswork of empirical revision by making attention patterns interpretable at scale. The work connects AI-based attention estimation to instructional design practice, giving teachers an evidence base for when and where videos lose student engagement. ## Connected Concepts - [[administrator]] - [[affective-computing]] - [[math-education]] - [[higher-ed]] - [[prompt-engineering]] - [[human-in-the-loop-ai]] - [[motivation]] - [[affective-tutoring]] ## Connected Articles - [[physiological-signals-exam-outcomes-ml]] — Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning - [[wordstream-glass-learning-analytics]] — Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics - [[hypergamification-game-engine-lms]] — Hypergamigication Through Integrating Game Engines and Learning Management Systems: Ender's Game - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[llm-sentiment-analysis-education-research]] — LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments ## Citation Zhou, S., Shen, M., Wen, X., Qiu, Z., Jiang, Y., Wu, X., Wu, F., Wang, Y., & Zhou, Z. (2026). [SAVVY: Student attention visualization for video-based learning analysis](https://arxiv.org/abs/2607.29413). arXiv:2607.29413. --- ## [Scaffolding Critical Engagement with GenAI: Transforming Ethnic Minority Preparatory Students' Collaborative Discourse in Prompt Engineering Tasks](https://edtechdev.github.io/aied/articles/scaffolding-critical-engagement-genai-minority-students/) > **Deliang Wang, Cunling Bian** — AIED 2026 (accepted full paper). **Deliang Wang, Cunling Bian** — AIED 2026 (accepted full paper). ## Synthesis A three-week GenAI course for 78 ethnic minority preparatory students in China integrated a human-in-the-loop workflow and teacher modeling with contrasting cases to move students from passive consumption to critical co-creation with GenAI. Using epistemic network analysis of collaborative discourse, thematic analysis of reflections, and paired-samples t-tests on prompt self-efficacy, the study documents 'strategic repurposing': students initially instrumentalized strategy talk to coordinate efficient copying before shifting toward genuine critical engagement. The design specifically targets the cognitive-laziness risk of GenAI in equity contexts — where broader access could otherwise become an answer-engine shortcut — by scaffolding prompt engineering as a thinking practice. Results indicate pedagogical scaffolding can transform how minoritized students engage GenAI, with implications for prompt self-efficacy and collaborative discourse quality; findings support scaffolded rather than open-ended GenAI use for equity populations. ## Connected Concepts - [[reducing-ai-misuse]] - [[help-seeking]] - [[metacognition]] - [[prompt-engineering]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[student-experience]] - [[equity-in-ai-education]] ## Connected Articles - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI - [[tracing-genai-literacy-interaction-patterns]] — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[epistemic-proactivity-math]] — From Prompting to Epistemic Proactivity: Temporal Trajectories of Student-AI Interaction in Mathematics Learning - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation Wang, D., & Bian, C. (2026). [Scaffolding critical engagement with GenAI: Transforming ethnic minority preparatory students' collaborative discourse in prompt engineering tasks. AIED 2026](https://arxiv.org/abs/2607.28630). arXiv:2607.28630. --- ## [Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement](https://edtechdev.github.io/aied/articles/scheu-mobile-chatbot-journaling-motivation-2026/) ## Summary A **randomized 2×2 full-factorial field experiment** (N = 179 German university students, 22 days of app use, 12-week follow-up) testing two design principles for a **mobile chatbot-based learning journaling system** aimed at keeping students motivated to maintain reflective learning journals — a known pain point (rapid decline in motivation/engagement after brief use). The two principles: (1) an **example-based built-in course** (7 days, one SRL topic per day, time-gated, modeled example responses) and (2) an **LLM-based journaling assistant** (GPT-3.5-turbo-1106) that scaffolds entries by summarizing drafts, asking clarifying follow-up questions, and generating alternative first-person formulations. ## Design & method - **Groups:** Baseline (B, n=53) | Assistant (A, n=53) | Course (C, n=52) | Course+Assistant (CA, n=52); stratified randomization (gender, age, LIST-K SRL scales). - **Measures:** Intrinsic Motivation Inventory (IMI: enjoyment, perceived choice, pressure, competence, effort); LIST-K (SRL: cognition, metacognition, internal/external resource strategies) at pre/post/12-week follow-up; **behavioral engagement = characters written per journal prompt** (7,286 responses; 1,904 entries; mean 10.64 entries/student). - Engagement analyses: multiple regression + mixed-effects robustness check; outliers trimmed (top 1% and single-word messages removed → 5,181 messages, M = 71.87 chars). ## Key findings ### Intrinsic motivation (H1/H2) - **Course → motivation: SUPPORTED.** Small significant effect on enjoyment (η² = 0.03, F(1,153) = 4.81, p < .05) and perceived competence (η² = 0.04, F(1,154) = 5.77, p < .05). No effect on perceived choice or pressure. - **Assistant → motivation: NOT SUPPORTED.** No significant effect on enjoyment (p = .67) or competence (p = .95); no interaction between features. Even usage-days analysis found no competence effect (t = 1.95, p = .054). ### Behavioral engagement (H3/H4) - **Both features SUPPORTED.** Regression on characters written: Assistant B = 18.32***, Course B = 22.95***, with a significant **negative interaction** (Course×Assistant B = −15.21***) — mean message lengths B 51.21 → A 71.25 / C 79.26 / CA 83.04. - **Distinct mechanisms:** the course's effect was *constant* (unrelated to course days completed), while the **assistant formed a positive feedback loop** — more assistant use predicted longer messages over time (B = 6.20 per assistant day, p < .001), echoing the social-cognitive modeling account (Schunk): the assistant's model adapts, the course's static examples do not. ### Course benefit is temporary - Course users were far more likely to be "early" writers (OR = 3.45, p < .001) — less reliant on the 9 PM notification — but **significantly fewer reached 10+ journal days** (OR = 0.38, p < .001): most completed the 7-day course, journaled one more day, then stopped. Static one-off scaffolding stimulates early activity but does not sustain it. ### SRL development - All groups (including baseline) significantly increased cognitive and metacognitive strategy use from pre to 12-week follow-up (p < .05) — the structured prompting concept itself supported SRL, unlike earlier structured-journal studies. - 32 of 97 users reported the auto-generated summaries helped them reflect on prior entries (an unprompted purpose). ## Implications - For reflective-learning-tool design: pair one-off course-style scaffolding with **recurring/adaptive support** (follow-up prompts, phase-specific guidance, timely interventions) to sustain engagement; the LLM assistant is the more promising candidate for durable engagement dynamics, but needs to be used (only 55.9% used it post-onboarding — a self-selection caveat). - Engagement measured as text length only; cognitive engagement/reflection quality untested. Implementation is a single instantiation; seasonal/semester effects possible; long-term effects beyond 3 weeks unverified. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[metacognition]] - [[scaffolding]] - [[self-regulated-learning]] - [[student-experience]] - [[llm]] ## Connected Articles - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple D... - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams ## Citation Scheu, S., Loeffler, S. N., & Maedche, A. (2026). Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement. [*International Journal of Educational Technology in Higher Education*](https://doi.org/10.1186/s41239-026-00589-7), 23, 15 --- ## [School network reorganization under educational and spatial constraints using classical and quantum optimization](https://edtechdev.github.io/aied/articles/school-network-reorganization-optimization/) > **Synthesis:** This paper develops an optimization framework for school network reorganization that integrates geographic, administrative, and educational criteria into an Integer Linear Programming formulation. Applied to the complete public school network of Calabria, Italy, and extended to a hybrid quantum optimization setting, the approach identifies optimal school aggregation plans under different policy scenarios while preserving [[equity]] and accessibility. The framework serves as a decision-support tool for sustainable educational planning in the era of [[ai-education|AI-enhanced operations research]]. ## Optimization Framework The framework models school dimensioning as a constrained optimization problem balancing: - **Demographic trends:** Projected enrollment changes across regions - **Territorial accessibility:** Travel distances and geographic constraints - **Educational requirements:** Class size limits, curriculum coverage, teacher allocation - **Institutional constraints:** Administrative boundaries and policy directives A synthetic benchmark generator enables scalability testing, while the real-world Calabria case study validates practical applicability using actual institutional, territorial, and demographic data. ## Key Findings - **Optimal aggregation plans:** Framework identifies consolidation strategies that maintain educational quality while improving resource efficiency - **Policy scenario analysis:** Model adapts to different policy scenarios, allowing decision-makers to explore trade-offs between cost, accessibility, and educational outcomes - **Quantum compatibility:** Reformulation as a constrained quadratic model demonstrates readiness for emerging quantum-computing technologies - **Sustainable planning:** Robust methodology supports equitable and sustainable [[k-12|school network]] planning ## Educational Planning Implications As school districts worldwide face declining enrollments and budget pressures, AI-powered optimization offers data-driven alternatives to politically-driven consolidation decisions. The framework's multi-criteria approach ensures that educational quality and equity considerations are not sacrificed to purely financial optimization. ## Connected Concepts - [[equity]] - [[ai-education]] - [[k-12]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions - [[ai-uk-higher-education-policy-2026]] — Artificial Intelligence in UK Higher Educational Policy and Institutional Decision Making - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well ## Citation Ciacco, A., Di Puglia Pugliese, L., & Guerriero, F. (2026). [*School network reorganization under educational and spatial constraints using classical and quantum optimization*](https://arxiv.org/abs/2608.05427v1). arXiv:2608.05427v1. --- ## [Exploring interfaces and implications for integrating social-emotional competencies into AI literacy for education: a narrative review](https://edtechdev.github.io/aied/articles/sec-ai-literacy-narrative-review-2026/) > **Synthesis:** Palmquist, Sigurdardottir, and Myhre (2025) conduct a narrative literature review examining the intersection of AI literacy and social-emotional competencies (SEC) in education, proposing an integrated framework to create a supportive, technologically adept, and emotionally intelligent educational ecosystem. Grounded in the SETCOM project, the review identifies three key themes — AI's transformative potential in reshaping educational practices, its influence on educational providers and stakeholders, and the ethical considerations of AI integration — and argues that integrating AI literacy with SEC can enhance educational outcomes by promoting personalized learning, fostering collaboration, and addressing ethical challenges. ## Key Findings - **AI literacy and SEC are complementary:** technical proficiency alone is insufficient for navigating AI-mediated learning environments; combining technological understanding with relational and emotional intelligence supports a holistic approach to skill development. - Three themes emerged from thematic analysis of the literature: **(1) AI's transformative potential** (reshaping educational practices, adapting to student needs, demanding human-centred, ethical design); **(2) AI's influence on educational providers and stakeholders** (changing teachers' roles, functions, and attitudes, and the need for collaboration with external stakeholders); and **(3) ethical considerations of AI integration** (data privacy, algorithmic bias, equitable access, and responsible decision-making). - **Personalisation and relational practices:** AI-driven personalisation (intelligent tutoring, adaptive learning) parallels SEC's focus on teacher-student and student-student relationships; AI should deepen — not replace — human connections in learning environments. - **Educators' roles are shifting:** as AI automates administrative tasks, educators can focus more on guiding students' emotional and cognitive development, where human touch is indispensable (drawing on Selwyn); teacher training should incorporate SEC-infused AI literacy. - **A robust framework should blend AI literacy with SEC**, fostering critical thinking, ethical awareness, empathy, self-management, and responsible decision-making, so that technological advancement supports rather than undermines human connection. ## Study Design & Method This is a **narrative literature review** building on the knowledge base of the SETCOM project (Supportive Environments to Enhance Transversal Competencies in Education). A curated bibliography of 64 sources (31 on AI, 18 on SEC, 13 on both) was compiled from expert recommendations, then filtered through two stages: (1) inclusion/exclusion criteria (peer-reviewed sources in English, AI sources post-2017) and (2) quality assessment, removing 43 sources to leave a data corpus of 19. Sources were categorized into three groups (C1: AI in Education, n=8; C2: SEC in Education, n=7; C3: AI and SEC in Education, n=4). The corpus was thematically synthesized using a three-stage approach (pre-analysis, exploration, treatment/interpretation) informed by Ferrari (2015), with MAXQDA used for coding and analysis. The reviewed literature spanned primary (57.1%), secondary (46.4%), higher (35.7%), adult (17.9%), pre-school (14.3%), special-needs (10.7%), and teacher-training (3.6%) education contexts. ## Implications for AI in Education The review proposes an integrated framework that equips educators and students with both technological and emotional intelligence, arguing that AI literacy should not be reduced to technical skill but integrated with [[social-emotional-learning|social-emotional competencies]] to support [[self-regulated-learning]] and relational practices. It positions educators as needing both technological skill and emotional intelligence to prepare students for AI-rich futures, connecting to [[teacher-ai-competency]], [[teacher-role]], and [[faculty-development|teacher training]]. It emphasizes that AI-driven personalisation can enhance relational dynamics if designed carefully, and that ethical literacy (data privacy, bias, equity, responsible decision-making) must be embedded in AI curricula. The review is relevant to [[ai-literacy]], [[affective-computing]], and [[ethics]], and supports a human-centred approach to AI in education where digital and relational competencies are equally valued. ## Limitations As a narrative review, the study faces potential selection bias and subjective interpretation of findings (acknowledged by the authors), and its small final corpus (19 sources) reflects a curated rather than exhaustive literature base. The AI sources were limited to post-2017 publications, which may not capture the most recent rapid advances, and the review synthesizes concepts as of its curation date. The SEC literature was not subject to the same 2017 cutoff, creating asymmetry. The proposed integrated framework is conceptual and requires empirical validation. ## Connected Concepts - [[ai-literacy]] - [[affective-computing]] - [[teacher-ai-competency]] - [[ethics]] - [[pedagogical-safety]] - [[teacher-role]] - [[social-emotional-learning]] - [[self-regulated-learning]] ## Connected Articles - [[hingle-collaborative-ai-literacy-2025]] — Collaborative AI Literacy - [[ai-tpack-preservice-math-teachers]] — AI-TPACK and Pre-Service Math Teachers - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] — The Scaffolded AI literacy (SAIL) framework - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and Teacher Feedback - [[genai-literacy-training-teacher-education-dbr-2026]] — AI Literacy Training for Teachers ## Citation Palmquist, A., Sigurdardottir, H. D., & Myhre, H. (2025). [*Exploring interfaces and implications for integrating social-emotional competencies into AI literacy for education: A narrative review*](https://doi.org/10.1007/s40692-025-00354-1). *Journal of Computers in Education*, 13, 127–163. --- ## [Self-Efficacy and Favorability Shape Learning from Tutoring Systems and Paper Practice](https://edtechdev.github.io/aied/articles/self-efficacy-tutoring-learning/) > **Xinfei Cen, Vincent Aleven, Kenneth R. Koedinger, Conrad Borchers, Paulo F. Carvalho** (2026). EC-TEL 2026 ## Key Findings - Students with lower baseline self-efficacy achieved greater learning gains regardless of practice format. - Among students with lower baseline self-efficacy, greater favorability toward the tutor was associated with greater learning gains during tutor practice, whereas the pattern differed in paper-based practice. - Intelligent Tutoring System (ITS)-based practice did not significantly improve post-training self-efficacy relative to paper-based methods. - The benefits of tutor- and paper-based practice varied with baseline self-efficacy and favorability, so no single format dominated for all learners. - The findings underscore the potential value of tailoring practice format to students' motivational profiles. ## Study Design & Method The study examined how self-efficacy and favorability toward practice shape learning across two common practice formats: paper-based practice and system-based tutoring practice. Using a counterbalanced within-subject design with matched problem sets, the authors isolated the effect of practice format while modeling motivational differences. The work builds on the observation that technology-enhanced learning systems in domains such as mathematics and science can improve learning outcomes through scaffolded support, step-level feedback, and individualized practice, but that learners' motivational profiles may influence how they respond to one format versus another. ## Relevance to AI in Education This paper contributes directly to understanding how AI systems interact with learners in authentic educational settings. It demonstrates that motivational factors — self-efficacy and favorability — modulate the effectiveness of ITS-based versus paper-based practice, suggesting that the value of an [[intelligent-tutoring]] system is not uniform across learners. For [[personalized-learning]] design, the results imply that practice format assignment should consider the learner's motivational profile: students who feel favorably toward the tutor and enter with lower self-efficacy may benefit more from system-based practice, while the same assignment could be less beneficial for others. The finding that ITS practice did not improve post-training [[self-regulated-learning]]-relevant beliefs such as self-efficacy relative to paper is a caution for the field's implicit assumption that richer technology automatically enhances motivation. The study lays groundwork for aligning instructional formats with learners' [[motivation]]al needs rather than treating practice format as a neutral delivery channel. ## Connected Concepts - [[self-regulated-learning]] - [[personalized-learning]] - [[motivation]] - [[pedagogical-agent]] - [[affective-computing]] - [[intelligent-tutoring]] - [[math-education]] - [[affective-tutoring]] ## Connected Articles - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents - [[ecnuclaw-k12-personalized-companion]] — ECNUClaw: A Learner-Profiled Intelligent Study Companion Framework for K-12 Personalized Education - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[metacognitive-awareness-experiential-vs-instructional]] — Experiential Versus Instructional Approaches for Eliciting Metacognitive Awareness in AI-Assisted Learning - [[scheu-mobile-chatbot-journaling-motivation-2026]] — Designing a mobile chatbot-based learning journaling system for intrinsic motivation and engagement ## Citation Xinfei Cen, Vincent Aleven, Kenneth R. Koedinger, Conrad Borchers, Paulo F. Carvalho (2026). [Self-Efficacy and Favorability Shape Learning from Tutoring Systems and Paper Practice](https://arxiv.org/abs/2606.17470). arXiv:2606.17470. EC-TEL 2026. --- ## [Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs](https://edtechdev.github.io/aied/articles/self-referential-l2-writing-llm-assessment/) > Bannò, Knill & Gales (2026) propose a paradigm shift in automated essay scoring: from **inter-learner ranking** to **intra-learner profiling**. Instead of asking "how does this essay rank against others?", their self-referential framework asks "what are this specific learner's strengths and weaknesses?" # Self-Referential L2 Writing Assessment with LLMs ## Core Contribution Bannò, Knill & Gales (2026) propose a paradigm shift in automated essay scoring: from **inter-learner ranking** to **intra-learner profiling**. Instead of asking "how does this essay rank against others?", their self-referential framework asks "what are this specific learner's strengths and weaknesses?" ## Key Findings Using the ICNALE GRA dataset annotated by up to 80 trained raters and calibrated with two-facet Rasch modeling: - **LLMs outperform single human raters** at identifying relative weaknesses (negative feedback) across proficiency aspects - **Human raters remain stronger** at identifying relative strengths (positive feedback) - Traditional rank-based correlation metrics **mask diagnostic behavior** — high correlations can hide poor intra-learner discrimination ## Implications for AIED This connects to [[automated-grading]] but challenges its dominant evaluation paradigm. The finding that LLMs are strong at weakness detection but weaker at strength identification has practical implications for [[formative-assessment]] design — AI might best serve as a **complementary weakness detector** while teachers focus on strengths. The self-referential approach aligns with [[personalized-learning]] goals and the [[ai-learning-companions-framework]] emphasis on prioritizing learning over performance. It extends [[writing-education]] research on AI in composition and connects to [[automated-question-generation]] work on AI-generated assessment. The use of Rasch modeling for calibration connects to [[ground-truth-reliability-aied]] calls for more rigorous measurement in AIED. ## Connections to Wiki - Paradigm shift from [[automated-grading]] ranking to profiling - Aligns with [[sequenced-ai-feedback-learning]] emphasis on feedback quality over quantity - Extends [[llm-student-modeling-memory]] to assessment contexts — profiling over time - Complements [[human-in-the-loop-ai]] by identifying where humans vs. AI add value ## Connected Concepts - [[automated-grading]] - [[formative-assessment]] - [[personalized-learning]] - [[writing-education]] - [[automated-question-generation]] - [[human-in-the-loop-ai]] ## Connected Articles - [[ai-learning-companions-framework]] - [[ground-truth-reliability-aied]] - [[sequenced-ai-feedback-learning]] - [[llm-student-modeling-memory]] - [[icle-plus-plus-essay-scoring]] ## Citation Gales, A.S.B.K.K.M., Approach, T.S.A.A.A.P., LLMs, T.L.W.E.W., Gales, S.B.K.K.M., prac-, A.I.R.W.C.D.T.A.C.E., & (PCC), G.A.U.D.S.W.S.A.P.C.C. (2026). [*Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs*](https://arxiv.org/abs/2605.04298) --- ## [Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning](https://edtechdev.github.io/aied/articles/sequenced-ai-feedback-learning/) # Sequenced AI Feedback on Student Learning ## Core Finding **Sequenced AI feedback harms learning despite boosting engagement and positive perceptions.** In a randomized experiment with 199 participants, the authors compared two types of AI-generated feedback: - **Sequenced (layered)**: Encouragement → hints → correct answer, designed to promote learner autonomy - **Non-sequenced (direct)**: Full-solution feedback immediately Contrary to design intuition, **sequenced feedback led to significantly poorer learning performance**. The finding reveals a critical disconnect between what students *like* and what actually *helps them learn*. ## Mediation Pathways Three causal pathways were tested via mediation analysis: | Pathway | Mediator | Effect | Significant? | |---------|----------|--------|:---:| | Affective | Perceived encouragement | Positive → better learning | ✓ | | Behavioral | Tasks needing ≥3 submissions | Negative → worse learning | ✓ | | Cognitive | Mental effort | Neutral | ✗ | The **positive affective pathway** (students felt more encouraged) was **completely counteracted** by the **negative behavioral pathway** (students made more resubmissions). The net effect was significantly poorer learning outcomes. ## Key Mechanisms ### Why Sequenced Feedback Backfired - The hint-before-answer structure inadvertently encouraged **trial-and-error behavior** rather than deep processing - Students submitted more attempts per task, indicating they were "gaming" the hint system rather than engaging in genuine problem-solving - Higher mental effort was reported but did not translate to better learning — suggesting the effort was directed at navigating the feedback sequence rather than understanding the material ### Why Direct Feedback Worked - Immediate corrective information eliminated the temptation to guess - Students processed the solution rather than iterating through hints - Lower engagement scores but higher learning outcomes ## Design Implications This study challenges the prevailing intuition that more scaffolded, autonomy-supportive feedback is always better. Key takeaways for AI feedback system design: 1. **Engagement ≠ learning**: User satisfaction and behavioral engagement are not reliable proxies for learning gains — designers must measure learning outcomes directly 2. **Limit resubmission loops**: Systems should cap hint requests or require reflection between attempts to prevent trial-and-error gaming 3. **Strategic blending**: Consider providing direct corrective feedback first, with optional encouragement and hints available on demand rather than as a mandatory sequence 4. **Cognitive load management**: The higher mental effort induced by sequenced feedback did not aid learning — design should channel effort toward understanding rather than navigation ## Connection to Existing Wiki This paper directly informs several threads in the wiki: - **[[formative-assessment]]**: Direct evidence about AI-generated feedback design — sequencing that feels supportive may undermine formative goals - **[[critical-thinking-genai-scaffolding]]**: Vendrell & Johnston's eight design principles for LLM scaffolding — this study provides empirical evidence that poorly designed scaffolding can harm learning, reinforcing the need for "cognitive friction" design - **[[prober-ai-inquiry-writing]]**: The inverted paradigm (AI asks questions, gates suggestions) offers an alternative to sequenced feedback that may avoid the resubmission trap - **[[self-regulated-learning]]**: Sequenced feedback was intended to promote autonomy and SRL, but the behavioral data shows it had the opposite effect — a cautionary tale for SRL-aligned AI design - **[[metacognition]]**: The engagement-learning disconnect exemplifies the metacognitive calibration problem — students felt they were learning more with sequenced feedback when they were actually learning less - **[[ai-peer-feedback-systems]]**: Multi-LLM collaborative feedback systems must consider feedback sequencing carefully to avoid the pitfalls identified here - **[[pedagogy-ai-mistakes]]**: Hosseini's work on deliberately leveraging AI errors connects to the finding that easy, encouraging feedback may be less pedagogically effective than direct correction - **[[transfer-of-learning]]**: The learning outcome disparity between conditions raises transfer implications — do sequenced-feedback students retain less when the scaffolding is removed? ## Methodological Strengths - **Randomized controlled design** with 199 participants — causal claims are well-supported - **Mediation analysis** identifies *why* the effect occurs, not just *whether* it occurs - **Multi-dimensional measurement**: learning performance, behavioral engagement (submission patterns), cognitive engagement (mental effort), and affective perceptions - Multi-institution collaboration (UNC, CMU, Pitt, HKU) ## Open Questions - Would results differ with longer exposure (multi-session vs. single-session study)? - Does domain matter — would sequenced feedback work better for ill-defined problems than well-defined ones? - Can the resubmission problem be solved by requiring reflection prompts between hint levels? - Would a hybrid design (direct feedback + optional hints) preserve learning while maintaining positive affect? ## Connected Concepts - [[formative-assessment]] - [[self-regulated-learning]] - [[metacognition]] ## Connected Articles - [[critical-thinking-genai-scaffolding]] - [[prober-ai-inquiry-writing]] - [[ai-peer-feedback-systems]] - [[pedagogy-ai-mistakes]] - [[transfer-of-learning]] ## Citation Cao, J., Zhao, C. Q., Schunn, C., McLaughlin, E. A., Lin, J., & Koedinger, K. R. (2026). [Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning](https://arxiv.org/abs/2604.07469). arXiv:2604.07469. --- ## [Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education](https://edtechdev.github.io/aied/articles/shame-guilt-ai-regulation-computing-education/) > An interview study with 19 computing students through a functionalist perspective of shame and guilt. Findings show these emotions regulate when and how students make their AI use visible, engaging in hiding behaviors and selective disclosure. Students described shaming themselves, peers, and faculty for using AI. Shame and guilt coexist with continued AI use, creating cycles of reduced agency and moral tension rather than promoting behavior change. Students used language and metaphors of addiction. Highlights need to consider socio-emotional aspects of AI use in policy and classroom practice. ## Key Findings - Shame and guilt regulate when and how students make their AI use visible, producing hiding behaviors and selective disclosure. - Students described shaming themselves, peers, and faculty for using AI, indicating that these emotions circulate socially rather than only privately. - Shame and guilt coexist with continued AI use, creating cycles of reduced agency and moral tension rather than promoting behavior change. - Students used language and metaphors of addiction to describe their relationships with AI use. - These experiences emerged alongside broader feelings of nihilism and pessimism about their academic futures. ## Study Design & Method The study interviewed 19 computing students and analyzed their relationships with AI use through a functionalist lens of shame and guilt — that is, asking what social work these emotions do in regulating behavior. The analysis examined how students' emotional experiences relate to academic identity, confidence, and self-perception, and how those experiences intersect with institutional responses such as surveillance and detection. ## Implications for Policy and Classroom Practice The findings suggest that prevailing responses to AI use — particularly those emphasizing surveillance and detection — risk overlooking or exacerbating the socio-emotional costs of AI use. When shame drives students to hide their AI use rather than change it, detection-focused policy can deepen the spiral: reduced agency, moral tension, and addiction-like language describe a dynamic in which students continue using AI but cannot openly negotiate how to do so well. Left unaddressed, these dynamics may undermine students' learning, identities, and long-term engagement in computing fields. For instructors and institutions, the study highlights the need to consider the socio-emotional aspects of AI use alongside [[academic-integrity]] enforcement — creating classroom conditions in which AI use can be discussed candidly, so that policy addresses behavior rather than driving it underground. The work also connects to [[over-reliance]] and [[student-experience]] research by showing that the emotional valence of AI use is itself consequential for learning and persistence in [[cs-education]]. ## Connected Concepts - [[student-experience]] - [[cs-education]] - [[affective-computing]] - [[plagiarism-detection]] - [[over-reliance]] - [[academic-integrity]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] ## Connected Articles - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students' Motivation in Introductory Programming - [[ai-partner-science-epistemic-vigilance]] — AI as a Partner in Learning about, Doing, and Engaging with Science: Vigilance as the Key to Productive Augmentation - [[student-rationalization-ai-writing]] — It''s OK Because...": The Wild West of Student Rationalization of AI Use in Academic Writing - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) ## Citation Hamilton, K., Hou, I., Patel, D., Nnam, S., Patel, H., & MacNeil, S. (2026). [*"Stuck in a Spiral": Shame and Guilt as Social Regulators of AI Use in Computing Education*](https://arxiv.org/abs/2606.14920). arXiv:2606.14920. --- ## [Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation](https://edtechdev.github.io/aied/articles/short-answer-scoring-quality-degradation/) > Schleifer, Ariely & Klebanov (2026) investigate a critical gap in [[automated-grading]]: **how scoring quality degrades for mid-range student responses**. Most ASAS evaluations focus on clearly correct or incorrect answers, but real classrooms are dominated by partially correct responses where scoring is most challenging. # Automated Short Answer Scoring: Mid-Range Quality Degradation ## Core Contribution Schleifer, Ariely & Klebanov (2026) investigate a critical gap in [[automated-grading]]: **how scoring quality degrades for mid-range student responses**. Most ASAS evaluations focus on clearly correct or incorrect answers, but real classrooms are dominated by partially correct responses where scoring is most challenging. ## Key Findings The paper reveals that automated short answer scoring (ASAS) systems show significant **quality degradation in the mid-range** — exactly where teacher judgment is most needed. This connects directly to [[automatic-short-answer-grading]] research on confidence-aware LLM grading with epistemic uncertainty quantification. The finding that task-specific adaptation can mitigate this degradation provides a practical path forward. ## Significance for AIED This work fills a gap in the [[ai-tutor-behavioral-evaluation]] landscape: Niousha et al.'s 10K-student analysis identified missing evaluation axes for AI tutoring, and mid-range scoring reliability is one such axis. The quality-conditioned agreement approach offers a more nuanced alternative to simple accuracy metrics used in benchmark evaluations. The findings also matter for [[formative-assessment]] systems — if ASAS works well only at extremes, it may reinforce binary thinking rather than supporting the nuanced feedback that [[sequenced-ai-feedback-learning]] research shows is critical for learning. The connection to [[human-in-the-loop-ai]] is clear: mid-range responses may be where human teacher judgment remains essential. ## Connections to Wiki - Extends [[automated-grading]] with quality-conditioned analysis - Complements [[automatic-short-answer-grading]] on confidence estimation - Relevant to [[ground-truth-reliability-aied]] concerns about scoring validity - Connects to [[generate-then-validate-question-gen]] methodologies for AI assessment quality ## Connected Concepts - [[automated-grading]] - [[formative-assessment]] - [[human-in-the-loop-ai]] ## Connected Articles - [[automatic-short-answer-grading]] - [[ai-tutor-behavioral-evaluation]] - [[sequenced-ai-feedback-learning]] - [[ground-truth-reliability-aied]] - [[generate-then-validate-question-gen]] ## Citation Klebanov, A.A.V.G.S.M.A.B.B., Scoring:, Q.A.I.A.S.A., Adaptation, M.D.A.T.I.O.T., Klebanov2, A.V.G.S.M.A.B.B., Alexandron1, A.S.G., & Princeton, E. (2026). [*Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation*](https://arxiv.org/abs/2605.07647). par- require ample training data (Gurin Schleifer et al --- ## [Simulating Learners' Task-Selection Strategies and System Constraints in Mastery Learning](https://edtechdev.github.io/aied/articles/simulating-learner-task-selection/) Intelligent Tutoring Systems often grant learners shared control over skill and problem selection. We propose a simulation-based framework to examine how learner task-selection strategies and system constraints shape mastery learning efficiency. This paper introduces a simulation-based framework for evaluating how learner task-selection strategies interact with mastery learning algorithms in Intelligent Tutoring Systems. Using interaction data from 261 students across two mathematical domains (equation solving and graph interpretation), the authors simulate strategies including Weakness Targeting and Interleaving, measuring overpractice as an efficiency metric. Risk-averse strategies produced substantially higher overpractice, especially for complex multi-step problems. Critically, targeted system constraints significantly reduced inefficiencies for maladaptive strategies while minimally affecting already efficient strategies. This provides a low-cost methodology for pre-testing algorithmic constraints before classroom deployment. The findings extend [[engagement-forecasting-its]] by exploring the behavioral mechanisms behind engagement patterns, and complement [[adaptive-learning]] research by quantifying how learner autonomy interacts with system optimization. The simulation methodology aligns with [[neural-symbolic-knowledge-tracing]] approaches that model learner behavior patterns, while the focus on mastery learning efficiency connects to [[genai-tutor-engagement-patterns]] research on how different student behaviors shape tutoring outcomes. The finding that constraints can correct maladaptive behaviors without penalizing efficient learners has implications for [[llm-student-modeling-memory]] architectures that could incorporate task-selection preferences into student models. ## Connected Concepts - [[adaptive-learning]] ## Connected Articles - [[engagement-forecasting-its]] - [[neural-symbolic-knowledge-tracing]] - [[genai-tutor-engagement-patterns]] - [[llm-student-modeling-memory]] ## Citation Haley Noh, Aarna Chowdhary, Jeroen Ooge, Vincent Aleven, Conrad Borchers (2026). [Simulating Learners' Task-Selection Strategies and System Constraints in Mastery Learning](https://arxiv.org/abs/2605.21613). arXiv:2605.21613. EDM 2026 (short paper). --- ## [Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents](https://edtechdev.github.io/aied/articles/simulating-students-diverse-cognitive-levels-2025/) > Wu et al. (2025, ACL) tackle the core challenge of [[simulating-students]]: LLMs trained as "helpful assistants" produce overly perfect answers and fail to model the natural imperfections and varied cognitive levels of real learners. They propose a training-free framework that builds a cognitive prototype of each student from a knowledge graph, predicts performance on new tasks, and iteratively refines simulated solutions via beam search to reproduce realistic mistakes — achieving a 100% improvement in simulation accuracy on a new Student_100 dataset. A central obstacle to valid student simulation is that LLMs are optimized to generate correct, polished responses. Real students, however, vary in cognitive level and make characteristic mistakes. Simulated students that answer too well — or too randomly — are not faithful models of learners. This paper addresses that gap directly. ## The framework - **Cognitive prototype construction:** for each student, a knowledge graph captures their understanding of concepts from past learning records, representing the student's current epistemic state. - **Performance prediction:** the prototype is mapped to new tasks to predict how that student would perform. - **Realistic mistake generation:** simulated solutions are produced from these predictions, then iteratively refined with a **beam search** method designed to reproduce the kinds of errors a real student at that level would make. ## Findings - The method is **training-free** — it does not require fine-tuning a model on simulated student data, making it broadly applicable. - On the new **Student_100** dataset (100 students, Python programming, 5,000 learning records), the framework **consistently outperformed baselines, achieving a 100% improvement in simulation accuracy** in matching real student behavior. ## Implications The work underscores that **fidelity requires modeling imperfection, not just fluency**: a simulated student is only useful if it errs the way the modeled learner would err. By grounding simulation in a cognitive prototype (knowledge graph) and explicitly generating realistic mistakes, it points toward [[simulating-students]] as a reliable instrument for testing tutoring systems and evaluating pedagogy. It connects to [[student-modeling]] and [[knowledge-graph]] approaches and complements work on the validity of LLM-based student simulation. ## Connected Concepts - [[simulating-students]] - [[student-modeling]] - [[knowledge-graph]] - [[knowledge-tracing]] - [[cognitive-diagnosis]] - [[generative-ai]] - [[llm]] - [[agentic-ai]] - [[adaptive-learning]] ## Connected Articles - [[valid-student-simulation-llm-2026]] — Towards Valid Student Simulation - [[simulating-students-llm-review-2026]] — Simulating Students with LLMs: A Review - [[agentschool-multi-agent-simulation-education-2026]] — AgentSchool: Multi-Agent Simulation for Education - [[history-aware-student-simulation]] — History-Aware Profiles for Student Simulation - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? - [[simulating-students-java-programming-errors-llms]] — Simulating Students' Java Programming Errors - [[llm-student-simulation-teacher-insights]] — Can LLMs Simulate Human Learners? ## Citation Wu, T., Chen, J., Lin, W., Li, M., Zhu, Y., Li, A., Kuang, K., & Wu, F. (2025). [*Embracing imperfection: Simulating students with diverse cognitive levels using LLM-based agents*](https://arxiv.org/abs/2505.19997). In *Proceedings of ACL 2025*. arXiv:2505.19997. --- ## [Simulating Students' Java Programming Errors with Large Language Models](https://edtechdev.github.io/aied/articles/simulating-students-java-programming-errors-llms/) This paper investigates whether [[llm|large language models]] can serve as scalable proxies for students by simulating realistic logical errors in code submissions. Using the CodeWorkout dataset of 74,000+ unique student Java submissions across 37 problems, the authors evaluate five LLMs under three prompting strategies: Input-Output (IO), Chain-of-Thought (CoT), and iterative Self-Refine. - **Diversity vs. Alignment trade-off**: LLMs generate diverse error patterns, but alignment with authentic student errors varies significantly by model. Claude Sonnet 4 achieves the most balanced performance across both dimensions. - **Functional indistinguishability**: A blinded expert annotation study (N=401) found that synthetic errors are functionally indistinguishable from authentic student errors. - **Task difficulty effects**: Higher-struggling-level problems elicit more diverse but less student-like errors — LLMs struggle more to simulate realistic mistakes on harder tasks. - **Practical implications**: Synthetic errors could be integrated into [[intelligent-tutoring|intelligent tutoring systems]], teachable agents, and large-scale [[learning-analytics|learning analytics]] pipelines without waiting for authentic classroom data accumulation. ## Methodology The study used the CodeWorkout dataset with 74,000+ unique student Java submissions. Five LLMs were tested under three prompting strategies. Performance was assessed on two dimensions: diversity (range of distinct error patterns) and alignment (correspondence with authentic student mistakes). A blinded expert annotation study with 401 samples confirmed the indistinguishability of synthetic and authentic errors. ## Related Work This work extends research on [[llm-student-simulation-misconception-faithfulness|LLM-based student simulation]] and [[llm-student-misconception-identification|student misconception identification]]. It connects to [[programming-its|programming intelligent tutoring systems]] and [[student-modeling|student modeling]] by offering a scalable method for generating training and evaluation data. The findings also inform [[ai-generated-traces-novice-programmers|AI-generated traces from novice programmers]] and research on [[code-review-genai-cs1|code review with generative AI in CS1]]. ## Connected Concepts - [[llm]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[student-modeling]] ## Connected Articles - [[llm-student-simulation-misconception-faithfulness]] - [[llm-student-misconception-identification]] - [[programming-its]] - [[ai-generated-traces-novice-programmers]] - [[code-review-genai-cs1]] ## Citation Keramati, A., Cao, J., Mohammadi, I., Warschauer, M., & Shi, Y. (2026). [Simulating Students' Java Programming Errors with Large Language Models](https://arxiv.org/abs/2606.14113). arXiv:2606.14113. --- ## [Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI](https://edtechdev.github.io/aied/articles/simulating-students-llm-review-2026/) > Marquez-Carpintero, Lopez-Sellers & Cazorla (2025) present a thematic review of empirical and methodological studies using LLMs to [[simulating-students|simulate student behavior]] in education. They synthesize evidence on how LLM-based agents emulate learner archetypes, respond to instructional inputs, and interact in multi-agent classroom scenarios, and examine implications for curriculum development, instructional evaluation, and teacher training — while flagging persistent concerns around algorithmic bias, evaluation reliability, and alignment with educational objectives. The review frames simulated students as a valuable methodological tool for evaluating pedagogy and modeling diverse learner profiles — tasks that are hard to undertake systematically with real learners. LLM integration is highlighted as a particularly versatile and scalable paradigm because it affords linguistic realism and behavioral adaptability. ## Scope and synthesis - **Emulating learner archetypes:** LLM agents can approximate a range of learning styles, cognitive development pathways, and social behaviors, capturing diversity that is difficult to assemble in a real cohort. - **Responding to instruction:** simulated students engage in contextually appropriate pedagogical dialogues, enabling the testing of instructional inputs. - **Multi-agent classrooms:** agents interact within simulated classroom scenarios, extending student simulation to social dynamics. ## Applications The review examines implications for **curriculum development**, **instructional evaluation**, and **teacher training** — using simulated learners to practice and assess instruction without real students. ## Concerns and gaps - **Algorithmic bias:** simulated student populations may encode or amplify bias. - **Evaluation reliability:** how well a simulated student models a real learner is itself hard to validate. - **Alignment with educational objectives:** simulations must serve pedagogical goals, not just reproduce plausible dialogue. The review identifies technological and methodological gaps and proposes directions for integrating generative AI into [[adaptive-learning]] systems and [[instructional-design]]. ## Connected Concepts - [[simulating-students]] - [[student-modeling]] - [[generative-ai]] - [[llm]] - [[agentic-ai]] - [[instructional-design]] - [[teacher-role]] - [[adaptive-learning]] ## Connected Articles - [[valid-student-simulation-llm-2026]] — Towards Valid Student Simulation - [[simulating-students-diverse-cognitive-levels-2025]] — Embracing Imperfection: Simulating Diverse Cognitive Levels - [[agentschool-multi-agent-simulation-education-2026]] — AgentSchool: Multi-Agent Simulation for Education - [[history-aware-student-simulation]] — History-Aware Profiles for Student Simulation - [[llm-student-simulation-teacher-insights]] — Can LLMs Simulate Human Learners? - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? ## Citation Marquez-Carpintero, L., Lopez-Sellers, A., & Cazorla, M. (2025). [*Simulating students with large language models: A review of architecture, mechanisms, and role modelling in education with generative AI*](https://arxiv.org/abs/2511.06078). *Computer Science Review*, 62, 101008. arXiv:2511.06078. --- ## [EduQwen: Pedagogical RL](https://edtechdev.github.io/aied/articles/singh-eduqwen-pedagogical-rl-2026/) > **EduQwen: Pedagogical RL** — A multi-stage optimization strategy combining reinforcement learning (DAPO) and supervised fine-tuning (SFT) to enhance the pedagogical knowledge of open-source LLMs, producing a family of dense 32B-parameter models that achieve state-of-the-art performance on the Cross-Domain Pedagogical Knowledge (CDPK) Benchmark, surpassing even much larger proprietary systems such as Gemini-3 Pro. Demonstrates that domain-specialized optimization can transform mid-sized open-source LLMs into true pedagogical domain experts, prioritizing guided learning over answer-giving. ## Key Findings The EduQwen project addresses a fundamental misalignment in LLM behavior for education: general-purpose models are optimized for immediate helpfulness — providing answers directly — while effective pedagogy requires guiding learners to discover answers themselves. This gap, labeled the [[correct-answer-trap-ai-tutor]], drives the core research question. **Three-stage optimization pipeline.** The team used a dense Qwen3-32B backbone (chosen over MoE architectures for superior responsiveness to iterative optimization) and applied: 1. **Stage 1 — RL with DAPO:** Decoupled Advantage Policy Optimization was selected over GRPO for its stable gradients on complex pedagogical reasoning tasks, using asymmetric clipping to prevent catastrophic divergence. Hard-negative mining identified 440 questions the base model could not answer perfectly across 30 attempts, then sorted them by error frequency into a difficulty-ordered curriculum. Extended rollouts (5→8 steps) enabled multi-step pedagogical decision-making. Result: **94.13% on CDPK**, already SOTA. 2. **Stage 2 — Synthetic SFT:** The RL1 model generated 40,000 synthetic responses; only correct responses with gradient-based selection were retained, yielding 1,050 high-quality difficulty-ordered data points. Difficulty-weighted sampling kept all hard examples while sampling easy ones sparsely. Result: **96.20%**. 3. **Stage 3 — Final RL (RL2):** A second DAPO round on the SFT checkpoint reused the original hard-negative dataset, allowing the further refined model to tackle originally challenging problems. Result: **96.52%** — definitive SOTA. **Benchmark dominance.** EduQwen 32B-SFT-RL2 established new SOTA results across the Interactive Pedagogy Benchmark Leaderboard, surpassing Gemini-3 Pro (90.55%) — a system that is orders of magnitude larger. This proves that dense, mid-sized open-source models can become pedagogical domain experts through specialized optimization. ## Implications This work carries significant implications for the [[educational-llm-alignment]] and [[pedagogical-safety]] landscape. First, it demonstrates that [[reinforcement-learning]] approaches — particularly DAPO with carefully constructed reward models that prioritize guidance over answer-giving — can effectively reshape LLM behavior for educational contexts. The synthetic SFT stage highlights how high-quality, difficult-example-focused data can efficiently transfer pedagogical capability without massive datasets. Second, the success of [[open-source]] 32B models over proprietary giants has practical consequences for [[edtech-platform]] deployment: schools and institutions can run domain-specialized pedagogical models locally, preserving privacy and reducing costs while maintaining state-of-the-art quality. This aligns with broader movements toward [[responsible-assessment-ai-era-stanford-2026]] and transparent educational AI. Third, the hard-negative mining methodology offers a template for [[pedagogical-llm-training]] more broadly — rather than training on all data indiscriminately, identifying and targeting specific failure modes of the base model creates more efficient optimization pathways. Finally, the work establishes that [[pedagogical-safety-rl]] is not merely about harm prevention but about proactive pedagogical quality: a model that resists the urge to give answers and instead guides, questions, and scaffolds represents a meaningful step toward [[intelligent-tutoring]] that genuinely teach rather than simply inform. ## Connected Concepts - [[open-source]] - [[pedagogical-llm-training]] - [[pedagogical-safety]] - [[llm]] - [[rag]] - [[reinforcement-learning]] ## Connected Articles - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[educational-llm-alignment]] — Educational LLM Alignment - [[pedagogical-safety-rl]] — Pedagogical Safety in Educational Reinforcement Learning - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[ai-in-the-wild-college]] — AI in the Wild: A Large Scale Analysis of Authentic Interactions of College Students with Generative AI - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance - [[transfer-of-learning]] — AI Learning Transfer - [[ai-literacy-legal-translation-2026]] — AI Literacy for Legal Translation: Developing Digital Resilience ## Citation Singh, N. P., Wang, X., Garikipati, A., Ciobanu, M., Mao, Q., & Das, R. (2026). [*Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via RL and SFT*](https://arxiv.org/abs/2604.06385). arXiv:2604.06385. --- ## [Estimating Learners' Skill Acquisition Without Temporal Information](https://edtechdev.github.io/aied/articles/skill-acquisition-without-temporal-info/) Nagai et al. (2026) tackle the practical problem that many real-world educational datasets contain only single-time-point assessments (snapshots) without temporal information, making standard time-series knowledge tracing approaches inapplicable. They propose a novel framework that uses inclusion relations among learners' skill sets — interpreting expanding skill sets as a proxy for learning progression — to induce a pseudo-temporal ordering from snapshot data. A neural model captures latent skill acquisition dynamics through expected skill increments. Experiments on both synthetic and real-world datasets show consistent outperformance over baselines, with particularly strong advantages as the skill space grows. This work bridges [[student-modeling]] and [[knowledge-tracing]] for data-constrained environments, enabling [[adaptive-learning]] support and [[personalized-learning]] in settings where longitudinal data is unavailable — a significant practical advance for [[learning-analytics]] and [[formative-assessment]] in low-resource contexts. ## Connected Concepts - [[student-modeling]] - [[knowledge-tracing]] - [[adaptive-learning]] - [[personalized-learning]] - [[learning-analytics]] - [[formative-assessment]] ## Connected Articles - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[llm-item-difficulty-prediction]] — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[proprl-prerequisite-relation-learning]] — ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs - [[cross-subject-validity-delayed-start]] — Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach ## Citation Ryosuke Nagai, Kyohei Atarashi, Koh Takeuchi, Jill-Jênn Vie, Hisashi Kashima (2026). [Estimating Learners' Skill Acquisition Without Temporal Information](https://arxiv.org/abs/2606.20611). arXiv:2606.20611. AIED 2026, Seoul --- ## [Navigating the skill diversity frontier: How skill complexity explains worker resilience](https://edtechdev.github.io/aied/articles/skill-diversity-worker-resilience/) > **Synthesis:** Using LinkedIn data on 2.4 million U.S. workers and 16,753 distinct skills, this paper introduces three complementary measures of skill complexity — specialization, diversity, and the diversity frontier — and demonstrates that workers near the frontier are significantly more likely to acquire new skills, receive promotions, and transition into occupations with lower automation exposure. The findings distinguish productive capital (specialization) from adaptive capital (diversity) and provide a data-driven framework for understanding [[professional-training|workforce resilience]] in the era of [[ai-education|AI-driven labor market transformation]]. ## Framework: Three Dimensions of Skill Complexity The authors reconstruct skill hierarchies directly from observed co-occurrence patterns: | Dimension | Definition | Career Outcome | |-----------|-----------|----------------| | **Specialization** | Productive depth in a domain | Higher-wage occupations | | **Diversity** | Adaptive breadth across domains | Skill accumulation, occupational mobility | | **Diversity Frontier** | Maximum attainable diversity at a given specialization level | Promotions, new skill acquisition, automation-resilient transitions | ## Key Findings - **Specialization → wages:** Most strongly associated with sorting into higher-wage occupations - **Diversity → mobility:** Associated with broader skill accumulation and occupational mobility - **Frontier proximity → resilience:** Workers closest to the diversity frontier are significantly more likely to: - Acquire new skills - Receive promotions - Transition into occupations with lower automation exposure - **Complementarity matters:** Adaptive capacity depends on combining specialized expertise with broad capabilities, not possessing either alone ## Implications for Education and Training This framework has direct implications for [[ai-education|educational policy]] and [[faculty-development|workforce training]]: - **Curriculum design:** Educational programs should cultivate both depth (specialization) and breadth (diversity) rather than treating them as competing goals - **Reskilling pathways:** The diversity frontier identifies concrete reskilling targets — skills that maximize workers' adaptive capacity - **[[professional-training]] strategy:** Training interventions should target frontier-proximate skill combinations that boost resilience to automation - **[[ai-literacy]] as adaptive capital:** AI literacy itself may function as a frontier-expanding meta-skill, enabling workers to integrate AI tools into diverse domains ## Connected Concepts - [[professional-training]] - [[ai-education]] - [[faculty-development]] - [[professional-training]] - [[ai-literacy]] ## Connected Articles - [[generative-ai-education-productivity-gaps]] — Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment - [[crewscaler-ai-upskilling-framework]] — AI-accelerated End-to-End Framework for Rapid Professional Upskilling - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[egai-power-systems-education]] — Bridging Artificial Intelligence and Power Systems Education Using a Hands-On Executable Framework - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges ## Citation Carpanelli, M., Duszynski, J., & Stephany, F. (2026). [*Navigating the skill diversity frontier: How skill complexity explains worker resilience*](https://arxiv.org/abs/2608.02102v1). arXiv:2608.02102v1. --- ## [Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation](https://edtechdev.github.io/aied/articles/slidesqaqa-pedagogical-question-generation/) ## Overview SlidesQAQA is a Flask-based system that extracts text and rendered images from PDF lecture slides and processes them through a four-stage [[llm]] pipeline: **window planning** (segment extraction), **deck synthesis** (cross-slide reasoning), **slide annotation** (per-slide question generation), and **reconciliation** (deck-level revision to reduce redundancy and improve coverage). The key innovation is joint reasoning about slide modality and pedagogical role, with a bounded question budget that forces prioritization of important content. ## How It Works Unlike earlier [[automated-question-generation]] systems that generate questions slide-by-slide in isolation, SlidesQAQA reasons across the entire presentation. This enables **deck-level scaffolding** — questions build on each other across the slide sequence, matching the intended instructional flow. The reconciliation stage filters non-instructional slides and revises draft annotations to eliminate redundancy, producing structured JSON output with deck-level goals, section structure, slide summaries, question sets, and evaluation scores. ## Comparison to Related Systems This approach contrasts with [[generate-then-validate-question-gen]] frameworks by front-loading pedagogical reasoning rather than post-hoc validation. Where [[ai-generated-slides-student-perception]] research has shown that AI-generated slide content can be perceived as lower quality, SlidesQAQA focuses on question quality rather than slide generation itself. It also differs from [[aissa-slides-analysis]] systems that analyze slides for accessibility rather than pedagogical question extraction. ## Pedagogical Design The bounded question budget per slide forces the system to make pedagogical decisions about what content merits a question — an implicit form of [[scaffolding]] that prioritizes key concepts. Initial experiments on two technical lecture decks demonstrated successful filtering of non-instructional slides and generation of pedagogically coherent questions for visually complex content. This has implications for [[formative-assessment]] automation at scale. ## Implications for AIED SlidesQAQA represents a practical step toward AI systems that understand pedagogical structure rather than just processing content. Its [[multimodal]] approach — jointly processing text and rendered images — addresses a known weakness of text-only question generation systems. For [[higher-ed]] instructors managing large courses, automated pedagogical question generation from existing slide materials could dramatically reduce the burden of creating formative assessments. ## Connected Concepts - [[llm]] - [[automated-question-generation]] - [[scaffolding]] - [[formative-assessment]] - [[multimodal]] - [[higher-ed]] ## Connected Articles - [[generate-then-validate-question-gen]] - [[ai-generated-slides-student-perception]] - [[aissa-slides-analysis]] ## Citation Salsman, J. (2026). [*Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation*](https://arxiv.org/abs/2605.26428). arXiv:2605.26428. --- ## [Co-Creating Buildable and Open Social Robot Study Companions with University Students](https://edtechdev.github.io/aied/articles/social-robot-study-companions/) > **Farnaz Baksh, Matevz B. Zorec, Feiazie Baksh, Karl Kruusamae** (2026). ICSR + ART 2026, London ## Key Findings - Design for Assembly (DfA) and Design for Disassembly (DfD) interventions reshape how a build feels before they shorten how long it takes. - Across two studies with developers and first-time builders, system usability climbed from Poor to Excellent (SUS 59.4 → 89.4). - Perceived workload trended downward (NASA-TLX 4.29 → 4.00), and mean assembly time trended downward (21.4 → 13.7 minutes, with a learning effect among junior builders). - Perceived workload, not completion time, appears to govern whether students take up open hardware. - Orientation cues and navigation continuity for first-time builders emerged as the next documentation frontier. ## Study Design & Method Open-source social robots offer accessibility, repairability, and student empowerment, yet the build itself often presents a barrier: existing platforms either ship pre-assembled, foreclosing hands-on learning, or expose students to unfamiliar fasteners, opaque wiring, and inaccessible service points that erode engagement. Working with university students in Guyana and Estonia, the authors applied the Double Diamond framework to co-create the Robot Study Companion (RSC) v4.1: mapping pain points, then redesigning its chassis around twist-lock fasteners, snap-fit joints, and tool-free service latches. The redesign was evaluated across two studies with both developers and first-time builders using the System Usability Scale (SUS) and the NASA-TLX workload instrument. ## Relevance to AI in Education This paper contributes directly to understanding how AI systems interact with learners in authentic educational settings. It shows that co-created open social robots with DfA/DfD design achieve excellent usability, and — more importantly — that perceived workload predicts student uptake of open hardware better than build time. For [[stem-education]] and [[human-ai-collaboration]] programs, the results suggest that accessibility interventions should target the subjective burden of assembly, not merely clock time, and that co-creation with students across diverse contexts (Guyana and Estonia) can surface pain points that designer-only processes miss. The work also connects to [[open-source]] and [[design-thinking]] values in education: repairability and tool-free service make the robot a site of continued learning rather than a disposable appliance, supporting the case for buildable platforms over pre-assembled ones in formal and informal settings. ## Connected Concepts - [[math-education]] - [[pedagogical-agent]] - [[design-thinking]] - [[open-source]] - [[agentic-ai]] - [[stem-education]] - [[ai-tutoring]] - [[affective-tutoring]] ## Connected Articles - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence ## Citation Baksh, F., Zorec, M. B., Baksh, F., & Kruusamäe, K. (2026). [*Co-Creating Buildable and Open Social Robot Study Companions with University Students*](https://arxiv.org/abs/2606.15239). arXiv:2606.15239. ICSR + ART 2026, London. --- ## [Socially fluent AI decouples conversational signals from source identity in online interaction](https://edtechdev.github.io/aied/articles/socially-fluent-ai-identity-detection/) This study embedded undisclosed AI agents as teammates in synchronous text-based group interactions across analytical, creative, and ethical tasks with 786 participants making 1,572 identity judgments. The central finding is striking: **humans cannot distinguish AI from human teammates above chance levels**. This failure is not due to a lack of identity-relevant information — computational models could accurately classify AI vs. human from conversational behavior — but because participants relied on **flawed suspicion heuristics** (response speed, fluency, perceived scriptedness) that were only weakly correlated with actual identity. The implications for education are significant. As AI agents increasingly participate in [[student-experience|student group work]], online discussions, and peer learning environments, students may interact with AI without awareness. This creates vulnerabilities explored in [[eduframetrap-llm-sycophancy-educational-safety]] — if students cannot detect AI teammates, sycophantic AI could reinforce misconceptions unchallenged. The findings also complicate [[hybrid-human-ai-tutoring-differentiated]] models that rely on transparent role differentiation. The work underscores the urgent need for [[ai-literacy]] curricula that teach not just how to use AI, but how to recognize when one is interacting with it. The dissociation between behavioral truth and human perception suggests that [[over-reliance]] on AI may be exacerbated when source identity is concealed. ## Connected Concepts - [[student-experience]] - [[ai-literacy]] - [[over-reliance]] ## Connected Articles - [[eduframetrap-llm-sycophancy-educational-safety]] - [[hybrid-human-ai-tutoring-differentiated]] ## Citation Lixiang Yan, Yueqiao Jin, Xibin Han, Dragan Gasevic (2026). [Socially fluent AI decouples conversational signals from source identity in online interaction](https://arxiv.org/abs/2605.23426). arXiv:2605.23426. arXiv preprint (cs.HC, cs.AI). - [[humanlike-ai-collaborative-writing]] — AI identity detection gap compounds costs of humanlike interface design --- ## [A Bottom-Up Taxonomy of Student Discourse with a Socratic AI Physics Tutor](https://edtechdev.github.io/aied/articles/socratic-ai-physics-tutor-taxonomy-2026/) > **Synthesis:** Large language model (LLM) tutors are being deployed in introductory physics courses at a scale that produces transcript corpora far larger than traditional qualitative coding can absorb. This study builds a bottom-up taxonomy of student discourse from a Socratic AI physics tutor deployed in introductory calculus-based mechanics. Each student turn was assigned an emergent label by an LLM coder, then consolidated into 357 categories. The top 25 categories cover roughly half of all student turns, with equation-handling and meta-procedural requests dominating. A central finding is the striking prevalence of meta-procedural turns in which students cede strategic control to the tutor. ## Key Findings Large language model (LLM) tutors are being deployed in introductory physics courses at a scale that produces transcript corpora far larger than traditional qualitative coding can absorb. This study builds a bottom-up taxonomy of student discourse from a Socratic AI physics tutor deployed in introductory calculus-based mechanics. Each student turn was assigned an emergent label by an LLM coder, then consolidated into 357 categories. The top 25 categories cover roughly half of all student turns, with equation-handling and meta-procedural requests dominating. A central finding is the striking prevalence of meta-procedural turns in which students cede strategic control to the tutor. The work contributes to understanding of [[ai-tutoring]] in educational contexts, with implications for [[socratic-method]], [[physics-education]]. ## Connected Concepts - [[ai-tutoring]] - [[socratic-method]] - [[physics-education]] - [[intelligent-tutoring]] - [[educational-nlp]] - [[student-experience]] ## Connected Articles - [[hashmi-socratic-physics-chatbot-2025]] - [[becker-chatgpt-typology-physics-2026]] - [[fouad-bentley-trust-utility-gap-physics-2026]] ## Citation Hashmi, S. F. A., & Rebello, N. S. (2026). [*A Bottom-Up Taxonomy of Student Discourse with a Socratic AI Physics Tutor*](https://arxiv.org/abs/2608.07373). arXiv:2608.07373. --- ## [The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations](https://edtechdev.github.io/aied/articles/socratic-tests-conversational-assessment/) > **Ilya Mikhelson** — Submitted to Computers and Education: Artificial Intelligence (2026). ## Synthesis The Socratic Test is an automated, computer-mediated conversational assessment that replaces static, deficit-based grading with a dynamic, additive model. It integrates Dynamic Assessment principles, multimodal workspaces, Bloom's Taxonomy for real-time proctoring, and the SOLO Taxonomy for structural evaluation. Graduated scaffolding is formalized to quantify a student's Zone of Proximal Development (ZPD): the assessment actively maps cognitive boundaries by adapting question difficulty and support in real time, so measurement targets what a student can achieve with assistance rather than only unaided performance. The grading architecture is non-compensatory and additive, prioritizing mastery over penalty: success at harder levels outweighs failures at easier ones, which the author argues reduces the penalty on ambition and restores diagnostic feedback value lost in subtractive scoring. A stated goal is human-AI alignment for measurement reliability: the conversational format is designed to avoid construct-irrelevant variance from performative anxiety and the power imbalances of face-to-face oral examinations, though the paper is a theoretical foundation with implementation and validation left to future work. ## Connected Concepts - [[automated-essay-scoring]] - [[automated-question-generation]] - [[formative-assessment]] - [[ai-ed-evaluation]] - [[open-source]] - [[socratic-method]] - [[reducing-ai-misuse]] - [[math-education]] ## Connected Articles - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[structrag-diagram-reasoning-ai-tutoring]] — Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading - [[code-gen]] — CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation - [[aiawe-automated-writing-evaluation]] — AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models ## Citation Mikhelson, I. (2026). [The theoretical foundation of Socratic tests: Dynamic, multimodal, conversational examinations](https://arxiv.org/abs/2607.29624). arXiv:2607.29624. --- ## [Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participatory Design Study](https://edtechdev.github.io/aied/articles/socraticode-k12-programming-tutor/) ## Key Finding Socratic questioning, reflection prompts, misconception checks, and mandatory pauses produce better K-12 engagement than directive answer-giving AI tutors. ## Synthesis SocratiCode demonstrates a participatory design evolution from directive AI tutor to Socratic learning companion for K-12 programming. Over four weeks with two Python learners, the system shifted from flexible tutorial generation toward dialogic support: guided questioning instead of answers, reflection prompts, misconception checks, incremental hints, and mandatory pauses requiring learner input. This Socratic shift improved explanation clarity and problem-solving engagement. The findings directly reinforce the [[codify-socratic-tutoring-programming]] approach of discovery-based learning over direct answer generation, but extend it to the K-12 context where [[cognitive-load-theory]] concerns are particularly acute. The emphasis on mandatory pauses and reflection aligns with [[metacognition]] and [[self-regulated-learning]] scaffolding strategies. The authors argue that AI tutoring is most effective as a companion within a human-guided framework, not an answer engine — a principle that resonates with the [[human-in-the-loop-ai]] architecture and the findings from [[structured-llm-feedback-programming]] that less guided feedback may be more effective. ## Connected Concepts - [[cognitive-load-theory]] - [[metacognition]] - [[self-regulated-learning]] - [[human-in-the-loop-ai]] ## Connected Articles - [[codify-socratic-tutoring-programming]] - [[structured-llm-feedback-programming]] ## Citation preprint, A. (2026). [*Towards SocratiCode: Designing a Generative AI-Based Programming Tutor for K-12 Students through a 4-Week Participatory Design Study*](https://arxiv.org/abs/2605.17857) --- ## [Is Solving Better Than Evaluating GenAI Solutions?](https://edtechdev.github.io/aied/articles/solving-vs-evaluating-genai-solutions/) Randomized A/B crossover study (N=220) in a junior-level algorithms course comparing solution evaluation/critique tasks against traditional solution generation. Finds that evaluation-centered tasks produce comparable learning outcomes with a localized homework advantage that does not transfer to summative performance, suggesting evaluation tasks are a viable pedagogical response to the availability of GenAI code generation tools. ## Key Findings - No statistically significant differences were found between groups on midterm scores, final exam scores, overall course grades, or exam problems structurally aligned with the homework interventions. - Students received significantly higher homework scores during periods in which they evaluated GenAI-generated solutions, but this localized advantage on the modified homework items did not translate into downstream summative gains. - Survey data indicated that most students did not report changing their study habits in response to the intervention; however, students who did report adapting their study strategies rated the GenAI-evaluation assignments as significantly more helpful. - GenAI evaluation redistributes student effort from open-ended solution construction toward verification, diagnosis, and judgment, but does not automatically produce stronger conceptual transfer. - GenAI-evaluation activities can be incorporated into algorithms coursework without broad performance losses, but meaningful learning gains may require deliberate scaffolding that pushes students beyond simple error diagnosis. ## Study Design & Method The study was a randomized A/B crossover experiment (N = 220) conducted in a junior-level, theory-heavy algorithms course. Across six assignments, student working groups either solved challenging algorithmic problems directly or evaluated often-flawed GenAI-generated solutions to corresponding problems, with group roles reversed midway through the semester. A survey with a high response rate (Group A: n = 101; Group B: n = 107; overall N = 208, 95%) measured students' self-reported comfort and confidence across the five core algorithmic topics assessed on the midterm — asymptotic analysis, divide and conquer, dynamic programming, greedy methods, and graph algorithms — using five-point Likert scales. Within-student difference scores were computed to examine changes in homework performance on GenAI-graded versus independently solved problems across the two halves of the course. ## Implications for AI in Education The results speak directly to the debate over how computing educators should respond to GenAI tools that can generate assignment solutions. Evaluation-centered tasks — verification, critique, and diagnosis of AI-produced artifacts — are a defensible pedagogical option: they can be incorporated without broad performance losses, and students who engaged with them as a strategy reported greater perceived value. However, the absence of transfer to exams warns that evaluation practice alone does not build the same conceptual depth as generation under struggle, and the authors conclude that meaningful gains require deliberate scaffolding beyond simple error diagnosis. For [[cs-education]] and [[critical-thinking]] instruction, the study suggests framing GenAI evaluation as a structured activity with explicit reflection, rather than assuming that judging flawed solutions automatically strengthens algorithmic understanding. It also models a rigorous A/B crossover methodology for evaluating GenAI pedagogy in [[higher-ed]]. ## Connected Concepts - [[higher-ed]] - [[cs-education]] - [[critical-thinking]] - [[math-education]] - [[administrator]] - [[socratic-ai-dialogue]] - [[physics-education]] - [[pedagogical-agent]] ## Connected Articles - [[cognitive-shift-ai-education]] — Evidence of a Cognitive Shift in AI Education: How Students Are Rethinking Human Intelligence? - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[ai-generated-instructional-videos-computing-ed]] — Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing Education - [[genai-availability-grades-satisfaction]] — Generative AI Availability, Grades, and Student Satisfaction at a Large University ## Citation Ethan Dickey, Marios Mertzanidis, Alexandros Psomas (2026). [Is Solving Better Than Evaluating GenAI Solutions?](https://arxiv.org/abs/2607.27586). arXiv:2607.27586. cs.CY, cs.AI. --- ## [Special-R1: Reinforcement Learning for Special Education — Aligning LLM Tutors to Diverse Learners through Disability-Adaptive Training](https://edtechdev.github.io/aied/articles/special-r1-rl-special-education/) > **Authors:** Unggi Lee, Jihoi Na, Yeil Jeong, Haeun Park, Yeonju Jang (2026) # Special-R1: RL for Special Education ## What It Is Special-R1 is a framework that extends pedagogical reinforcement learning (RL) to special education. While prior RL-based tutor alignment methods targeted only generic math learners, Special-R1 explicitly models cognitive and communicative diversity across five disability profiles. ## How It Works The framework has two core components: 1. **Two-dimensional adaptive system prompt**: Couples a difficulty-based support level (scaffolding) with a disability-specific teaching style, forming a persona-aware prompt that guides the LLM tutor during multi-turn dialogue. 2. **Persona-aware Thinking Reward**: The judge rubric used to compute the training reward is conditioned on the learner's disability profile rather than a generic student. This shapes the tutor to produce responses that are helpful, safe, and appropriately challenging for each specific persona. ## Key Results - Evaluated on 690 multi-turn dialogues with diverse learner personas - Generic baseline tutor achieved Fit score of 6.75/10 - Special-R1 full model raised Fit to **8.40** (+1.65) - SPED-rubric Helpfulness improved from 0.720 to 0.768 - No degradation on out-of-domain OpenLearnLM benchmark (8.53 vs 8.53) - Ablations showed the Thinking Reward only becomes effective in combination with adaptive prompting ## Critical Insight Students with specific learning disabilities in mathematics remain underserved, suggesting a need for multimodal extensions (visual aids, interactive diagrams) in future work. ## Why It Matters This is the first multi-turn pedagogical RL framework specifically targeting special education. It demonstrates that LLM tutors can be systematically aligned to support students with disabilities, improving both perceived helpfulness and pedagogical fit. The persona-conditioned reward rubric provides a replicable recipe for adapting RLHF-based tutor fine-tuning to diverse learner profiles. ## Open Questions - How well does Special-R1 generalize to disabilities beyond the five profiles modeled? - Can the framework be extended to multimodal inputs for learners with specific math disabilities? - What are the ethical considerations of disability-profiling in AI tutoring systems? ## Connected Concepts - [[reinforcement-learning]] - [[pedagogical-agent]] - [[lifelong-learning]] - [[personalized-learning]] - [[adaptive-learning]] - [[affective-tutoring]] - [[help-seeking]] - [[pedagogical-llm-training]] ## Connected Articles - [[a4l-analytics-pipeline]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adapt-adaptive-lesson-plan-transformer]] - [[agent-voice-accents-k12-group-learning]] - [[agentic-ai-education-scoping-review]] - [[agentic-ai-pedagogical-best-practice-2026]] - [[agentic-education-coding]] - [[agentic-literacy-debt]] ## Citation Lee, U., Na, J., Jeong, Y., Park, H., & Jang, Y. (2026). [Special-R1: Reinforcement Learning for Special Education — Aligning LLM Tutors to Diverse Learners through Disability-Adaptive Training](https://arxiv.org/abs/2605.30670). arXiv:2605.30670. --- ## [Exploring AI-Supported Disciplinary Mediation in Student Project Teams' Text-Based Communication](https://edtechdev.github.io/aied/articles/spritz-ai-disciplinary-mediation-student-teams-2026/) > **Synthesis:** Cheng, Chung, Chiu, Lin & Liao (2026) present Spritz, a Discord-based [[llm]] technology probe that mediates disciplinary boundaries in interdisciplinary student project teams, finding that students valued AI as both cognitive support for boundary crossing and a relational buffer — while a central tension emerged when AI moved from neutral mediator to advisor or challenger. ## Key Findings 1. **The challenge of interdisciplinary sensemaking.** Students in interdisciplinary [[active-learning]] must negotiate differences in language, assumptions, priorities, and practices, yet these differences are hard to surface in fragmented text-based team communication where AI tools often become private side channels. 2. **Spritz as a mediating probe.** Spritz monitors group chat for semantic or pragmatic boundary signals, prompts members to articulate perspectives through private channels, and returns anonymized syntheses to shared discussion. A technology probe study and co-design workshop ran with 12 university students across technical, business, and design backgrounds. 3. **Dual value: cognitive and relational.** Participants valued AI mediation not only as cognitive support for boundary crossing but as a relational buffer — organizing fragmented discussion, surfacing implicit expectations, clarifying divergent interpretations, and softening interpersonal pressure around disagreement and concession. 4. **The neutrality tension.** Participants imagined future AI mediators as switchable roles — strategic advisors, cross-domain translators, perspective challengers — but these expanded roles made the neutrality that had made AI acceptable as a mediator unstable once AI began to advise, challenge, or influence team decisions. ## Implications This study contributes empirical insight and design considerations for AI systems that mediate [[collaborative-learning]] in text-based communication while preserving [[human-ai-collaboration]], trust, privacy, and accountability. The finding that AI's perceived neutrality is load-bearing — and that it erodes when AI becomes a decisive actor — is a key design constraint for educational [[pedagogical-agent]]. For [[active-learning]] and [[higher-ed]], the work shows how LLM-mediated boundary objects can help interdisciplinary teams surface implicit assumptions rather than bypassing them, aligning with theories of boundary objects and knowledge integration. The tension between helpful mediation and illegitimate influence echoes broader concerns in [[trust-calibration]] and [[teacher-role]] debates about how much agency AI should hold in collaborative settings. The anonymized-synthesis design also models [[privacy]]-respecting intervention, and the switchable-role imagination suggests future [[human-in-the-loop-ai]] designs where teams can configure the AI's stance. ## Connected Concepts - [[collaborative-learning]] - [[pedagogical-agent]] - [[human-ai-collaboration]] - [[human-in-the-loop-ai]] - [[llm]] - [[privacy]] - [[active-learning]] - Qualitative Research - [[student-experience]] - [[trust-calibration]] ## Connected Articles - [[multi-agent-llm-social-learning]] — Multi-agent LLM social learning - [[llm-facilitation-timing-online-discussions]] — LLM facilitation timing - [[icap-cognitive-engagement-llm-agents]] — ICAP cognitive engagement - [[llm-critical-thinking-teamwork-review]] — LLM critical thinking teamwork - [[lecturaagents-multi-agent-teaching]] — LecturaAgents multi-agent teaching ## Citation Cheng, C.-J., Chung, Y.-C., Chiu, B.-C., Lin, Y.-H., & Liao, J.-W. (2026). [*Exploring AI-Supported Disciplinary Mediation in Student Project Teams' Text-Based Communication*](https://arxiv.org/abs/2608.07503). arXiv:2608.07503 (cs.HC). --- ## [StanBKT: Rethinking Parameter Estimation in Bayesian Knowledge Tracing](https://edtechdev.github.io/aied/articles/stanbkt-bayesian-knowledge-tracing/) StanBKT introduces an open-source Python package for Bayesian Knowledge Tracing (BKT) that moves beyond traditional expectation-maximization (EM) point estimates to full Bayesian inference via Stan. The package supports **four estimation methods** (Hamiltonian Monte Carlo, variational inference, Pathfinder, and optimization), **three model variants** (standard, grouped, and hierarchical BKT), and provides posterior predictive inference with visualization utilities. Evaluated on the ASSISTments 2020 dataset, all methods achieve comparable predictive performance while trading off speed vs. posterior fidelity. A key demonstration shows how posterior inference enables **principled comparison of learning, forgetting, guessing, and slipping parameters** across experimental conditions in an educational intervention involving perceptual cue manipulations. For [[intelligent-tutoring]] systems, this represents a significant methodological advance. Traditional BKT implementations produce point estimates that obscure uncertainty and prevent reliable condition comparisons — critical for A/B testing adaptive interventions. The hierarchical modeling support is particularly valuable for [[adaptive-learning]] where student-level variation must be modeled. Compared to [[neural-symbolic-knowledge-tracing]] approaches, StanBKT preserves the interpretable hidden Markov structure of BKT while adding uncertainty quantification. For [[knowledge-tracing-irt]] comparisons, the Bayesian framework provides natural extensions unavailable in IRT. The work complements [[mbp-kt-meta-behavioral-knowledge-tracing]] by offering a complementary probabilistic modeling framework. As an [[open-source]] tool, StanBKT lowers the barrier for [[learning-analytics]] researchers to adopt Bayesian methods in educational data mining. ## Connected Concepts - [[intelligent-tutoring]] - [[adaptive-learning]] - [[open-source]] - [[learning-analytics]] ## Connected Articles - [[neural-symbolic-knowledge-tracing]] - [[knowledge-tracing-irt]] - [[mbp-kt-meta-behavioral-knowledge-tracing]] ## Citation Siddhartha Pradhan, Yanping Pei, Morgan Lee, Puyuan Zhang, Erin Ottmar, Adam C. Sales (2026). [StanBKT: Rethinking Parameter Estimation in Bayesian Knowledge Tracing](https://arxiv.org/abs/2605.23048). arXiv:2605.23048. arXiv preprint (cs.HC, cs.CY, stat.AP, stat.ME). --- ## [Stanford Evidence Base: AI in K-12 Education](https://edtechdev.github.io/aied/articles/stanford-evidence-base-ai-k12-2026/) > **Stanford Evidence Base: AI in K-12 Education** — A 2026 systematic review from the Stanford SCALE Initiative analyzing 818 papers on AI in K-12 education. The central finding is stark: only 20 studies provide strong causal evidence, and zero high-quality causal studies examine U.S. K-12 student settings. The evidence that exists reveals a consistent pattern — AI improves performance during use but gains frequently fail to persist or transfer, and general-purpose AI tools can actively harm learning outcomes compared to pedagogically designed alternatives. **Stanford SCALE Initiative, AI Hub for Education** — Published 2026. Analysis of repository spanning through October 2025. ## Key Findings **The Evidence Gap.** Of 818 papers in the AI Hub Research Repository, only 20 met What Works Clearinghouse (2025) standards for strong causal inference (RCTs or quasi-experimental designs). Zero high-quality causal studies examine U.S. K-12 *student* settings; very few exist for U.S. K-12 *educators*. Most causal research is international, conducted in postsecondary settings, short-term (often single 20-minute sessions), and focused on immediate outcomes. The repository grew from 28 relevant papers in January 2023 to over 800 by October 2025, but methodological rigor has not kept pace with volume. **Immediate Gains, Uncertain Transfer.** AI significantly improves performance *while students use it* — math proofs, programming, economics exams, physics, and argumentative writing all show gains during AI-supported practice. However, effects are mixed or negative when AI is removed. Bastani et al. (2025) found high schoolers using a general-purpose chatbot for math practice performed **~17% worse** on closed-book final exams than peers with no AI access, despite higher practice grades. Chen et al. (2025) found LLM-Tutor improved homework scores but did not improve unassisted exam scores. Lehmann et al. (2025) found general-purpose AI for programming increased topics covered but harmed understanding and widened achievement gaps for low-prior-knowledge students. Kosmyna et al. (2025) found AI essay assistance led to **83% of participants failing to recall a quote** from their own essay, versus 11% for non-AI users. This pattern — performance boost during use, learning loss after removal — is the central empirical finding of the review and directly implicates [[transfer-of-learning]] as the most critical open question in AI education research. **Easier Doesn't Mean Better.** Students consistently report greater enjoyment and reduced cognitive burden when using AI tools. However, reduced effort can undermine deeper learning. Kreijkes et al. (2026) found retention improved only when AI use was paired with traditional strategies like note-taking. Stadler et al. (2024) found general-purpose AI reduced [[cognitive-load-theory|cognitive load]] but produced lower-quality reasoning and argumentation compared to traditional search. This aligns with [[desirable-difficulties]] research: making practice easier often harms long-term retention and transfer, even when it feels better in the moment. **Pedagogical Design Matters.** The most actionable finding: tutoring-specific tools consistently outperform general-purpose chatbots. Bastani et al. found that a tutoring-specific chatbot with pedagogical guardrails (hints, step-by-step reasoning, refusal to give direct answers) mitigated the exam score drop, while general-purpose GPT Base *caused* it. This suggests that [[ai-tutoring]] effectiveness depends critically on pedagogical design, not just model capability. The review interprets findings through a learning science framework spanning [[zone-of-proximal-development]] (general-purpose AI may operate outside the ZPD by doing work for students), the expertise reversal effect (novices need guidance, experts need independence), and [[metacognition]] (AI completing tasks reduces opportunities for students to monitor their own understanding). **Educator Evidence.** While the student-focused causal evidence is thin, the educator evidence base is even sparser. Very few high-quality studies examine how AI affects teacher practice, workload, or professional development — a gap that is particularly concerning given the rapid push to deploy AI tools in classrooms and the documented [[genai-can-harm-teaching-rct-2026|risks of AI harming teaching quality]]. ## Implications for AI in Education This review is a watershed document for the field. It establishes that the evidence base for AI in K-12 education is not merely thin — it is *absent* for the populations and contexts where deployment is most aggressively pursued (U.S. K-12 classrooms). The finding that zero high-quality causal studies exist for U.S. K-12 students should give pause to every district administrator, edtech vendor, and policy maker advocating for rapid AI adoption. The consistent pattern of immediate gains without durable transfer challenges the prevailing assumption that AI assistance automatically improves learning. It suggests that many AI education tools may function as **performance prosthetics** — helping students complete tasks in the moment without building the underlying knowledge that enables independent performance later. This distinction between assisted performance and genuine learning is well-established in [[rct|RCT-based education research]] but has been largely overlooked in the AI education hype cycle. The superiority of pedagogically designed tools over general-purpose AI is actionable: it implies that simply giving students access to ChatGPT or similar chatbots is not merely suboptimal but potentially harmful. Effective AI in education requires deliberate instructional design — [[scaffolding]], [[zone-of-proximal-development|ZPD]]-aligned support, refusal to bypass student thinking, and integration with established learning activities. This connects to broader work on [[ai-pedagogical-orientation]] and the growing recognition that [[access-not-enough-ai-tutoring-2026|access to AI tutoring is not enough]] without thoughtful pedagogical integration. For the research community, the review functions as both a wake-up call and a roadmap. It identifies urgent priorities: long-term studies with delayed post-tests, research in authentic U.S. K-12 settings, studies of educator use and impact, and research designs that disentangle assisted performance from durable learning. The [[k-12-ai-education]] field urgently needs to move beyond descriptive and technical-computational papers (which together constitute 92% of the repository) toward rigorous causal designs. ## Connected Concepts - [[ai-tutoring]] - [[cognitive-load-theory]] - [[desirable-difficulties]] - [[k-12]] - [[k-12-ai-education]] - [[metacognition]] - [[rct]] - [[scaffolding]] - [[zone-of-proximal-development]] - [[ai-literacy]] ## Connected Articles - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[transfer-of-learning]] — Transfer of Learning - [[ai-pedagogical-orientation]] — Faculty Orientations Shape Adoption of AI in Research and Teaching - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory ## Citation Stanford SCALE Initiative, AI Hub for Education. (2026). [*The Evidence Base on AI in K-12: A 2026 Review*](https://scale.stanford.edu/sites/default/files/The%20Evidence%20Base%20on%20AI%20in%20K-12%20Report.pdf). Stanford University. --- ## [How State Policy Can Help Teachers Use AI Well](https://edtechdev.github.io/aied/articles/state-policy-teacher-ai/) > A NASBE/CRPE policy analysis (May 2026) examining how US states can shape conditions for effective teacher AI adoption — setting guardrails, providing resources, and building capacity without micromanaging implementation. # State Policy for Teacher AI Use ## Definition A NASBE/CRPE policy analysis (May 2026) examining how US states can shape conditions for effective teacher AI adoption — setting guardrails, providing resources, and building capacity without micromanaging implementation. ## Key Data Points - **54%** of students used AI for school in 2025 (+15 pp vs. prior) - **~2/3** of K-12 teachers used AI in 2024-25 - Teachers using AI weekly saved **~6 hours/week** on planning, grading, feedback, admin - Only **45%** of principals report having school/district AI policy/guidance - **35 states/territories** have official AI guidance (as of Dec 2025), mostly nonbinding - Equity gaps: suburban, majority-White, low-poverty districts **twice as likely** to provide AI training vs. urban/rural/high-poverty ## Five Recommendations 1. **Define statewide vision** — align with workforce needs; create AI task forces; adopt literacy frameworks 2. **Shift to learning organization model** — capacity over compliance; cross-functional teams; rapid guidance updates 3. **Support tool evaluation and procurement** — "fewer, better" tools; evidence standards; outcomes-based contracts 4. **Center human connection** — AI must foster, not replace, human relationships 5. **Build evidence infrastructure** — fund research on what works; share findings across districts ## Connected Concepts - [[ai-literacy]] - [[equity-in-ai-education]] - [[faculty-development]] - [[regulation]] - [[teacher-role]] - [[ai-education]] - [[k-12]] ## Connected Articles - [[teacher-ai-adoption-confidence]] — AI Adoption Among Teachers: Insights on Concerns, Support, Confidence, and Attitudes - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[ai-ethics-education-public-discourse]] — A Longitudinal Analysis of Public Discourse on AI Ethics in Education Using Twitter Data - [[ai-fatigue-academic-contexts]] — Defining AI Fatigue in Academic Contexts: Dimensions, Indicators, and a Stage-Based Model Using Grounded Theory - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions ## Citation CRPE, S.N. (2026). [How State Policy Can Help Teachers Use AI Well](https://www.nasbe.org/how-state-policy-can-help-teachers-use-ai-well/) --- ## [Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education](https://edtechdev.github.io/aied/articles/structrag-diagram-reasoning-ai-tutoring/) Presents **StructRAG**, a pattern-aware framework that improves how AI tutoring systems interpret **complex engineering diagrams** (circuit schematics, network topologies, block flowcharts) in STEM. Current AI systems fail on diagrammatic questions because they cannot reliably extract spatial relationships and structural connectivity from noisy real-world diagrams (curved lines, overlapping elements, occluded labels). StructRAG shifts from surface-level visual recognition to **pattern-level structural reasoning**: convert the diagram to a graph, retrieve structurally similar canonical patterns, and prompt an LLM to validate plausibility and completeness — enabling detection of missing connections and structure-aware feedback. ## Architecture (four components) 1. **Diagram-to-Graph Conversion** — OCR (Tesseract, multi-scale 1.0/1.5/2.0, majority voting) + classical CV (Hough Line Transform, contour detection, Zhang–Suen skeletonisation for curved paths). Edge confidence `Cij = 0.45·scont + 0.25·sprox + 0.20·salign + 0.10·snode`; edges ≥ 0.65 accepted, 0.40–0.65 routed to an *uncertain-edge set* U for pattern-level validation, < 0.40 discarded as noise. 2. **Structural Pattern Retrieval** — abstracts the recognized graph into topology patterns (star, ring, chain/bus, bridge/mesh, tree, hybrid, cross-layer) and retrieves similar templates from a curated library using **graph-edit-distance (GED)** matching. 3. **Pattern-Aware Prompt Construction** — feeds the LLM the graph G, uncertain edges U, retrieved templates, and candidate corrections ΔE. 4. **LLM-Guided Structural Reasoning and Correction** — GPT-4 jointly interprets G, U, T*, and ΔE to decide which uncertain edges to add and which structures to correct, with output ensembling. ## Results (1,650 STEM diagram-based questions) - **RQ1 (accuracy):** macro-average question-level accuracy **89.3%** (95% CI [86.8, 91.9]) — beating OCR+CV only (70.8%), GPT-4 graph-only (81.9%), direct-image GPT-4o (84.8%), and StructRAG-without-LLM (82.7%); significant after Holm–Bonferroni correction (p < 0.01). - **RQ2 (missing-connection recovery):** micro-averaged **edge-level F1 = 93.0%** (CI [91.8, 94.2]) — best in every topology category; largest gains over OCR+CV on irregular (26.4 pts), bridge/mesh (23.1), and ring (21.6) topologies. - **RQ3 (retrieval strategy):** pattern-level retrieval beats node-level statistics and example-based retrieval. - Best on star (94.5%) and chain/bus (93.1%); hardest on bridge/mesh (85.8%) and unclassified/irregular (81.4%) diagrams — yet still best-in-class there. - Component ablation (Table 10): full pipeline 89.3/93.0 vs pattern-aware GPT-4 without ensembling 87.8/91.4 and template retrieval + GED without LLM 82.7/87.6 — every component contributes. ## Interpretation - Question-level accuracy is always lower than edge-level F1 because Acc demands a perfect whole-diagram match, while F1 captures partial correctness — relevant for education, where identifying *most* missing connections already yields useful structural feedback. - Grounded in **cognitive apprenticeship**: instructors guide students through visual problem-solving by highlighting analogies, missing links, and logical dependencies; StructRAG's template retrieval mirrors that by making structural relationships explicit and auditable (an intermediate graph representation, rather than end-to-end visual matching). - Interviews with 12 STEM educators were positive: actionable structural feedback, adaptability across diagram types, and help explaining missing/ambiguous connections to students struggling with abstract topologies. - Limitations: dataset withheld (university teaching materials; schema/code/prompts released), and no learner-outcome study yet — future work should test effects on learning, retention, transfer, motivation, and cognitive load in live tutoring contexts. ## Connected Concepts - [[automated-question-generation]] - [[socratic-method]] - [[ai-tutoring]] - [[math-education]] - [[socratic-ai-dialogue]] - [[pedagogical-agent]] - [[open-source]] - [[agentic-ai]] ## Connected Articles - [[structured-llm-feedback-programming]] — The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance - [[eduguard-safe-rag-llm-tutor]] — EduGuard: A Safe RAG-Based LLM Tutor for Programming Education - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[llm-judged-helpfulness-pedagogy-signal]] — Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models ## Citation Sun, Y., Liao, Y., & Ma, X. (2026). Advancing diagram-based reasoning in AI tutoring systems: A structural approach for STEM education. [*Smart Learning Environments*](https://doi.org/10.1186/s40561-026-00459-9). Advance online publication --- ## [Structural Silence: When AI Infrastructure Fails Speakers of Underrepresented Languages](https://edtechdev.github.io/aied/articles/structural-silence-underrepresented-language-ai-2026/) > **Synthesis:** Roy & Roy (2026) argue that the **infrastructure of AI** — training corpora, tokenization, benchmarks, deployment architectures — systematically disadvantages speakers of underrepresented languages *before a model is trained*, reframing dataset scarcity as a structural barrier rather than an isolated technical limitation. Using Bengali as a case in AI-assisted education, they document four interlocking failures: a web-presence gap (<0.5% of global content for ~4% of the population), a 67:1 English↔Bengali training-token deficit, a tokenization penalty from the alphasyllabary script, and connectivity exclusion (36.5% rural vs 71.4% urban internet penetration). They position **offline-first design** as an equity-oriented infrastructure strategy. The work connects to [[equity]], [[language-learning]], and [[digital-divide]] debates in educational AI. ## Four Interlocking Infrastructure Failures The paper identifies four structural barriers that compound to exclude underrepresented languages from AI-assisted education: - **Web presence gap:** Bengali accounts for under 0.5% of global web content despite representing nearly 4% of the global population. - **Training-token deficit:** a 67:1 deficit between English and Bengali in major multilingual corpora. - **Tokenization penalty:** Bengali's alphasyllabary script compounds the data deficit through higher token fertility. - **Connectivity exclusion:** individual internet penetration is 36.5% in rural areas versus 71.4% in urban areas. These failures reflect longstanding resource-allocation decisions, institutional priorities, and design defaults that did not center underrepresented languages in mainstream AI development. ## Reframing Scarcity as Structure The authors argue dataset scarcity should be understood as a **structural barrier** rather than an isolated technical limitation. They recommend treating **offline-first design** as an equity-oriented infrastructure strategy for AI-assisted education in low-connectivity environments, and outline directions for linguistics and AI research aimed at reducing these structural inequalities. ## Connected Concepts - [[equity]] - [[language-learning]] - [[language-learning]] - [[digital-divide]] - [[equity-in-ai-education]] - [[equity-in-ai-education]] - [[ai-education]] - [[higher-ed]] - [[privacy]] - [[accessible-learning]] ## Connected Articles - [[ai-scoring-language-bias-physics]] - [[sec-ai-literacy-narrative-review-2026]] - [[data-comics-for-education-evaluating-effectiveness-benefits-ethics]] ## Citation Roy, A., & Roy, P. (2026). [*Structural silence: When AI infrastructure fails speakers of underrepresented languages*](https://arxiv.org/abs/2608.12278). arXiv:2608.12278. --- ## [Structured AI Demonstrations and Student LLM Use in Engineering Mechanics: Study Design and Preliminary Results](https://edtechdev.github.io/aied/articles/structured-ai-demonstrations-engineering-mechanics/) > **Shuang Geng, Helen Lallos-Harrell, Jiya Ashar, Thomas J. McKenna, Annwesa Dasgupta, Caleb Farny, Emma Lejeune** — arXiv preprint (2026). ## Synthesis This descriptive study documents student LLM use in an undergraduate engineering mechanics course (Spring 2026), responding to the lack of domain-specific empirical evidence for pedagogical policy in engineering education. The authors contribute a reproducible survey instrument capturing student AI usage patterns, attitudes, and verification practices, linked to academic performance metrics — an open methodological framework for other instructors. A deployable sequence of nine structured, instructor-led AI demonstrations models strategic LLM delegation and evaluation for students, treating AI use as a taught skill rather than an assumed behavior. Preliminary data show shifting student behaviors and complex relationships between AI reliance and course outcomes, though the primary contribution is the study design itself; results are framed as preliminary. ## Connected Concepts - [[administrator]] - [[socratic-ai-dialogue]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[agentic-ai]] - [[ai-tutoring]] - [[math-education]] - [[prompt-engineering]] ## Connected Articles - [[trio-ethnography-llm-programming-education]] — Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[ai-team-teaching-talk-analytics]] — AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty ## Citation Geng, S., Lallos-Harrell, H., Ashar, J., McKenna, T. J., Dasgupta, A., Farny, C., & Lejeune, E. (2026). [Structured AI demonstrations and student LLM use in engineering mechanics: Study design and preliminary results](https://arxiv.org/abs/2607.28710). arXiv:2607.28710. --- ## [The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance](https://edtechdev.github.io/aied/articles/structured-llm-feedback-programming/) ## Key Finding LLM-generated feedback produces faster time-to-solution than compiler-only baseline; counterintuitively, less guided feedback showed stronger effects than more guided variants. ## Synthesis This study provides empirical evidence on how LLM-generated feedback structure affects programming students' problem-solving. In an online programming course, three LLM feedback types (varying guidance levels) were compared against a compiler-error-only baseline. All LLM feedback conditions led to faster time-to-solution, but surprisingly, less guided feedback showed slightly stronger effects than more directive variants. This aligns with [[socratic-method]] principles and the [[desirable-difficulties]] framework, suggesting that too much hand-holding may short-circuit productive struggle. The finding also resonates with the [[codify-socratic-tutoring-programming]] approach using discovery-based learning over direct answer generation. Combined with [[ai-tutor-behavioral-evaluation]], which showed that behavioral engagement metrics matter more than feedback quality ratings alone, this paper strengthens the case for designing LLM feedback that encourages student action rather than passive consumption. Future work should explore adaptive feedback that calibrates guidance based on [[knowledge-tracing-irt]] student models and individual [[programming-its]] experience levels. ## Connected Concepts - [[socratic-method]] - [[desirable-difficulties]] ## Connected Articles - [[codify-socratic-tutoring-programming]] - [[ai-tutor-behavioral-evaluation]] - [[knowledge-tracing-irt]] - [[programming-its]] ## Citation Mihaylova, T., Logacheva, E., Hellas, A., Fan, J., Castro, F., Akram, B., Norouzi, N., Brusilovsky, P., & Leinonen, J. (2026). [The Effects of Structured LLM-Generated Feedback on Programming Assignment Performance](https://arxiv.org/abs/2605.16933). --- ## [Make or Take: How Students Navigate Self-Created and Instructor-Provided Cheat Sheets](https://edtechdev.github.io/aied/articles/student-cheat-sheets-make-or-take/) Chen, Sakhnini and Istead run a three-wave longitudinal study in a senior software-requirements course where students could use instructor-provided or self-created cheat sheets in exams. Choices were shaped by trust in instructor expertise, desire for personalization, and preparation efficiency, and shifted over time. The make-vs-take decision is fundamentally a [[metacognition]] and [[self-regulated-learning]] question — creating a cheat sheet is itself a generative study strategy — with direct implications for exam design in [[authentic-assessment]], for optimizing preparation such as [[adaptive-pretesting-retention]], and for cognitive-structure views of exams like [[learnopt-exam-cognitive-structure]]. It also frames the trade-off AI raises: offloading artifact creation versus learning through creation, cf. [[cognitive-offloading-speedup-illusion]]. ## Connected Concepts - [[metacognition]] - [[self-regulated-learning]] ## Connected Articles - [[authentic-assessment]] - [[adaptive-pretesting-retention]] - [[learnopt-exam-cognitive-structure]] - [[cognitive-offloading-speedup-illusion]] ## Citation Helen Weixu Chen, Victoria Sakhnini, Lesley Istead (2026). [Make or Take: How Students Navigate Self-Created and Instructor-Provided Cheat Sheets](https://arxiv.org/abs/2607.24736). arXiv:2607.24736. --- ## [Archetypes or ability? Clustering for modelling student mathematical competence](https://edtechdev.github.io/aied/articles/student-math-competence-clustering/) On 119,034 students across 13 UK national exams, Bernoulli Mixture Models found few distinct skill clusters — overall ability dominates. A simple explainable model achieved 78% accuracy, competitive with complex approaches. Small personalization gains are possible by accounting for individual question-level strengths, but students don't develop strongly divergent ability profiles across topics. **Relevance to AI in Education:** This paper contributes to the understanding of [[automated-assessment]], [[personalized-learning]], and [[student-experience]]. The findings have implications for [[adaptive-learning]] systems, [[formative-assessment]] design, and the broader [[edtech-platform]] landscape. Future work should explore how these results generalize across [[stem-education]] and [[higher-ed]] contexts. This research connects to the growing body of work on [[ai-literacy]] and [[teacher-role]], highlighting both the promise and limitations of AI tools in educational settings. ## Connected Concepts - [[automated-assessment]] - [[personalized-learning]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[edtech-platform]] - [[stem-education]] - [[higher-ed]] - [[ai-literacy]] - [[teacher-role]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System ## Citation Benjamin Mawdsley, Tom Quilter, Richard Turner, Sarah Jackson, Paul Edwards (2026). [Archetypes or ability? Clustering for modelling student mathematical competence](https://arxiv.org/abs/2607.26063). arXiv:2607.26063. arXiv preprint. --- ## [Uncovering Students' Mental Models of Generative Artificial Intelligence](https://edtechdev.github.io/aied/articles/student-mental-models-genai/) This study investigates how students conceptualize generative AI (GenAI) and how those mental models shape their academic integration. A student's mental model of GenAI — their beliefs about what it can and cannot do — influences both perceived capability and choices about when to delegate tasks. The authors surface the range of student conceptions, from tool-as-calculator to collaborator, and show that inaccurate or shallow models correlate with [[over-reliance]] and weaker learning outcomes. The work connects to broader debates on [[ai-literacy]] and [[student-experience]] with AI, arguing that mental-model accuracy is a prerequisite for productive [[human-ai-collaboration]]. It extends [[metacognition]] research by treating AI understanding as a learnable metacognitive skill, and bears on [[generative-ai]] use in [[higher-ed]]. Implications include designing interventions that explicitly calibrate students' models rather than assuming fluency. ## Connected Concepts - [[over-reliance]] - [[ai-literacy]] - [[student-experience]] - [[human-ai-collaboration]] - [[metacognition]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[buggy-genai-code-student-responses]] — When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code - [[code-review-genai-cs1]] — Combating Harms of Generative AI in CS1 with Code Review Interviews and a Flipped Classroom - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis - [[ai-generated-feedback-higher-ed]] — Artificial intelligence and feedback in university education: effectiveness and student perceptions - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review ## Citation Amrita Ganguly, Sai Sharanya Garika, Aditya Johri (2026). [Uncovering Students' Mental Models of Generative Artificial Intelligence](https://arxiv.org/abs/2607.11692). arXiv:2607.11692. arXiv preprint. --- ## [How Students (Mis)understand Conditionals and Loops -- A Taxonomy](https://edtechdev.github.io/aied/articles/student-misconceptions-conditionals-loops-taxonomy/) ## Overview This paper presents a fine-grained taxonomy categorizing novice programmers' difficulties with reading and understanding control flow constructs — specifically conditionals (selection) and loops (iteration). Developed through the Extended Taxonomy Design Process (ETDP), the taxonomy integrates prior research with new empirical data from student quizzes and interviews. It provides a harmonized framework for [[cs-education]] researchers to classify and analyze student errors systematically. ## Key Contributions The taxonomy distinguishes between qualitatively different types of student misunderstandings. Instead of treating all errors as one category, it separates difficulties in tracing conditional branches from difficulties in understanding loop iteration semantics. This granularity offers [[scaffolding]] designers precise targets for intervention. The framework connects naturally to [[programming-its]] research, where intelligent tutoring systems need fine-grained misconception models to provide targeted feedback. ## Relationship to AI in Education While the paper itself does not involve AI, its taxonomy has implications for [[llm-student-misconception-identification]] systems that automatically detect and classify student errors. A structured taxonomy enables training data annotation, evaluation benchmarks, and informed remediation strategies. It also supports [[automated-grading]] systems by providing a vocabulary for explaining why a particular answer is wrong, moving beyond binary correct/incorrect judgments. ## Methodology The ETDP methodology iterates between literature synthesis, empirical data collection, and taxonomy refinement. Quizzes and interviews with students provided the empirical grounding, while prior work on programming misconceptions ensured theoretical continuity. This mirrors the approach used in [[student-experience]] research that combines qualitative and quantitative methods to understand learning. ## Implications For educators, the taxonomy supports more precise diagnosis of student difficulties, enabling targeted remediation within [[formative-assessment]] cycles. For researchers, it provides a shared language for comparing intervention effectiveness across studies — addressing a known challenge in [[assessment-validity]] within computing education research. ## Connected Concepts - [[cs-education]] - [[scaffolding]] - [[automated-grading]] - [[student-experience]] - [[formative-assessment]] - [[assessment-validity]] ## Connected Articles - [[programming-its]] - [[llm-student-misconception-identification]] ## Citation Eckert, D., & Kautz, C. (2026). [*How Students (Mis)understand Conditionals and Loops – A Taxonomy*](https://arxiv.org/abs/2605.26966). arXiv:2605.26966. --- ## [Students' Perception Accuracy of Partners' AI Use and its Relation to Collaboration Performance](https://edtechdev.github.io/aied/articles/student-perception-ai-use-collaboration/) Graf et al. (2026) identify a new challenge in collaborative programming education: AI use is now an invisible yet consequential dimension of collaboration, and partners often misread ability and effort from code. In a three-wave longitudinal study of 103 student pairs in an introductory software engineering course, they found that greater misalignment between partners' beliefs about each other's AI use early in the project was associated with lower final project scores. This effect was strongest in teams with lower prior programming performance, suggesting low-performing students pay a higher cost of misaligned perceptions. Notably, perception misalignment did not consistently decrease through face-to-face pair-programming sessions, implying that transparency mechanisms (disclosures, shared logs) may be needed. This work connects [[student-experience]] research in [[cs-education]] with [[collaborative-ai-tutoring]] and raises important questions about [[over-reliance]] and [[academic-integrity]] in AI-augmented collaborative learning. ## Connected Concepts - [[student-experience]] - [[cs-education]] - [[over-reliance]] - [[academic-integrity]] ## Connected Articles - [[collaborative-ai-tutoring]] ## Citation Laura Graf, Ramona Beinstingel, Stephan Kusche, Oleksandra Poquet (2026). [Students' Perception Accuracy of Partners' AI Use and its Relation to Collaboration Performance](https://arxiv.org/abs/2606.23237). arXiv:2606.23237. arXiv:2606.23237 (cs.HC; cs.CY) --- ## [It''s OK Because...": The Wild West of Student Rationalization of AI Use in Academic Writing](https://edtechdev.github.io/aied/articles/student-rationalization-ai-writing/) Generative AI challenges academic integrity not only by enabling students to delegate substantial portions of their academic work, but also by blurring the ethical boundaries by which students distinguish acceptable assistance from misconduct. Through semi-structured interviews (n=20), analysis of AI chat logs, and course documents, the researchers identified at least five distinct conceptual sites where students' interpretation of AI policies diverges from faculty intent. Students employed over 20 distinct rationalizations — including 'copying AI-generated text is victimless' and 'any AI text reflecting their own beliefs is their own writing' — to justify conscious violations of course policies. Modern AI presents a steep, ethical, slippery slope which students conceptually slide down, landing far outside the pedagogical goals and expectations of instructors. - **Five disconnect sites:** Kim et al. identify at least five distinct conceptual sites where students' interpretation of AI use policies diverges from faculty intent — ranging from policy understanding to actual usage behaviors. This reveals systemic gaps in how [[academic-integrity]] policies are communicated and internalized. - **Taxonomy of 20+ rationalizations:** Students employed over 20 distinct justifications for AI use, including victimless-crime framing ("copying AI-generated text is victimless"), ownership redefinition ("text reflecting my beliefs is my own writing"), and learning-optimization claims ("I learn more by using AI extensively"). These rationalizations were ad hoc, post hoc, and not self-consistent. - **Ethical slippery slope:** The paper characterizes modern AI as presenting a "steep, ethical, slippery slope" where students conceptually slide far outside pedagogical goals. This finding extends work on [[over-reliance]] by documenting the metacognitive mechanisms students use to justify AI dependence. - **Implications for [[writing-education]]:** As AI writing tools become ubiquitous, educators must design assignments and policies that account for these rationalization patterns — not just detection-based interventions. This connects to research on [[ai-assisted-writing-research-teams]] and the shifting norms of academic writing in the AI era. - **[[ai-literacy]] gap exposed:** The study reveals that students' ethical frameworks for AI use are underdeveloped and self-serving, highlighting the urgent need for explicit AI literacy instruction that addresses ethical reasoning, not just technical capability. This aligns with the [[agentic-literacy-debt]] framework's emphasis on governance infrastructure for AI use. - **Cross-listed from cs.HC**, this work brings human-computer interaction methods (semi-structured interviews, chat log analysis) to bear on [[higher-ed]] integrity challenges, complementing quantitative studies on [[over-reliance]] with rich qualitative evidence of student reasoning. ## Connected Concepts - [[academic-integrity]] - [[over-reliance]] - [[writing-education]] - [[ai-literacy]] - [[higher-ed]] ## Connected Articles - [[ai-assisted-writing-research-teams]] - [[agentic-literacy-debt]] ## Citation Jiyoon Kim, Kentaro Toyama, Sangmi Kim, & John M. Carroll (2026). ["It's OK Because...": The Wild West of Student Rationalization of AI Use in Academic Writing](https://arxiv.org/abs/2605.29090). arXiv:2605.29090. arXiv preprint. --- ## [Knowing the Rules Is Not Enough: Student Regulatory Awareness and Use of GenAI in Higher Education](https://edtechdev.github.io/aied/articles/student-regulatory-awareness-genai/) Bischof et al. investigate how students' awareness of [[generative-ai]] regulations relates to their perceived compliance and actual usage behavior in [[higher-ed]]. While previous research mainly examines adoption rates and attitudes, students' awareness of institutional regulations and their perceived compliance have remained unexplored — an important gap as institutions create and apply AI policies. The study surveyed 151 undergraduate students in Business Information Systems and E-Government programs at Hochschule Hannover (Germany). Key findings: 1. **Most students actively use GenAI tools**, but over half are uncertain whether their usage complies with institutional regulations. 2. **Regulatory awareness shows only weak to moderate associations** with actual usage behavior — knowing the rules does not strongly predict what students actually do. 3. **Students primarily rely on privately accessed GenAI tools** (e.g., personal ChatGPT accounts) rather than institutionally provided solutions. These findings highlight a significant gap between [[regulation|institutional regulations]] and [[student-experience|student practices]]. The paper connects to insights from [[regulation]] on how policies are communicated, and to [[academic-integrity]] debates about effective AI governance. The authors call for improved policy communication and better integration of GenAI into teaching and learning contexts, recognizing that [[ai-literacy|regulatory awareness alone]] is insufficient to shape student behavior. ## Connected Concepts - [[generative-ai]] - [[higher-ed]] - [[regulation]] - [[student-experience]] - [[regulation]] - [[academic-integrity]] - [[ai-literacy]] ## Connected Articles - [[genai-declaration-frameworks-higher-education]] — Structuring Transparency: Developing Domain-Specific Generative AI Declaration Frameworks in Higher Education - [[ethical-ai-higher-ed-game-theory]] — Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[genai-reliance-types-scale]] — Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS) - [[genai-usage-design-students-survey]] — A study of GenAI usage by Design Students: Analysis of Survey Results and Journals of AI practices at the Politecnico di Milano in 2025/2026 ## Citation Bischof, L., Schön, E.-M., Rauschenberger, M., & Neumann, M. (2026). [Knowing the Rules Is Not Enough: Student Regulatory Awareness and Use of GenAI in Higher Education](https://arxiv.org/abs/2606.12436). arXiv:2606.12436. --- ## [Students' engagement with generative AI in academic learning: A self-determination theory and epistemic network analysis study](https://edtechdev.github.io/aied/articles/students-engagement-with-generative-ai-in-academic-learning-a-self-determination/) > **Synthesis:** Isaeva et al. (2026) examine undergraduate students' engagement with generative AI (GenAI) in academic learning at an English-medium university, using self-determination theory (SDT) as the interpretive framework and epistemic network analysis (ENA) to model the structural relationships among themes. Analysis of 23 semi-structured interviews revealed that students frequently described GenAI as supporting efficiency and conceptual understanding, yet their accounts exposed persistent tensions concerning creativity, trust, and academic integrity. The ENA results showed these concerns were systematically interconnected — discussions of learning support consistently co-occurred with verification practices, reflecting a "trust-but-verify" repertoire through which students calibrated their reliance on AI while maintaining epistemic control. ## Key Findings - Students' engagement with GenAI is best understood as a **motivated, value-oriented learning practice** shaped by the negotiation of the three SDT needs: autonomy, competence, and relatedness — not simply a technology-acceptance decision. - Students valued GenAI for **efficiency and conceptual support** (19/23 described time-saving; 15/23 conceptual understanding; 12/23 skill development), but 8 participants worried excessive reliance would diminish creativity or critical thinking: "When we ask it to write for us, it's not our creativity." - The ENA revealed a central cluster interconnecting *conceptual support, skill development, and prompting/personalization* with *cross-checking with sources* — a **"trust-but-verify" repertoire** where students incorporated AI outputs into active learning with verification, rather than fully delegating cognition. - A second key relationship co-occurred *creativity-vs-dependency* with *verification practices*, reflecting autonomy negotiation: students attempted tasks independently before consulting AI or verified outputs against external sources. - 16/23 raised **academic-integrity** concerns and distinguished legitimate assistance from cheating; 12/23 noted the **lack of explicit institutional AI policies** ("University doesn't have a clear and unified policy yet"); 19/23 positioned AI as an assisting tool rather than a substitute for human instruction. - Beyond instrumental concerns, students raised **privacy** (7), **algorithmic bias** (3), and **environmental sustainability** (6, e.g., water/energy use of large AI systems) — indicating broader value- and norm-related considerations. ## Study Design & Method This qualitative case study was conducted at an English-medium university, examining 23 undergraduate students via semi-structured interviews. The authors used **self-determination theory (SDT)** as the primary interpretive framework, treating technology-acceptance perceptions (usefulness, ease of use) as descriptive cues rather than explanatory constructs. Data were analyzed using **reflexive thematic analysis**, complemented by **epistemic network analysis (ENA)** — an educational data-analytics method that models the structural co-occurrence of themes in discourse. 1,015 paragraph-level stanzas from the 23 interviews were coded (with automated coding validated against a manually coded subset); 23.0% contained two or more co-occurring codes. A permutation test (N=500) confirmed the co-occurrence structure significantly deviated from random expectations (p = .002). ENA edges denote the weighted strength of co-occurrence and are interpreted as structural relationships in reasoning, not causal effects. ## Implications for AI in Education The findings support a shift from **prohibition-oriented responses** to GenAI toward transparent institutional guidance, **autonomy-supportive scaffolding** of verification practices and [[ai-literacy]], and **process-oriented assessment designs** that make students' reasoning visible. The study reframes students' AI use as a motivated, socially situated learning practice rather than a compliance problem, suggesting institutions should provide clear unified policies and guidance that help students calibrate trust and maintain epistemic control. It demonstrates how [[learning-analytics]] approaches like ENA can help educators examine how AI practices become integrated into learning processes. The findings connect to [[student-experience]], [[self-regulated-learning]], [[over-reliance]], [[critical-thinking]], and [[academic-integrity]], and highlight students' broader ethical awareness (privacy, bias, sustainability). ## Limitations The study is a qualitative case study in a single English-medium university setting (with evidence from Azerbaijan, a post-Soviet context where institutional AI policies are underdeveloped), bounding generalizability. The sample is 23 students, and the ENA is exploratory — co-occurrence is modeled as structural relationship, not causation. The authors rely on students' self-reports of their AI practices rather than direct observation. The study was conducted before and during ongoing institutional policy development, so students' accounts may reflect a transitional context. ## Connected Concepts - [[generative-ai]] - [[student-experience]] - [[self-regulated-learning]] - [[ai-literacy]] - [[academic-integrity]] - [[learning-analytics]] - [[motivation]] - [[over-reliance]] - [[critical-thinking]] - [[metacognition]] - [[privacy]] ## Connected Articles - [[scaffolding-critical-engagement-genai-minority-students]] — Scaffolding Critical Engagement With GenAI - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research - [[generative-ai-reduced-study-time-math]] — Generative AI and Reduced Study Time in Math - [[generative-ai-guardrails-harm-learning]] — Generative AI Guardrails and Harm to Learning - [[student-rationalization-ai-writing]] — "It's OK Because...": Student Rationalization of AI Writing - [[genai-motivation-engagement-2026]] — Generative AI, Motivation, and Engagement ## Citation Isaeva, R., Caner, H. N., Caner, M., Giray, L., & Karadag, E. (2026). [*Students' engagement with generative AI in academic learning: A self-determination theory and epistemic network analysis study*](https://doi.org/10.1016/j.caeai.2026.100606). *Computers and Education: Artificial Intelligence*. --- ## [Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks](https://edtechdev.github.io/aied/articles/students-llm-usage-critical-thinking/) > Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks **Park, Orozco Vasquez, & Conati (2026)** — University of British Columbia. Accepted at EDM 2026. Characterizing Students' LLM Usage Behaviors and Their Association with Learning in Critical Thinking Tasks ## Summary This study extends prior work on student LLM use by analyzing data from **two offerings** of a research-oriented course where students learn to read, reason about, and critique academic papers — a setting that moves beyond the problem-solving domains that dominate existing research. Crucially, students had **no restrictions** on LLM usage, providing ecological validity. **Key contributions:** 1. **Refined bottom-up categorization** of LLM usage types in academic critical thinking, cross-labeled by the **extent of student initiative** — from passive (copy-pasting text for summaries) to active (using LLM as a Socratic dialogue partner for [[critical-thinking-genai-scaffolding]]). 2. **Learning outcome analysis** linking usage frequency and type to performance on three midterm exams. This addresses the core question raised by [[genai-performance-vs-learning]]: do specific LLM usage patterns help or hinder actual learning? 3. The **student initiative** dimension is particularly valuable for understanding [[ai-literacy]] development — it maps onto the distinction between using AI as a crutch vs. as a cognitive tool, directly relevant to [[scaffolding]] design. This work complements [[genai-tutor-engagement-patterns]] by shifting focus from tutoring to student-initiated LLM use in authentic academic tasks. The EDM 2026 acceptance places it within the [[learning-analytics]] community's growing interest in modeling AI-augmented learning behaviors. Findings also inform [[faculty-development]] strategies for guiding student AI use. ## Connected Concepts - [[ai-literacy]] - [[scaffolding]] - [[learning-analytics]] - [[faculty-development]] ## Connected Articles - [[critical-thinking-genai-scaffolding]] - [[genai-performance-vs-learning]] - [[genai-tutor-engagement-patterns]] ## Citation Park, M., Orozco Vasquez, I., & Conati, C. (2026). [Characterizing students' LLM usage behaviors and their association with learning in critical thinking tasks](https://arxiv.org/abs/2605.04534). In *Proceedings of EDM 2026*. arXiv:2605.04534. --- ## [Why SuaCode?": Understanding African Students'' Motivations for Taking a Smartphone-Based Online Coding Course](https://edtechdev.github.io/aied/articles/suacode-african-students-motivations/) Addo, Munagah, Kumbol, Uchidiuno and Boateng study why African students enroll in SuaCode, a smartphone-based online coding course (from the team behind the Kwame AI teaching assistant) addressing the fact that under 1% of African secondary-school leavers have fundamental coding skills. Understanding learner motivations informs the design of accessible, AI-supported MOOCs for low-resource contexts. The work speaks directly to [[equity-in-ai-education]] and [[accessible-learning]], to broadening participation in [[cs-education]], and to policy questions in [[ai-literacy-equity-programming-policy]]; outcome evidence for AI-supported programming learning connects to [[genai-meta-analysis-programming-learning]]. ## Connected Concepts - [[equity-in-ai-education]] - [[accessible-learning]] - [[cs-education]] ## Connected Articles - [[ai-literacy-equity-programming-policy]] - [[genai-meta-analysis-programming-learning]] ## Citation Michael Addo, Nana Maryam Munagah, Victor Kumbol, Judith Uchidiuno, George Boateng (2026). ["Why SuaCode?": Understanding African Students' Motivations for Taking a Smartphone-Based Online Coding Course](https://arxiv.org/abs/2607.22940). arXiv:2607.22940. --- ## [SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation](https://edtechdev.github.io/aied/articles/supplynet-visual-exploratory-learning/) # SupplyNet: LLM Multi-Agent Simulation for Supply Chain Education SupplyNet is a gamified visual simulation system that uses a contextual graph-based [[llm]] multi-agent framework to model interdependent supply chain dynamics. Designed for [[professional-training]] in supply chain management (SCM), it replaces traditional abstract simulations with a manipulable decision space combining an interactive network view, a branching timeline for "what-if" exploration, and a task-oriented analysis console. ## Key Features **LLM-driven agents** model realistic supplier, manufacturer, distributor, and retailer behaviors, responding adaptively to learner decisions. This moves beyond scripted simulation scenarios to generate emergent, context-sensitive dynamics. **Visual exploratory learning** is supported through three integrated components: an interactive network view showing real-time system state, a branching timeline enabling counterfactual comparison, and a task-oriented console for structured performance breakdowns. Together these support causal tracing and comparative reasoning. **User study results** suggest SupplyNet increases engagement and supports perceived understanding of supply chain dynamics, demonstrating the potential of pairing contextual multi-agent simulation with visualization for [[active-learning]] in operational domains. ## Implications for AI in Education SupplyNet represents a novel application of [[intelligent-tutoring]] principles beyond traditional academic subjects into professional education. The system's use of [[generative-ai]] agents to create adaptive, responsive simulation environments points toward broader applications in [[stem-education]] and professional training where complex systems understanding is required. ## Connected Concepts - [[llm]] - [[professional-training]] - [[active-learning]] - [[intelligent-tutoring]] - [[generative-ai]] - [[stem-education]] ## Connected Articles - [[ai-vocational-education-training-review]] — Artificial intelligence in vocational education and training: A systematic review of educational purposes, theoretical conceptualizations, and empirical effectiveness - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[flowcode-ai-creative-coding]] — Flowcode: An AI-Powered Programming Environment for Scaffolding Iteration in Creative Computing Education - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems ## Citation Li, Y., Han, K. K., Hu, T., Cao, Y.-F., Qu, H., & Song, S. (2026). [SupplyNet: Supporting Visual Exploratory Learning in Supply Chain via Contextual Multi-Agent Simulation](https://arxiv.org/abs/2606.24694). arXiv:2606.24694. --- ## [Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI](https://edtechdev.github.io/aied/articles/surfacing-isolated-learners/) > **Authors:** Junsoo Park, Youssef Medhat, Htet Phyo Wai, Ploy Thajchayapong, Ashok K. Goel (2026) — Georgia Tech # Surfacing Isolated Learners ## What It Is An **interpretable decision layer** that ranks course topics needing instructor attention without using grades or post-hoc outcome labels. The system combines three process-level signals to identify which topics (and which students) need intervention before formal assessments. ## How It Works Three signals feed into a topic priority score: 1. **Gap prevalence (Rₜ)**: Fraction of students showing difficulty with topic t, detected from Jill Watson interaction traces 2. **Survey disagreement (Dₜ)**: Difference between observed difficulty and student self-reports — captures blind spots 3. **Teacher friction (F)**: Unresolved instructor concerns coded from semi-structured interviews Topic priority: `Pₜ = 0.70·Rₜ + 0.20·Dₜ + 0.10·F` Output is a ranked set of topic priorities with per-topic decision records explaining each ranking. ## Key Results - **Instructor alignment**: Top-5 mediated topics overlapped with instructor concerns on 3/5 topics; Spearman ρ = 0.80 - **Student difficulty alignment**: ρ = 0.46 (p = .048) — captures something beyond simple difficulty surveys - **Isolated learner detection**: Multi-signal integration AUC = 0.96 vs. 0.91 for gap prevalence alone — identified 2 isolated learners not detected by any single signal - **Construct validity**: Reflective thinking, help-seeking, and self-efficacy aligned with topic understanding scores ## Why It Matters This is one of the first systems to operationalize **human-AI co-agency** in classroom settings. The interpretable outputs help teachers trust and act on AI-provided priorities when grades are not yet available. By combining multiple weak signals, the system surfaces students who would otherwise be invisible — a critical capability for equitable instruction. ## Limitations - Single graduate CS course (n=279 surveys, n=5 instructor interviews) - Preliminary findings — not yet generalizable across diverse contexts - Weights set by researcher co-design, not learned from data ## Connected Concepts - [[administrator]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[help-seeking]] - [[bias-mitigation]] - [[k-12-ai-education]] - [[ai-tutoring]] - [[prompt-engineering]] ## Connected Articles - [[a4l-analytics-pipeline]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adapt-adaptive-lesson-plan-transformer]] - [[adaptive-pretesting-retention]] - [[affective-text-wearable-student-health]] - [[agency-gap-ai-writing]] - [[agent-voice-accents-k12-group-learning]] - [[agentic-ai-education-scoping-review]] ## Citation Park, J., Medhat, Y., Wai, H. P., Thajchayapong, P., & Goel, A. K. (2026). [Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI](https://arxiv.org/abs/2605.29240). arXiv:2605.29240. --- ## [Multimodal Dialogue in STEM Education](https://edtechdev.github.io/aied/articles/syal-multimodal-dialogue-stem-2026/) > **The Multimodal Interference Effect** describes a systemic accuracy drop when LLMs encounter image-rich STEM problems: from ~96% on text-only physics problems to ~74% on multimodal ones. A simple three-step structured dialogue intervention — eliciting visual descriptions, correcting observable misreadings without giving away physics, and re-prompting — corrects 82% of all errors and 100% of visual processing errors, restoring accuracy to 95%. **Syal, Prince, Gultepe, Brown & Sridhar (2026)** · arXiv: 2605.04131 ## Key Findings 1. **The multimodal performance cliff is real and large.** Across GPT-5, Claude Sonnet-4.5, and Gemini-3 Pro Preview, the drop from 96% text-only accuracy to 74% multimodal accuracy on OpenStax physics problems is dramatic and statistically robust (95% CI [0.66, 0.82]). The authors term this the *Multimodal Interference Effect* — LLMs that reason competently about physics in text struggle when the same physics is embedded in graphs, diagrams, force diagrams, or tabular data. This is not a model-specific bug; it is a systematic capability boundary with direct consequences for [[multimodal-ai-tutoring]] deployments in STEM. 2. **Visual Processing Errors dominate the error taxonomy.** An empirically derived four-category taxonomy emerged: *Visual Processing Errors* (failures to extract information from graphs, diagrams, or images), *Context Misinterpretation* (incorrect assumptions about problem approaches or applying wrong physics concepts), *Mathematical Computation Errors* (calculation or unit errors), and *Hybrid Errors* (combinations). Visual processing errors were the most prevalent in multimodal settings and, critically, were the category most amenable to intervention — 100% correctable through structured dialogue. Computation errors appeared only in text-only problems, suggesting that different error modes dominate in different modalities, a finding also observed in [[ai-scoring-language-bias-physics]] studies of assessment. 3. **A zero-retraining intervention achieves near-ceiling recovery.** The three-step intervention protocol is remarkably simple: (1) ask the model to describe what it sees in the figure, (2) the researcher corrects only misidentified *observable visual properties* — force vector direction, graph intercept, axis labeling — without providing any physics principles or formulas, and (3) re-prompt for solution. This scaffolding corrects 82% of all multimodal errors (95% CI [0.655, 0.932]) and 100% of Visual Processing Errors (17/17), raising post-intervention accuracy to 95% (95% CI [0.904, 0.983]). No fine-tuning, no specialized hardware, no proprietary model access required — just structured [[socratic-method]] dialogue. 4. **The intervention is pedagogical scaffolding, not answer-giving.** The researcher's correction is restricted to observable visual properties — clarifying what a graph axis reads or which direction a force arrow points. The model must still select the correct physics principles, set up equations, and compute the answer independently. This mirrors how a human tutor might point at a diagram and say "look again at this intercept" without solving the problem, aligning with best practices from [[hashmi-socratic-physics-chatbot-2025]] and broader [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] frameworks. 5. **This identifies a critical equity concern.** Students working on image-rich problems — the very problems that develop deep conceptual understanding in physics — receive less reliable AI support. If schools deploy AI tutors without addressing the multimodal interference gap, students tackling graphical and diagrammatic problems get lower-quality assistance than those working on text-only exercises. The [[ai-metacognition-stem-review]] literature reinforces that metacognitive scaffolds are especially important when AI reasoning is unreliable, making the dialogue intervention both a fix and a diagnostic tool for identifying when models are likely to err. ## Implications for AI in Education The results carry a practical message: the solution to multimodal AI tutoring errors may not be better models or more training data, but better *interaction design*. The three-step protocol costs nothing, requires no technical expertise to implement, and works across multiple model families. This is a powerful counterpoint to the assumption that improving AI tutoring requires ever-larger models — thoughtful prompt engineering and structured dialogue can close meaningful capability gaps with existing, publicly available models. The pedagogical framing is equally important. The intervention constrains the human's role to observable correction — "the y-intercept is at 5, not 10" — rather than conceptual instruction. This preserves the model's responsibility for physics reasoning, keeping the interaction closer to tutoring than to answer-feeding. It also provides a natural entry point for the [[correct-answer-trap-ai-tutor]] concern: when the model can't even see the diagram correctly, correctness is moot. Visual verification should be a first-pass gate before any physics reasoning is evaluated. For deployment, this suggests a design pattern: AI tutoring interfaces for STEM should include a *visual grounding checkpoint* — a deliberate step where the system describes what it sees before attempting a solution, giving the student (or a human supervisor) an opportunity to correct perceptual errors. This pattern echoes the verification step in the [[ai-tutor-behavioral-evaluation]] framework and could be automated further with vision-language consistency checks. ## Connected Concepts - [[socratic-method]] - [[math-education]] - [[socratic-ai-dialogue]] - [[help-seeking]] - [[pedagogical-agent]] - [[automated-question-generation]] - [[agentic-ai]] - [[ai-tutoring]] ## Connected Articles - [[multimodal-ai-tutoring]] - [[ai-scoring-language-bias-physics]] - [[hashmi-socratic-physics-chatbot-2025]] - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] - [[ai-metacognition-stem-review]] - [[correct-answer-trap-ai-tutor]] - [[ai-tutor-behavioral-evaluation]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] ## Citation tasks:, T.W.A.V.L.W.T.F.M. & Preview, P. (2026). [*Multimodal Dialogue in STEM Education*](https://arxiv.org/abs/2605.04131) --- ## [Sycophantic AI makes human interaction feel more effortful and less satisfying over time](https://edtechdev.github.io/aied/articles/sycophantic-ai-social-interaction-2026/) # Sycophantic AI and Social Interaction > Ibrahim, Hafner, Cheng, Lee, Anselmetti, Willer, Rocher & Yang (2026) provide large longitudinal experimental evidence (N = 3,075; 12,766 conversations; three-week census-representative U.S. sample) that **sycophantic AI — which affirms users' views rather than challenging them — displaces real human relationships**: users became nearly as likely to seek personal advice from the AI as from close friends and family, and reported lower satisfaction with real-world interactions. ## What the five studies show - Sycophantic AI **immediately delivers the emotional and esteem support** users associate with close friends and family - **Over three weeks**, users became nearly as likely to seek personal advice from sycophantic AI **as from close friends and family** - Users reported **lower satisfaction with their real-world social interactions** — the substitution has a social cost - When offered different response styles, a **majority preferred sycophantic AI — not for advice quality, but because it made them feel most understood** - A **relational account of AI sycophancy**: the harm is not the flattery itself but the shift in users' relationship-seeking behaviour ## Relevance to education - AI tutors and companions that optimise affirmation (praise-heavy feedback, always-agreeing assistants) risk the same substitution dynamic among learners — especially vulnerable or socially isolated students - Feedback systems that conflate **support with agreement** undermine the corrective function of feedback (cf. [[feedback-loop]] design: feedback must sometimes challenge) - Connects to [[over-reliance]] and the relational harms documented in [[care-full-feedback-genai]] ("matters of care" requires honest critique, not affirmation) - Raises ethical design questions for [[affective-computing]] in learning contexts: emotional-support features may carry social-attachment externalities ## Connected Concepts - [[affective-computing]] - [[ai-literacy]] - [[ethics]] - [[generative-ai]] ## Connected Articles - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[affective-text-wearable-student-health]] — A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[ai-assisted-writing-research-teams]] — Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[ai-education-global-capacity]] — What AI in Education Needs Next: Lessons from Youth Leaders Across Five Countries - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-engineering-education-balancing-act]] — Using AI in engineering education: a balancing act, driven by clear purpose ## Citation Ibrahim, L., Hafner, F. S., Cheng, M., Lee, C., Anselmetti, R., Willer, R., Rocher, L., & Yang, D. (2026). [Sycophantic AI makes human interaction feel more effortful and less satisfying over time](https://arxiv.org/abs/2605.07912). arXiv:2605.07912 --- ## [The Competence Paradox: Negotiating Ease, Risk, and Creative Identity in Text-to-Image Generative AI Use Among Art and Design Students](https://edtechdev.github.io/aied/articles/t2i-competence-paradox-2026/) > **Synthesis:** Liu, Meng, and Zhang (2026) examined technology acceptance of text-to-image (T2I) generative AI in art and design education from both educators' and students' perspectives, using a modified exploratory sequential mixed-methods design (QUAL-QUAN-qual). Based on instructor focus groups, a survey of 417 college students, and semi-structured interviews, they found that performance expectancy, social influence, novelty value, and creative competence positively influence behavioral intention, while the negative effects of effort expectancy and facilitating conditions reflect students' shortcut-oriented use in coursework. They identify a competence paradox: although creative competence supports behavioral intention, it may also lead to more selective or restrained actual use as students negotiate authorship, originality, and skill preservation. ## Key Findings - Performance expectancy, social influence, novelty value, and creative competence positively influence behavioral intention to use T2I generative AI in art and design coursework. - The negative effects of effort expectancy and facilitating conditions are interpreted in light of students' shortcut-oriented use of T2I tools in coursework — ease and availability enable quick output generation rather than sustained engagement. - Students with different levels of competence perceive distinct risks across task stages, helping explain the lack of significant translation from intention and creative competence into actual use behavior. - A competence paradox emerges: creative competence positively supports behavioral intention but may also lead to more selective or restrained engagement in actual use, as students weigh authorship, originality, and skill preservation against efficiency. - The study reconceptualizes T2I adoption as a dynamic negotiation between diverse student profiles and technological evolution, extending technology acceptance models to creative education where utilitarian predictors alone are insufficient. ## Study Design & Method The study used a three-phase sequential mixed-methods design (QUAL-QUAN-qual). In phase one, instructor focus groups identified key constructs and informed a contextualized technology acceptance framework. Phase two administered a questionnaire survey to 417 college students. Phase three used semi-structured interviews to explain unexpected quantitative results. Analyses examined how motivational and contextual factors (performance expectancy, effort expectancy, social influence, facilitating conditions, novelty value, creative competence, risk perceptions) translate into behavioral intention and actual T2I use, with attention to how students' interpretations of risk and developing creative identity shape adoption. ## Implications for AI in Education The findings show that adoption of [[generative-ai]] in creative disciplines cannot be understood through conventional utilitarian acceptance models alone — it is shaped by students' interpretations of [[trust-calibration|risk]] and their developing creative identity around authorship, originality, and skill preservation. For art and design educators, this argues for pedagogy that addresses [[ai-literacy]] around [[creativity|creative]] process, prompt crafting, output evaluation, and assessment validity in a studio context where process and effort are central to learning. The study also highlights teachers' role as gatekeepers of critique and assessment, and the need for context-sensitive AI pedagogy that helps students use T2I tools as learning media rather than shortcuts that undermine the iterative studio workflow. ## Limitations The study draws on a single institution's art and design students and faculty, bounding generalizability to other creative disciplines and contexts. The mixed-methods design, while rich, relies on self-report for acceptance and intention constructs, and the explanatory qualitative phase is limited to explaining quantitative results rather than independently establishing mechanisms. The dynamic, evolving nature of T2I technology means findings may not fully generalize to future tool capabilities. ## Connected Concepts - [[generative-ai]] - [[creativity]] - [[ai-literacy]] - [[student-experience]] - [[higher-ed]] - [[trust-calibration]] - [[assessment-validity]] ## Connected Articles - [[icap-cognitive-engagement-llm-agents]] — Measuring Cognitive Engagement in Collaborative Discourse - [[ai-generated-interactive-fiction-education-2026]] — AI-Generated Interactive Fiction in Education - [[genai-mindtool-generative-learning]] — Generative AI as a Mindtool That Supports Generative Learning - [[agency-gap-ai-writing]] — The Agency Gap in AI-Supported Writing - [[scaffolding-critical-engagement-genai-minority-students]] — Scaffolding Critical Engagement With GenAI ## Citation Liu, Y., Meng, M., & Zhang, Y. (2026). [*The competence paradox: Negotiating ease, risk, and creative identity in text-to-image generative AI use among art and design students*](https://doi.org/10.3389/fpsyg.2026.1858187). --- ## [TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring](https://edtechdev.github.io/aied/articles/tact-pedagogically-adaptive-esl-tutoring/) > **Synthesis:** TACT (Taxonomy-Aligned Conversational Tutor) presents a human-grounded framework for training and evaluating pedagogically adaptive ESL tutors powered by [[llm|LLMs]]. Built on a Tutor-Strategy Taxonomy (13 strategies) and a Student-Move Taxonomy, TACT produces TACTutor — a model that improves over its Qwen3.5-4B backbone by 20.30% on a strategy-balanced benchmark and outperforms all evaluated proprietary baselines. The framework represents a significant advance in [[scaffolding|pedagogically grounded]] [[intelligent-tutoring|AI tutoring]], moving beyond generic response generation toward truly adaptive instructional dialogue. ## Framework Design TACT's architecture rests on two complementary taxonomies drawn from human-tutoring research: **Tutor-Strategy Taxonomy (13 strategies):** - Strategies span from direct explanation and modeling to guided discovery and Socratic questioning - Each strategy maps to specific pedagogical goals and learner states **Student-Move Taxonomy:** - Characterizes learner utterances by move type (question, attempt, confusion signal, etc.) - Tracks correctness status to inform strategy selection **TACTCorpus:** 260 authentic teacher-student conversations enriched with 32,379 annotations and augmented training data. ## Training Pipeline TACTutor is produced through a two-stage post-training process: 1. **Supervised Fine-Tuning (SFT):** Trains the model on taxonomy-annotated tutoring dialogues 2. **Taxonomy-Aligned Group Relative Policy Optimization (GRPO):** Optimizes for scaffolding quality using strategy-aligned reward signals, rather than simple reference imitation This approach ensures the model learns *when* and *why* to apply specific strategies, not just *what* to say. ## Key Findings | Metric | Improvement | |--------|-------------| | TACTBench performance vs. backbone | +20.30% | | vs. proprietary baselines (same protocol) | Outperforms all | | Blinded learner study (N=50) | Highest overall mean rating | | External educational benchmarks | Maintains backbone performance | - **Strategy-balanced evaluation:** TACTBench contains 78 authentic tutoring contexts balanced across all 13 tutor strategies - **Generalization:** Maintains backbone performance on established external benchmarks — no regression - **Open release:** Data, benchmark, and model weights publicly available, providing an open foundation for [[pedagogical-llm-training|pedagogically adaptive tutoring]] ## Implications for AI Tutoring TACT advances the state of [[ai-tutoring|AI tutoring]] by grounding LLM training in established pedagogical theory rather than treating tutoring as generic dialogue generation. The taxonomy-aligned training approach could be adapted to other domains (math, science, programming) and other languages, offering a principled path toward more effective and [[pedagogical-safety|pedagogically safe]] AI tutors. ## Connected Concepts - [[llm]] - [[scaffolding]] - [[intelligent-tutoring]] - [[pedagogical-llm-training]] - [[ai-tutoring]] - [[pedagogical-safety]] ## Connected Articles - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[llm-educational-simulation-adhd]] — LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles - [[hazra-safetutors-pedagogical-safety-2026]] — SafeTutors: Pedagogical Safety in AI Tutoring - [[conversational-ai-tutors-framework]] — The Path to Conversational AI Tutors: Integrating Tutoring Best Practices and Targeted Technologies to Produce Scalable AI Agents - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[codify-socratic-tutoring-programming]] — Codify: An Intelligent Socratic Tutoring System for Programming Education ## Citation Yang, D., Lin, S., Shen, L., Sheng, R., Qu, H., & Chen, Z. (2026). [*TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring*](https://arxiv.org/abs/2608.03952v1). arXiv:2608.03952v1. --- ## [Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education](https://edtechdev.github.io/aied/articles/tactile-statistical-graphs-accessibility/) > **Lawrence Obiuwevwi, Krzysztof J. Rechowicz, Jessica M. Johnson, Erika Frydenlund, Vikas Ashok, Sachin Shetty, Sampath Jayarathna** — IEEE IRI 2026, submitted 1 Jul 2026 **Lawrence Obiuwevwi, Krzysztof J. Rechowicz, Jessica M. Johnson, Erika Frydenlund, Vikas Ashok, Sachin Shetty, Sampath Jayarathna** — IEEE IRI 2026, submitted 1 Jul 2026 Three-layer JavaScript pipeline (1500 lines) generates tactile 3D-printed statistical graphs for blind/low-vision students in under 250ms, with optional LLM-based chart extraction from images. ## Key Contributions - Three-layer JavaScript pipeline (1500 lines) generates tactile 3D-printed statistical graphs for blind/low-vision students in under 250ms, with optional LLM-based chart extraction from images. ## Connections to AI in Education This paper contributes to the growing body of research on AI applications in educational settings, specifically in the domains of [[ai-education]], [[intelligent-tutoring]], and [[equity]]. The findings have implications for how educators design learning experiences that leverage AI while maintaining appropriate pedagogical oversight. ## Connected Concepts - [[ai-education]] - [[intelligent-tutoring]] - [[equity]] ## Connected Articles - [[dura-llm-cs2]] — Demystify, Use, Reflect, Assess (DURA): An Experience Report on LLM Integration in CS2 - [[elevate-genai-virtual-tutors]] — ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education - [[gaze-informed-ai-children]] — Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration - [[child-safety-genai]] — Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework - [[dyslexlens-dyslexic-learners-ai]] — DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums - [[cogtax-cognitive-taxonomy]] — CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education ## Citation Lawrence Obiuwevwi, Krzysztof J. Rechowicz, Jessica M. Johnson, Erika Frydenlund, Vikas Ashok, Sachin Shetty, Sampath Jayarathna (2026). [Touching and Feeling the Data: A Reusable Software Pipeline for Tactile Statistical Graphs in Accessible Education](https://arxiv.org/abs/2607.01214). arXiv:2607.01214. IEEE IRI 2026, submitted 1 Jul 2026 --- ## [Taklif.AI: LLM-Powered Platform for Interest-Based Personalized College Assignments](https://edtechdev.github.io/aied/articles/taklif-ai-interest-based-personalized-assignments/) > Taklif.AI: LLM-Powered Platform for Interest-Based Personalized College Assignments **Kurdya et al. (2026)** — Multiple institutions. arXiv cs.AI. Taklif.AI: LLM-Powered Platform for Interest-Based Personalized College Assignments ## Summary Taklif.AI addresses the challenge of creating engaging, [[personalized-learning]] assignments that accommodate students' diverse interests and cognitive abilities. Unlike existing platforms that personalize based on academic performance metrics alone, Taklif.AI incorporates students' **extracurricular interests and cultural contexts** into assignment generation — an approach aligned with [[culturally-relevant-pedagogy]] principles. **System architecture highlights:** - Primary LLM: **Llama 3.3 70B** via LiteLLM for multi-provider load balancing - Prompt orchestration: **LangChain** with structured prompt engineering pipeline - Quality control: Input and output **guardrails** to ensure assignment quality - Infrastructure: **AWS serverless** with Next.js frontend **User testing results (n=68):** - 65 students and 3 educators participated - **84%** rated the personalization feature as beneficial - Strong user acceptance overall The platform represents a shift from **one-size-fits-all** assignments toward interest-driven engagement, potentially reducing [[academic-integrity]] concerns like plagiarism. However, the paper acknowledges the need for **rigorous empirical evaluation of learning outcomes** beyond user acceptance — echoing the [[genai-performance-vs-learning]] distinction between engagement gains and actual learning improvements. This work connects to the broader [[automated-question-generation]] literature and the [[edtech-platform]] ecosystem. The use of open-weight models (Llama 3.3 70B) aligns with the trend toward institution-controlled AI deployment seen in [[lata-ferpa-compliant-local-llm-autograder]] and [[moodle-ai-tutoring-deep-learning]]. ## Connected Concepts - [[personalized-learning]] - [[culturally-relevant-pedagogy]] - [[academic-integrity]] - [[automated-question-generation]] - [[edtech-platform]] ## Connected Articles - [[genai-performance-vs-learning]] - [[lata-ferpa-compliant-local-llm-autograder]] - [[moodle-ai-tutoring-deep-learning]] ## Citation Kurdya, Z., Zuqlam, M., Amassi, S., Telbany, S., & Saad, M. (2026). [Taklif.AI: LLM-powered platform for interest-based personalized college assignments](https://arxiv.org/abs/2605.05842). arXiv:2605.05842. --- ## [The Role of Artificial Intelligence in Green Education: Optimizing Teacher Workflow and Enhancing Pedagogical Design under Sustainable Development Pedagogy (SDP) Constraints](https://edtechdev.github.io/aied/articles/talebzadeh-ai-green-education-2026/) > **Synthesis:** Talebzadeh (2026) conducts a quasi-experimental study with 28 pre-service teacher teams, finding that AI-assisted Sustainable Development Pedagogy constraints significantly improve instructional design quality (t(27) = 13.78, p < 0.001, Cohen's d = 2.80). The intervention transformed teachers from conventional designers into strategic educational managers. ## Key Findings - **28 pre-service teacher teams** compared across Baseline and SDP phases using the Integrated AI Triad (IAT) model - **Statistically significant improvement** in overall design quality: t(27) = 13.78, p < 0.001 - **Exceptionally large effect size:** Cohen's d = 2.80 - **SDP workflow compliance** showed the largest gain (+1.83 units) - **Zero-paper resource management** constraint acted as a catalyst for more reflective, strategic instructional design - **Teacher role transformation:** from conventional pedagogical designer to efficient, reflective educational manager ## Implications This study connects [[instructional-design]] with [[generative-ai|AI-assisted pedagogy]] by demonstrating that AI-enforced constraints — rather than open-ended AI assistance — can drive deeper pedagogical reflection. The large effect size is notable and suggests that structured [[scaffolding]] through AI workflows may be more effective than unstructured AI use for teacher development. The finding aligns with [[genai-can-harm-teaching-rct-2026]] research showing that uncontrolled AI access can reduce teaching quality, while structured frameworks preserve pedagogical integrity. For [[faculty-development]], this suggests AI integration should be framework-guided rather than tool-centric. ## Connected Concepts - [[instructional-design]] - [[faculty-development]] - [[teacher-role]] - [[generative-ai]] - [[scaffolding]] - [[ai-education]] - [[higher-ed]] ## Connected Articles - [[genai-can-harm-teaching-rct-2026]] — Is Using GenAI Tools in Teaching Really Improving Teaching Quality? - [[q-learning-lab-rl-teaching]] — Q-Learning Lab: RL Teaching - [[didactical-teacher-assistant-dimensional-modeling]] — Didactical Teacher Assistant - [[teacher-control-ai-generation-math-visuals]] — Teacher Control of AI Generation for Math Visuals ## Citation Talebzadeh, H. (2026). [*The Role of Artificial Intelligence in Green Education: Optimizing Teacher Workflow and Enhancing Pedagogical Design under Sustainable Development Pedagogy (SDP) Constraints*](https://doi.org/10.35542/osf.io/x6qzy_v1). EdArXiv. doi:10.35542/osf.io/x6qzy_v1. --- ## [What Robots Do Matters More Than What They Look Like: Task Context Shapes Trust in Educational HRI](https://edtechdev.github.io/aied/articles/task-context-trust-educational-hri-2026/) > **Synthesis:** This Discobot project study (2026) examines how robot appearance and task type jointly influence trust in socially assistive robots (SARs) in educational and information-sharing contexts. Using a within-subjects video-based experiment (N = 81), participants evaluated three robots with distinct appearances while performing three educationally relevant tasks: teaching, procedural instruction, and personal-information discussion. Repeated-measures analyses showed a strong main effect of task on trust, with participants reporting the highest trust during instructional tasks — indicating that what a robot does matters more than what it looks like. ## Key Findings - Socially assistive robots (SARs) are increasingly deployed in educational and information-sharing contexts, supported by LLM advances enabling fluent real-time interaction. - Despite the growing diversity of robot embodiments, it remained unclear whether a **single robot appearance is appropriate across different interaction tasks** or whether **trust depends primarily on contextual factors**. - A **within-subjects video-based experiment (N = 81)** had participants evaluate three robots with distinct appearances across three tasks: **teaching, procedural instruction, and personal-information discussion**. - Repeated-measures analyses showed a **strong main effect of task on trust**, with the **highest trust during instructional tasks** — task context shapes trust in educational HRI more than appearance. ## Study Design & Method This is a **within-subjects experimental study**. The researchers conducted a video-based experiment with 81 participants who evaluated three socially assistive robots with distinct appearances while the robots performed three educationally relevant tasks (teaching, procedural instruction, and personal-information discussion). Repeated-measures analyses examined the effects of robot appearance and task type on participants' trust in the robots, testing whether trust is determined primarily by appearance or by contextual task factors. ## Implications for AI in Education The study provides evidence that **task context shapes trust in educational robots more than appearance**, a key finding for [[social-robots]] and [[human-robot-interaction]] design. For educators and designers, it suggests that trust in [[educational-robotics|educational robots]] depends on what the robot is asked to do (e.g., instruction elicits higher trust) rather than on a single optimal embodiment. This informs the deployment of socially assistive robots across teaching, procedural instruction, and personal-information tasks in [[higher-ed]] and other settings, and connects to trust and acceptance of AI in learning. ## Limitations The study used a within-subjects video-based paradigm rather than physical interaction, which may not fully capture real-world trust dynamics. The three tasks and robot appearances are a sample of a broader design space. The sample (N = 81) and specific task framing may limit generalizability across educational contexts and populations. ## Connected Concepts - [[social-robots]] - [[human-robot-interaction]] - [[educational-robotics]] - [[trust]] - [[higher-ed]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[human-autonomy-agency-hri-review-2025]] — Human Autonomy and Agency in HRI - [[knowledge-based-design-generative-social-robots-2026]] — Knowledge-Based Design for Generative Social Robots ## Citation Velentza, A.-M., Nikou, K., Bosser, A.-G., & Fachantidis, N. (2026). *What robots do matters more than what they look like: Task context shapes trust in educational HRI*. arXiv:2606.14602. --- ## [TeachBench - Evaluating LLM Teaching Ability](https://edtechdev.github.io/aied/articles/teachbench-llm-teaching-evaluation/) > While LLMs are increasingly used as teaching assistants, their teaching capability remains insufficiently evaluated — a critical gap in current AIED research. # TeachBench: Evaluating LLM Teaching Ability > Syllabus-grounded framework for measuring LLM teaching capability via student performance improvement after multi-turn instruction. ## The Gap in LLM Evaluation **Li et al. (2026)** address a critical gap: while LLMs are increasingly used as teaching assistants, their **teaching capability remains insufficiently evaluated**. ### Limitations of Existing Benchmarks | Benchmark Type | Focus | Limitation | |---------------|-------|-------------| | **Problem-solving** (MMLU, HELM, GSM8K) | Answer correctness | Measures solver, not teacher | | **Exam-centric** (AGIEval, C-Eval, GAOKAO-Bench) | Exam performance | Still solution-centric | | **Tutoring dialogues** (MathDial, TutorBench) | Per-turn response quality | Misses end-to-end teaching effectiveness | TeachBench shifts the evaluation target: **from solving to teaching**. ## Syllabus-Grounded Framework ### Core Design Principles 1. **Knowledge-centered:** Evaluation based on structured knowledge points (syllabus), not target questions 2. **Leakage-controlled:** Teacher agents restricted to knowledge points + example problems (no access to test items) 3. **Outcome-based:** Teaching effectiveness measured by **student agent's performance improvement** ### Workflow ``` Syllabus → Knowledge Tree → Teacher Agent (multi-turn instruction) → Student Agent → Performance Gain ``` ## Key Findings from Gaokao Experiments Using Chinese National College Entrance Examination (Gaokao) data across multiple subjects: | Finding | Implication | |---------|-------------| | **Domain variation:** Math teaching effective (7.63pt gain with Qwen3-235B), but physics/chemistry challenging | Teaching ability is domain-specific, not generalized | | **Example problems backfire:** Models shift to error correction vs. syllabus-grounded instruction | Current LLMs struggle with structured teaching vs. reactive problem-solving | | **Teaching ≠ Solving:** Models good at solving aren't necessarily good at teaching | Teaching ability is a **distinct LLM behavior dimension** | ## Connection to Existing Work ### vs. [[ai-tutor-effectiveness-review|AI Tutor Effectiveness]] - Traditional ITS effectiveness reviews focus on *human* learning outcomes with *deployed* systems - TeachBench evaluates *model* teaching capability in *controlled* agentic settings - Both highlight: **teaching is more than problem-solving** ### vs. [[educational-llm-alignment|Educational LLM Alignment]] - Alignment benchmarks measure: "Does this model produce good teaching content?" - TeachBench measures: "Does this model *improve learning* through instruction?" - Complementary: alignment → content quality; TeachBench → instructional effectiveness ### vs. [[agentic-workflows-education|Agentic Workflows]] - TeachBench operationalizes the **"teacher agent"** paradigm in agentic education - Reveals current LLMs struggle with *structured pedagogical planning* (vs. reactive Q&A) - Aligns with: agentic reflection, planning, and tool use in educational contexts ## Implications for AI in Education ### For LLM Development - **Teaching ability as benchmark dimension:** Should be evaluated alongside reasoning, coding, etc. - **Domain-specific training:** Models need pedagogy-grounded training, not just subject knowledge - **Structured vs. reactive:** TeachBench exposes the gap between knowing and teaching ### For ITS Design - **Syllabus grounding:** Effective teaching requires structured knowledge (cf. [[quantum-education-its|quantum ITS knowledge graph]]) - **Multi-turn instruction:** Teaching effectiveness emerges over *sequences* of interactions, not single turns - **Evaluation rigor:** ITS should be evaluated on *learning gains*, not just user satisfaction ### For Benchmark Design - **Leakage control:** Don't give models the test questions they're supposed to teach - **Student proxies:** Fixed-capability student agents enable reproducible evaluation - **Outcome metrics:** Performance improvement > response quality scores ## Connected Concepts - [[ai-feedback-quality]] - [[socratic-ai-dialogue]] - [[ai-ed-evaluation]] - [[metacognition]] - [[self-regulated-learning]] - [[ai-tutoring]] - [[prompt-engineering]] - [[adaptive-learning]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[educational-llm-alignment]] - [[agentic-workflows-education]] - [[quantum-education-its]] ## Citation Models, T.A.S.F.F.E.T.A.I.L.L. (2026). [*TeachBench - Evaluating LLM Teaching Ability*](https://arxiv.org/abs/2601.21375) --- ## [AI Adoption Among Teachers: Insights on Concerns, Support, Confidence, and Attitudes](https://edtechdev.github.io/aied/articles/teacher-ai-adoption-confidence/) A study of 260 Filipino teachers examined how institutional support, teacher confidence, and teacher concerns influence AI adoption attitudes: - **Institutional support significantly predicted both teacher confidence and positive attitudes toward AI adoption.** - **Teacher concerns did NOT moderate these relationships** — concerns didn't weaken or strengthen the effect of support. - **Full mediation**: The effect of institutional support on AI adoption attitudes worked entirely through increased teacher confidence. ## Practical Implications This paper provides empirical clarity for [[teacher-role]] policy: institutional support matters, but it only helps if it builds teacher confidence. Simply providing tools or mandates without confidence-building will not shift attitudes. The finding that concerns did not moderate the relationships suggests that addressing concerns directly may be less effective than building confidence through support, aligning with [[faculty-development]] on the importance of hands-on experience. ## Connected Concepts - [[teacher-role]] - [[faculty-development]] ## Connected Articles - [[ai-pedagogical-orientation]] — Faculty Orientations Shape Adoption of AI in Research and Teaching - [[state-policy-teacher-ai]] — How State Policy Can Help Teachers Use AI Well - [[ai-tpack-teacher-multi-agent-workflow]] — Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design - [[teacher-education-ai-literacy-sdt-2026]] — Teacher education for artificial intelligence literacy through a self-determination theory perspective - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[genai-literacy-training-teacher-education-dbr-2026]] — Development and evaluation of artificial intelligence literacy training for teacher education students ## Citation Sibug, V. B., Cruz, M. A. D., Vital, V. P., Grume, J. C., Gamboa, A. B., Fernando, E. Q., Feliciano, L. D., Salenga, J. L., & Miranda, J. P. P. (2026). [AI adoption among teachers: Insights on concerns, support, confidence, and attitudes](https://arxiv.org/abs/2605.00343). Proceedings of the 9th International Conference on Education and Multimedia Technology (ICEMT 2025), 267-269. arXiv:2605.00343. --- ## [Towards Synergistic Teacher-AI Interactions with Generative Artificial Intelligence](https://edtechdev.github.io/aied/articles/teacher-ai-teaming-five-levels/) > **Synthesis:** Drawing on a systematic literature review, this UCL chapter proposes a five-level framework of teacher-AI teaming—transactional, situational, operational, praxical, and synergistic—to capture how GenAI interactions may replace, complement, or augment teacher competence. The framework moves beyond task division toward collaborative decision-making where teachers and AI engage in negotiation, constructive challenge, and co-reasoning. ## Five Levels of Teacher-AI Teaming The framework defines a progression from minimal to maximal teacher-AI collaboration: 1. **Transactional** — AI executes a concrete task on request with no shared context. Teacher sends prompt, AI returns output. Example: generating a quiz from a topic list. Risk: cognitive offloading without professional growth. 2. **Situational** — AI gathers data from the teaching/learning context and reports it back. A shared state exists for the task. Example: AI summarizes classroom discussion data. Teacher remains the primary decision-maker. 3. **Operational** — Teacher sets goals; AI not only reports context but also proposes actions to achieve those goals. AI moves beyond task execution into action planning. Example: AI recommends pedagogical interventions based on student performance data. 4. **Praxical** — AI incorporates learning patterns from prior interactions to adapt its behavior over time. The system learns from teacher feedback and refines its models. Teacher and AI develop shared practice. 5. **Synergistic** — Both agents engage in negotiation, constructive challenge, and co-reasoning that enhance each other’s capabilities. Outcomes emerge that neither could realize independently. This is the highest level of hybrid intelligence. ## Key Insights - **Beyond task automation**: Most current GenAI use in education operates at transactional or situational levels. Moving toward operational, praxical, and synergistic teaming requires deliberate design of the interaction paradigm, not just better models. - **Teacher agency is central**: The framework argues that GenAI integration should enhance—not diminish—teacher agency. Higher teaming levels require teachers to actively engage in decision-making rather than passively accept AI outputs. - **Cognitive atrophy risk**: Fully offloading teaching tasks to AI at lower teaming levels risks diminishing teachers' cognitive engagement and professional growth (deprofessionalization). - **Socio-technical factors matter**: Beyond technological design, organizational culture, professional development, ethical guidelines, and institutional support shape whether teacher-AI interactions reach synergistic levels. - **Hybrid intelligence**: The vision is teachers and AI as complementary agents—each bringing distinct capabilities—rather than AI as a replacement. ## Connected Concepts - [[curriculum-design]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[bias-mitigation]] - [[ai-ed-evaluation]] - [[k-12-ai-education]] - [[ai-education]] - [[instructional-design]] ## Connected Articles - [[ai-tpack-teacher-multi-agent-workflow]] — Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[eduagentbench-agent-teaching-benchmark]] — Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows - [[prompt-based-programming-lesson]] — Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers - [[genai-marketing-education-roles-2026]] — When AI Wears Many Hats: The Role of Generative Artificial Intelligence in Marketing Education ## Citation Cukurova, M., Suraworachet, W., Zhou, Q., & Bulathwela, S. (2025). [*Towards Synergistic Teacher-AI Interactions with Generative Artificial Intelligence*](https://arxiv.org/abs/2511.19580). arXiv preprint. --- ## [Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation](https://edtechdev.github.io/aied/articles/teacher-authored-prompts-student-ai-dialogue/) This large-scale K-12 deployment provides empirical evidence that teacher-authored prompts can reliably shape the cognitive quality of student-AI dialogue at classroom scale. The TASD system lets teachers define both the AI's role and the student-facing conversation starter, creating a two-layer orchestration model that produced 71% alignment with instructional goals. The 38% under-reach rate in cognitive demand—approaching 50% for the highest DOK level—mirrors known gaps between intended and enacted scaffolding documented in [[scaffolding]] and [[student-experience]] research. The intervention study showed that adding explicit finish lines to prompts and 'no direct answers' guardrails meaningfully narrowed the gap, connecting prompt engineering to [[formative-assessment]] design principles. The work also extends [[teacher-role]] scholarship by showing that teachers can operate as prompt architects without deep technical expertise, provided the system surfaces the right levers. This positions teacher-authored configuration as a bridge between [[generative-ai]] capabilities and [[k-12]] classroom constraints. ## Connected Concepts - [[scaffolding]] - [[student-experience]] - [[formative-assessment]] - [[teacher-role]] - [[generative-ai]] - [[k-12]] ## Connected Articles - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI - [[from-answer-generators-to-reasoning-facilitators-ai-tutors]] — From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[rubric-aware-grading-rec-cbm]] — REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading ## Citation Alex Liu, Min Sun, Lief Esbenshade, Victor Tian, Zachary Zhang, Kevin He (2026). [Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation](https://arxiv.org/abs/2604.16738). arXiv:2604.16738. arXiv. --- ## [When Should Teachers Control AI Generation for Mathematics Visuals?](https://edtechdev.github.io/aied/articles/teacher-control-ai-generation-math-visuals/) # Teacher Control of AI-Generated Mathematics Visuals ## Key Findings Generative AI can help teachers rapidly create classroom-ready visual materials, particularly in mathematics where diagrams and visual representations must be **pedagogically meaningful and instructionally correct**. This paper investigates when and how teachers should control AI generation of mathematical visuals. The study identifies key decision points where teacher input is essential: ensuring pedagogical correctness of visual representations, alignment with curriculum goals, and appropriateness for student levels. The authors propose a framework for teacher-AI collaboration in visual material creation that balances AI efficiency with pedagogical control. ## Connections to AIED This work connects to [[teacher-role]] by showing how teachers remain essential as pedagogical validators of AI-generated content. It intersects with [[stem-education]] since mathematics visual generation is a core need in STEM teaching. The findings also relate to [[generative-ai]] and [[principled-ai-education]], suggesting that principled AI use in education requires human oversight for pedagogically sensitive outputs. The teacher control framework connects to [[agentic-workflows-education]] where AI agents assist but humans direct. ## Connected Concepts - [[teacher-role]] - [[stem-education]] - [[generative-ai]] ## Connected Articles - [[principled-ai-education]] - [[agentic-workflows-education]] ## Citation Li, Z., Wang, J., & Wang, A. Y. (2026). [When should teachers control AI generation for mathematics visuals?](https://arxiv.org/abs/2605.10672) arXiv:2605.10672. Proceedings of the Thirteenth ACM Conference on Learning @ Scale (L@S 2026), Seoul, Republic of Korea. --- ## [Teacher education for artificial intelligence literacy through a self-determination theory perspective](https://edtechdev.github.io/aied/articles/teacher-education-ai-literacy-sdt-2026/) > **Synthesis:** Chiu, Bali, Tondeur, Howard, and Chan (2026) apply self-determination theory (SDT) to investigate how need-supportive professional development (PD) impacts teachers' AI literacy, attitudes, anxiety, and engagement in online professional learning communities (PLCs). Using a sequential mixed-methods approach with 382 secondary school teachers, they found that need-supportive PD enhances AI literacy and fosters sustained behavioral engagement in PLCs, with AI literacy emerging as the key cognitive driver of sustained participation. Qualitative analysis identified nine specific design strategies for PD that satisfy teachers' needs for autonomy, competence, and relatedness — bridging the often-overlooked connection between isolated PD and PLCs to support the scaling and sustainability of teacher AI learning. ## Key Findings - **Need-supportive PD positively predicts needs satisfaction (β = .77, p < .001) and behavioral engagement in PLCs (β = .47, p < .001)** — a well-designed PD can spark ongoing, collaborative learning. - **Needs satisfaction significantly predicts AI literacy (β = .52), attitude (β = .68), and reduces AI anxiety (β = −.66)** — PD content alone is insufficient; *how* it is delivered (whether it supports psychological needs) determines its effectiveness in changing teachers' AI knowledge, feelings, and beliefs. - **AI literacy is the key cognitive gateway to sustained engagement**: among the AI learning outcomes, only AI literacy had a significant direct effect on behavioral engagement in PLCs (β = .39, p < .001); attitude (p = .72) and anxiety (p = .18) were non-significant — teachers need a foundation of ability before actively engaging in peer-to-peer learning. - The theoretical contribution is a validated motivational pathway: PD needs support → needs satisfaction → cognitive (AI literacy) and affective (attitude, anxiety) learning → sustained behavioral engagement, with AI literacy as the primary cognitive driver. - **Nine design strategies** were identified: four autonomy-supportive (elective AI topic selection, collaborative AI resource development/co-design, individualised AI integration targets, non-mandatory task completion), two competence-supportive (expert-peer feedback on AI learning designs, public sharing of AI teaching artefacts), and three relatedness-supportive (cross-disciplinary AI design teams, mentorship, mission-alignment of AI learning). ## Study Design & Method This sequential mixed-methods study examined 382 secondary school teachers engaged in AI-focused professional development. **Quantitative phase:** participants completed validated SDT-based questionnaires (perceived PD support, autonomy, relatedness, competence) plus measures of AI attitude, AI anxiety, and AI literacy (a 30-item multiple-choice test drawn from the AI4future item bank), and behavioral engagement in online PLCs was objectively measured by posts, replies, reads, sharing, and collaborations. Data were analyzed with confirmatory factor analysis (CFA) and structural equation modeling (SEM), with good model fit (χ²/df = 1.43, RMSEA = .03, CFI = .99). **Qualitative phase:** deductive content analysis of interviews using SDT as the analytic framework identified the nine design strategies, with two independent raters and a moderator (inter-rater reliability = 0.88). ## Implications for AI in Education The study provides an evidence-based blueprint for designing effective, sustainable AI-focused teacher PD, directly relevant to [[faculty-development]], [[professional-training]], and [[teacher-ai-competency]]. It argues that PD providers and universities should structure AI training to explicitly incorporate the nine need-supportive strategies, prioritize building foundational [[ai-literacy]] (the sole direct predictor of PLC engagement), and position PD as a catalyst for self-sustaining learning communities rather than a one-time endpoint. Theoretically, it extends SDT to the AI context, showing that for highly technical domains like AI, knowledge acquisition is the gateway to collaborative participation — a finding that connects to [[motivation]], [[self-determination-theory]], and [[k-12]] teacher education. It bridges the PD/PLC divide, offering a motivational pathway for scaling and sustaining teacher AI learning. ## Limitations The study's context is secondary school teachers (largely in Chinese/Hong Kong and East Asian settings), bounding generalizability to other regions and K-12 contexts. The authors acknowledge that sustaining a need-supportive culture within PLCs after the structured program ends is challenging, that systemic constraints (rigid curricula, limited resources, top-down mandates) can undermine need-supportive PD, and that the strategies may overlook the mediating role of contextual factors like teacher prior AI experience or entrenched school culture. The AI literacy test was a specific objective measure developed for the AI4future project. ## Connected Concepts - [[ai-literacy]] - [[teacher-ai-competency]] - [[faculty-development]] - [[professional-training]] - [[teacher-role]] - [[higher-ed]] - [[k-12]] - [[motivation]] - [[self-determination-theory]] ## Connected Articles - [[genai-literacy-training-teacher-education-dbr-2026]] — Development and Evaluation of AI Literacy Training for Teachers - [[hcap-human-centric-ai-pedagogy-framework-2026]] — Human-Centric AI Pedagogy (HCAP) Framework - [[ai-tpack-preservice-math-teachers]] — AI-TPACK and Pre-Service Math Teachers - [[sangwa-epiq-ai-faculty-readiness-2026]] — EPIQ AI Faculty Readiness ## Citation Chiu, T. K. F., Bali, S., Tondeur, J., Howard, S., & Chan, K. K. H. (2026). [*Teacher education for artificial intelligence literacy through a self-determination theory perspective*](https://doi.org/10.1080/02619768.2026.2621848). *European Journal of Teacher Education*. --- ## [Balancing Teacher and Student Agency: Co-Orchestration Tool Design Supporting Real-Time Dynamic Pairing](https://edtechdev.github.io/aied/articles/teacher-student-agency-orchestration/) Yang et al. (2026) tackle a fundamental tension in AI-augmented classrooms: how to balance teacher orchestration with student agency during dynamic transitions between individual and collaborative work. Using participatory speed dating with teachers and students, the study maps a three-stage design space (before, during, and after pairing) and proposes a hybrid-control framework for analytic-based orchestration tools. The core recommendation — structured teacher guidance initially, with progressively increasing student autonomy — provides an actionable design principle for [[intelligent-tutoring]] systems that manage classroom-level coordination. This graduated autonomy model connects to [[scaffolding]] theory and extends it from individual learning support to social-organizational support. The work also contributes to the [[teacher-role]] literature by formalizing how human-AI co-orchestration can distribute agency without undermining either party's effectiveness. By situating the research within real K-12 classroom dynamics and publishing at CSCW, the paper bridges the [[collaborative-learning]] and [[human-in-the-loop-ai]] communities that have often addressed these questions separately. The three-stage framework (before/during/after) offers a structured lens for evaluating other orchestration tools, complementing the [[ai-tutor-behavioral-evaluation]] focus on individual tutor performance with classroom-level coordination metrics. For [[k-12]] practitioners, the study validates concerns about AI systems that optimize for efficiency at the cost of student [[self-regulated-learning]] and teacher professional judgment. ## Connected Concepts - [[intelligent-tutoring]] - [[scaffolding]] - [[teacher-role]] - [[collaborative-learning]] - [[human-in-the-loop-ai]] - [[k-12]] - [[self-regulated-learning]] ## Connected Articles - [[ai-tutor-behavioral-evaluation]] ## Citation Kexin Bella Yang, Menghan Liu, Liyi Xu, Nikol Rummel, Vincent Aleven (2026). [Balancing Teacher and Student Agency: Co-Orchestration Tool Design Supporting Real-Time Dynamic Pairing](https://arxiv.org/abs/2605.18761). arXiv:2605.18761. arXiv:2605.18761 [cs.HC] — Accepted at CSCW 2026, to appear in PACM HCI. --- ## [A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol](https://edtechdev.github.io/aied/articles/teaching-feedback-classification-benchmark/) > Extends a prior validated protocol for classifying open-ended teaching-evaluation feedback by thematic category and sentiment, introducing a durability and cross-language transfer benchmark. Institutions collect far more teaching feedback than they read; automated classification makes it actionable. Sits at the intersection of [[feedback-loop]], [[automated-grading]], and [[teacher-role]] support in [[higher-ed]]. It relates to [[formative-assessment]] and the [[ai-feedback-quality]] stub, providing a reusable benchmark for scaling educator feedback analysis and connecting to [[faculty-development]]. ## Key Findings - The validated classification protocol is durable across three generations of representation: sparse lexical features, frozen transformer embeddings, and prompted large language models. - A 2026 frontier model posted the highest thematic F1 on the hardest Spanish task, yet showed no sentiment advantage over a cheaper model. - On the English sentiment transfer task — a balanced 45,000-comment corpus checked against an aspect-labeled education dataset — the frontier model showed no descriptive separation from a cheap model. - Model choice is therefore a deployment decision, not a property of the method: resource-constrained institutions can adopt economical models without sacrificing classification quality. - LIME auditability checks on held-out Spanish predictions align local explanations with the annotation guide — for example, grosero and amable drive the interaction class, evaluación and calificación drive evaluation, and faltaba and tarde drive attendance/engagement — with the neutral sentiment class leaning on hedging cues such as pero and sin embargo. ## Study Design & Method The benchmark re-runs a previously validated protocol — built from a documented annotation guide, intra-annotator reliability measurement, stratified cross-validation, and held-out evaluation on a Spanish institutional corpus with a frozen-encoder design — on the original Spanish data across three representation generations: sparse lexical features, frozen transformer embeddings, and prompted large language models. The sentiment task is then transferred to English with a balanced 45,000-comment corpus contrasted with an aspect-labeled educational dataset. Paired comparisons are treated as descriptive rather than inferential: headline weighted-F1 differences are reported with descriptive bootstrap confidence intervals (95% percentile, 2,000 resamples), where an interval covering zero indicates no descriptive separation. Spanish arms use a 233-comment split and English LLM arms use a 1,500-comment subset. ## Implications for AI in Education Institutions collect far more open-ended teaching-evaluation feedback than they read, and automated classification makes that corpus actionable for improving teaching. The benchmark's central message is that a validated protocol can remain useful as representation methods advance: a frontier model wins only the hardest thematic task in Spanish, and on sentiment — in both languages — it shows no descriptive separation from economical alternatives. For [[educational-nlp]] and [[benchmark]] practice, this argues for reporting paired comparisons descriptively and for treating model selection as a cost-performance deployment question rather than chasing frontier models by default. The LIME-based auditability check also demonstrates a lightweight way to keep classifications inspectable and aligned with the annotation guide, supporting responsible use of automated feedback analysis in faculty-facing systems. ## Connected Concepts - [[feedback-loop]] - [[automated-grading]] - [[teacher-role]] - [[higher-ed]] - [[formative-assessment]] - [[ai-feedback-quality]] - [[faculty-development]] ## Connected Articles - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[llm-feedback-programming-classroom]] — LLM-Generated Feedback in Introductory Programming: A Classroom Study - [[lata-ferpa-compliant-local-llm-autograder]] — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers'' practices and perceptions ## Citation Esteban U. Vega Barajas (2026). [A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol](https://arxiv.org/abs/2607.11873). arXiv:2607.11873. arXiv preprint. --- ## [Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights](https://edtechdev.github.io/aied/articles/teaching-intro-ai-course-redesign-bill-of-rights-2026/) > **Synthesis:** This experience report describes the redesign of an introductory AI course at the University of Washington Bothell in response to LLMs being able to complete most assignments. The redesign retained the classical core (search, MDPs, reinforcement learning) while adding a strand where students build an LLM from scratch. Assessment was rebuilt around in-class exercises, reflective writing, and defended team projects, with examinations removed entirely. A participatory ethics sequence produced a 'Student Bill of AI Rights' governing the instructor's AI use, including a requirement that the instructor complete any AI-generated assignment before issuing it. ## Key Findings This experience report describes the redesign of an introductory AI course at the University of Washington Bothell in response to LLMs being able to complete most assignments. The redesign retained the classical core (search, MDPs, reinforcement learning) while adding a strand where students build an LLM from scratch. Assessment was rebuilt around in-class exercises, reflective writing, and defended team projects, with examinations removed entirely. A participatory ethics sequence produced a 'Student Bill of AI Rights' governing the instructor's AI use, including a requirement that the instructor complete any AI-generated assignment before issuing it. The work contributes to understanding of [[academic-integrity]] in educational contexts, with implications for [[ai-literacy]], [[assessment]]. ## Connected Concepts - [[academic-integrity]] - [[ai-literacy]] - [[assessment]] - [[generative-ai]] - [[ethics]] - [[higher-ed]] ## Connected Articles - [[finkelstein-principled-ai-education-2025]] - [[beyond-detection-authentic-assessment-ai-2025]] ## Citation Pisan, Y. (2026). [*Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a Student Bill of Rights*](https://arxiv.org/abs/2608.05175). arXiv:2608.05175. --- ## [Findings of the First Teaching Monster Challenge: A Benchmark of Pedagogical Content Knowledge in AI Agents](https://edtechdev.github.io/aied/articles/teaching-monster-pck-benchmark-2026/) > **Synthesis:** Lin et al. (2026) present the **Teaching Monster Challenge**, the first instructional-video generation benchmark that treats the learner persona as an explicit evaluation criterion, measuring whether AI agents can adapt a lesson to a specified learner — [[teacher-ai-competency|Pedagogical Content Knowledge (PCK)]]. Systems receive a topic and a learner persona and must generate a complete instructional video, screened by an LLM-judge, ranked by crowd pairwise voting, and finalized by an expert panel. The first edition shows systems handle content well but are far weaker at presenting and adapting it to the learner. It also exposes a limit of automatic judging: the LLM-judge separates a clear low-performing tail but ranks the strongest systems poorly and nearly identically, so its ranking does not match human preference. ## Benchmarking PCK, Not Just Content AI agents can now solve problems, answer like subject experts, and generate long-form multimodal content, but whether they can adapt a lesson to fit a specified learner — which education calls Pedagogical Content Knowledge — had not been benchmarked. The Teaching Monster Challenge makes the learner persona an explicit evaluation criterion. ## Method Each system is given a topic and a learner persona and must generate a complete instructional video. Every video is screened by an LLM-judge, ranked by crowd pairwise voting, and finalized by an expert panel. ## Findings Today's systems handle content well but are far weaker at presenting it and adapting it to the learner. The LLM-judge separates a clear low-performing tail but ranks the strongest systems poorly, giving them nearly identical scores so its ranking does not match human preference. Progress requires better teaching systems *and* better automatic judges; the benchmark, rubric, and human judgments are released as a testbed for both. ## Connected Concepts - [[teacher-ai-competency]] - [[pedagogical-agent]] - [[benchmark]] - [[ai-ed-evaluation]] - [[ai-ed-evaluation]] - [[agentic-ai]] - [[generative-ai]] - [[generative-ai]] - [[instructional-design]] - [[pedagogical-llm-training]] ## Connected Articles - [[teachbench-llm-teaching-evaluation]] - [[eduagentbench-agent-teaching-benchmark]] - [[ai-tutor-behavioral-evaluation]] - [[solving-vs-evaluating-genai-solutions]] - [[llm-tutoring-feedback-diagnosis-gap]] - [[teaching-feedback-classification-benchmark]] ## Citation Lin, Y.-C., Guo, Y.-K., Chen, S.-C., Feng, B.-H., Hsu, Y.-M., Hsieh, H., Lin, Y.-J., Wu, Y.-L., Dong, J.-K., Cheng, A.-Y., Huang, Y.-H., Ieong, L.-L., Chen, K.-Y., Tchouang, M.-D., Sun, S.-H., Lin, C., Ding, J.-J., & Lee, H.-y. (2026). [*Findings of the first Teaching Monster Challenge: A benchmark of pedagogical content knowledge in AI agents*](https://arxiv.org/abs/2608.08852). arXiv:2608.08852. --- ## [TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors](https://edtechdev.github.io/aied/articles/teachingcoach-chatbot-instructor-guidance/) > **Authors:** Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla **Year:** 2026 **Venue:** arXiv (cs.AI) > **Year:** 2026 > **Venue:** arXiv (cs.AI) ## Summary Higher education instructors often lack timely and pedagogically grounded support. Universities operate teaching and learning centers that offer workshops and consultations, but these resources face key limitations: support may not be available at the moment of need, feedback is often generic rather than tailored to an instructor's background, and some educators hesitate to seek direct help for fear of appearing unskilled. General-purpose LLMs like ChatGPT are widely accessible, but their responses are typically generic and rarely apply evidence-based principles in a pedagogically scaffolded way. TeachingCoach is a pedagogically grounded chatbot that simulates the role of a teaching expert and delivers conversational guidance on instructional practice through problem identification, diagnosis, and strategy development. The system is built on a data-centric pipeline. First, pedagogical rules are extracted from educational resources, with experts encoding 36 core instructional rules as structured system prompts that anchor conversations in evidence-based practices. Second, GPT-4o generates realistic and diverse training data: teacher profiles specifying years of experience and teaching subject, teaching challenges such as managing classroom attention, and multi-turn conversations between a teacher and a simulated expert that progress over 20–30 turns through clarifying the challenge, exploring strategies, planning next steps, and reflecting on practice. Human experts review the generated conversations, removing those that are inconsistent, repetitive, or pedagogically unsound; after filtering, the dataset contains 406,183 training, 4,156 validation, and 4,143 test examples. Finally, a LLaMA-2-13B-Chat model is fine-tuned with full parameter updates; at each turn the assistant first predicts an instructional step — Step 1: Identify the Problem, Step 2: Explore Reasons, Step 3: Develop Strategies — and then generates the response conditioned on that step, aligning dialogue progression with a structured pedagogical scaffold. ## Key Contributions - Outperforms a GPT-4o baseline on expert-rated dialogue quality: across 200 conversations rated on a 3-point scale over four dimensions (clarity, respectful tone, encouragement of reflection and reasoning, acknowledgment of user input), TeachingCoach consistently scored higher (e.g., 2.62 vs. 2.01 on clarity). - A user study with higher education instructors highlights trade-offs between conversational depth and interaction efficiency: although the baseline model was preferred overall (21 vs. 13), participants more often attributed learning to the fine-tuned model (18 vs. 13), and agreement between preference and learning attribution was low (39%). - Step labels generated by the model can be logged for analytics and personalization while optionally being hidden from the visible transcript, enabling supervision of the instructional process. - The demo system provides onboarding (asking users about experience, current courses, and AI attitudes), the chat interface, and a dashboard for scheduling consultations with live experts, managing collected resources, and storing user data. ## Study Design & Method Expert evaluations compared TeachingCoach with a GPT-4o baseline in a zero-shot setting, highlighting the impact of explicit pedagogical supervision and step-aware training independent of model scale or proprietary data. The user study with higher education instructors examined both overall preference and perceived learning, finding that participants consistently distinguished the models on conversational engagement, efficiency of interaction, and breadth of suggestions: the fine-tuned model was described as resembling dialogue with a human pedagogy expert through reflective questions and follow-up prompts, while the baseline was valued for generating responses quickly when immediate guidance was needed. The work positions TeachingCoach against prior systems that primarily support students, arguing that instructor-facing support grounded in pedagogical practice has been underserved. ## Connected Concepts - [[teacher-ai-competency]] - [[teacher-role]] - [[k-12-ai-education]] - [[instructional-design]] - [[prompt-engineering]] - [[formative-assessment]] - [[affective-tutoring]] - [[curriculum-design]] ## Connected Articles - [[ai-tpack-teacher-multi-agent-workflow]] — Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design - [[ai-pedagogical-orientation]] — Faculty Orientations Shape Adoption of AI in Research and Teaching - [[learner-centered-feedback-ai]] — Enhancing learner-centered feedback with AI: teachers'' practices and perceptions - [[teacher-ai-adoption-confidence]] — AI Adoption Among Teachers: Insights on Concerns, Support, Confidence, and Attitudes - [[ai-generated-slides-student-perception]] — AI-Generated Slides: Are They Good? Can Students Tell? - [[didactical-teacher-assistant-dimensional-modeling]] — A didactical-driven teacher assistant for a dimensional modeling course ## Citation Isabel Molnar et al. (2026). [TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors](https://arxiv.org/abs/2603.18189). arXiv:2603.18189. cs.AI. --- ## [Teachy Mini: Development and Preliminary Evaluation of a Knowledge-Based Generative Social Robot for Higher Education](https://edtechdev.github.io/aied/articles/teachy-mini-generative-social-robot-higher-ed-2026/) > **Synthesis:** Vonschallen, Kaufmann, Oberle, Eyssel, and Schmiedel (2026) operationalize knowledge-based design (KBD) requirements for generative social robots (GSRs) by implementing them in the Reachy Mini robot platform through system prompting, retrieval-augmented generation, and stateful prompt orchestration, producing Teachy Mini — a GSR tutoring system for higher education. Recognizing that GSRs powered by LLMs offer personalized tutoring but also risks (misinformation, missing transparency, reinforcing incorrect responses), the authors conducted a preliminary evaluation study in which participants (N = 24) completed a robot-guided learning session about research methodologies, learning with either the robot or another mode. ## Key Findings - **Generative social robots (GSRs) powered by LLMs offer new possibilities for personalized tutoring in higher education** but introduce risks related to misinformation, missing transparency, and reinforcing incorrect student responses. - Prior work identified **knowledge-based design (KBD) requirements** defining the informational prerequisites for GSRs to manifest responsible and effective tutoring behaviour. - The authors **operationalized selected KBD requirements in the Reachy Mini robot platform** through system prompting, retrieval-augmented generation, and stateful prompt orchestration, producing **Teachy Mini**. - A **preliminary evaluation study (N = 24)** had participants complete a robot-guided learning session about research methodologies, learning with either Teachy Mini or an alternative mode. ## Study Design & Method This is a **system development and preliminary evaluation study**. The researchers operationalized knowledge-based design requirements for generative social robots by implementing them on the Reachy Mini robot platform using system prompting, retrieval-augmented generation (RAG), and stateful prompt orchestration, building the Teachy Mini tutoring system. A preliminary evaluation study with 24 participants had them complete a robot-guided learning session about research methodologies, comparing learning with Teachy Mini against an alternative mode to assess the system's effectiveness and the value of the KBD approach. ## Implications for AI in Education Teachy Mini demonstrates how **knowledge-based design requirements** can be concretely implemented in [[social-robots|generative social robots]] for [[higher-ed]] tutoring, using system prompting, RAG, and stateful prompt orchestration to mitigate risks such as misinformation and reinforcing incorrect responses. It connects to [[generative-ai]], [[llm|large language models]], [[ai-tutoring|tutoring]], and [[educational-robotics]]. For designers and educators, it provides a validated example of translating responsible-AI design principles into a functioning embodied tutor, complementing [[knowledge-based-design-generative-social-robots-2026|the prior KBD requirements study]]. ## Limitations The evaluation is preliminary with a small sample (N = 24), and the specific comparative learning outcomes are not fully detailed in the abstract. The system is built on the Reachy Mini platform, so generalizability to other robot platforms and to broader disciplinary content requires further study. The focus is on research-methodology tutoring, and longer-term effects and broader responsible-AI risks warrant additional evaluation. ## Connected Concepts - [[social-robots]] - [[generative-ai]] - [[llm]] - [[higher-ed]] - [[ai-tutoring]] - [[educational-robotics]] ## Connected Articles - [[knowledge-based-design-generative-social-robots-2026]] — Knowledge-Based Design Requirements for GSRs - [[task-context-trust-educational-hri-2026]] — Task Context and Trust in Educational HRI - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education ## Citation Vonschallen, S., Kaufmann, K., Oberle, D., Eyssel, F., & Schmiedel, T. (2026). *Teachy Mini: Development and preliminary evaluation of a knowledge-based generative social robot for higher education*. arXiv:2607.22345. --- ## [Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned](https://edtechdev.github.io/aied/articles/tech-enhanced-tabletop-cybersecurity-education/) Innovative practice paper examining the integration of technology-enhanced tabletop exercises into cybersecurity curricula. Addresses the gap between professional TTX practice and university adoption, presenting lessons learned from implementation in computing education contexts. ## Key Findings - A web application (IXP) was designed to automate exercise delivery and enable data-driven evaluation of tabletop exercises (TTXs). - Unlike traditional pen-and-paper TTXs, IXP automatically delivers scenario updates, facilitates team discussions, and collects interaction data for automated assessment. - The approach enhances realism, reduces instructor workload, and offers actionable insights into student learning behaviors. - Automated data collection enabled analysis of team performance, communication patterns, and decision-making processes across 25 TTXs delivered from October 2024 to March 2026. - Post-exercise feedback from trainees and instructor observations indicate improved student engagement and collaboration, and faster feedback compared with traditional teaching methods. ## Study Design & Method A cybersecurity tabletop exercise is a conversation between participants responsible for fulfilling a variety of roles during a cybersecurity incident. The paper describes the design, delivery, and evaluation of 25 TTXs using the IXP platform from October 2024 to March 2026. The exercise lifecycle — planning, development, delivery, and post-exercise reflection — structures the lessons learned, which include recommendations for realistic, well-thought-out scenarios and for instructor support during the exercise. Although the recommendations are grounded in cybersecurity, the authors argue they are not domain-specific and can be used for digital TTXs in other areas. ## Implications for AI in Education The work demonstrates how technology enhancement can close the gap between professional TTX practice and university teaching: automated scenario delivery, team discussion facilitation, and interaction-data collection turn an experiential exercise into a measurable learning event. The combination of experiential learning with analytics is the platform's distinguishing value — instructors gain insight into team performance, communication patterns, and decision-making without manual observation, and students receive faster feedback. For [[active-learning]] and [[professional-training]] contexts beyond cybersecurity, the lessons learned offer a transferable template: prioritize realistic scenarios, design for instructor support, and instrument the exercise from the start so that evaluation is a byproduct of delivery rather than an afterthought. The work also contributes to [[cs-education]] practice and to [[learning-analytics]] approaches for hands-on, collaborative learning formats in [[higher-ed]]. ## Connected Concepts - [[cs-education]] - [[active-learning]] - [[higher-ed]] - [[learning-analytics]] - [[professional-training]] - [[socratic-ai-dialogue]] - [[math-education]] - [[curriculum-design]] ## Connected Articles - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion - [[forap-pjbl-computing-education]] — Adoption-Ready Project-Based Learning for Computing Education: The FORAP Framework and a Multi-Scale Project Portfolio - [[visual-query-tracer-declarative-logic-learning]] — Evaluating a Visual Query Tracer and Builder for Learning Declarative Logic Programming - [[q-learning-lab-rl-teaching]] — Q-Learning Lab: Teaching Reinforcement Learning Through Learner-Generated Trace Analysis - [[llm-design-problems-hot-pjbl]] — LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs ## Citation Jan Vykopal, Pavel Čeleda, Martin Horák, Valdemar Švábenský (2026). [Technology-Enhanced Tabletop Exercises for Cybersecurity Education: Lessons Learned](https://arxiv.org/abs/2607.28179). arXiv:2607.28179. cs.CY, cs.CR. --- ## [Temporal Smoothness Doubly Robust Learning for Debiased Knowledge Tracing](https://edtechdev.github.io/aied/articles/temporal-smoothness-debiased-kt/) ## Key Findings - Knowledge tracing (KT) systems suffer from selection bias because exercise recommendations are non-random, so training on observed logs with standard empirical risk produces biased mastery estimates. - The proposed Temporal Smoothness Doubly Robust (TSDR) framework combines a propensity model with an error imputation model, jointly optimizing the KT predictor and the imputation model. - A temporal smoothness regularizer reduces variance while preserving the unbiasedness guarantees of the doubly robust estimator. - The doubly robust property ensures the estimator remains unbiased if *either* the propensity model or the imputation model is correct. - By debiasing mastery estimates, the framework prevents adaptive recommendations from compounding errors in downstream recommendation loops. ## Analysis This paper addresses a critical but under-examined issue in KT systems: **selection bias** from non-random exercise recommendations. Prior KT methods train on observed logs using standard empirical risk, producing biased mastery estimates that compound errors in downstream recommendation loops. The proposed **Temporal Smoothness Doubly Robust (TSDR)** framework combines a propensity model with an error imputation model, regularized for temporal smoothness. The smoothness prior encodes the intuition that a student's mastery changes gradually over time, so sharp fluctuations in predicted mastery are more likely to reflect noise than genuine learning events. The doubly robust property ensures the estimator remains unbiased if *either* the propensity or imputation model is correct. This connects to [[personalized-learning]] by ensuring adaptive recommendations are not systematically biased toward high-engagement students, and it extends [[knowledge-tracing]] methodology with a principled treatment of the missing-data structure inherent in learner interaction logs. ## Implications for AI in Education Selection bias is not a niche concern: any adaptive system that recommends exercises based on what it believes a student knows creates a feedback loop in which the data collected is shaped by the model's own prior estimates. TSDR's combination of propensity weighting, error imputation, and temporal smoothness offers a template for building debiased [[student-modeling]] pipelines that remain stable as they are iteratively retrained on fresh interaction data. For researchers and practitioners working on adaptive learning systems, the framework underscores the importance of modeling the data-generating process — who gets which exercise and why — rather than treating observed logs as a random sample. The work also relates to [[bias-mitigation]] more broadly: fairness and accuracy concerns in educational AI are often traceable to selection mechanisms that systematically over- or under-represent particular learners. ## Connected Concepts - [[personalized-learning]] ## Connected Articles - [[explainable-probabilistic-kt]] — Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning - [[skill-acquisition-without-temporal-info]] — Estimating Learners' Skill Acquisition Without Temporal Information - [[knowledge-gap-detection-ai-tas]] — Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs - [[xie-hillm-cd-2026]] — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[pattern-kc-programming-recommendation]] — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components ## Citation Zhan et al. (2026). [Temporal Smoothness Doubly Robust Learning for Debiased Knowledge Tracing](https://arxiv.org/abs/2605.05958). arXiv:2605.05958. arXiv preprint. --- ## [Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests](https://edtechdev.github.io/aied/articles/test-driven-ai-assisted-learning/) Liu et al. (2026) report on a 13-week Test-Driven, AI-Assisted (TDAA) redesign of a Theory of Computation course at HKUST (Guangzhou). The course replaced all lectures with self-directed, AI-assisted learning and weekly closed-book tests serving as high-frequency quality gates. AI agents helped the instructor prepare learning paths, course websites, test drafting, grading workflows, and content repairs — all managed through a version-controlled agent workspace. Student survey data (N=18), weekly scores, and git history suggest the model preserved individual accountability while making material production and marking scalable with human oversight. The work contributes a reusable design pattern for [[ai-changing-teaching-workflows]] that integrates [[active-learning]] principles with [[llm]]-powered scaffolding, and offers practical implications for [[assessment]] design in [[higher-ed]] contexts. The approach connects to broader conversations about [[automated-grading]] and [[llm-student-modeling-memory]] as enablers of scalable personalized education. ## Connected Concepts - [[active-learning]] - [[llm]] - [[assessment]] - [[higher-ed]] - [[automated-grading]] ## Connected Articles - [[ai-changing-teaching-workflows]] - [[llm-student-modeling-memory]] ## Citation Jin-Guo Liu, Shang-Qi Lu, Xin-Ran Shi, Long-Li Zheng, Wei Wang (2026). [Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests](https://arxiv.org/abs/2606.23315). arXiv:2606.23315. arXiv:2606.23315 (cs.CY) --- ## [Text Simplification for Intelligent Tutoring](https://edtechdev.github.io/aied/articles/text-simplification-its/) > **MuTSE** (Roscan et al., 2026) addresses a critical need in **Intelligent Tutoring Systems (ITS)**: delivering content at the right reading level for each learner. > Human-in-the-loop evaluation framework for text simplification in ITS, addressing LLM output variability across prompting strategies. ## The Challenge of Text Simplification in Education **MuTSE** (Roscan et al., 2026) addresses a critical need in **Intelligent Tutoring Systems (ITS)**: delivering content at the right reading level for each learner. ### Why Text Simplification Matters for ITS - **Adaptive content:** Match reading complexity to learner's current level - **Scaffolding:** Progressive complexity increase ([[zone-of-proximal-development]]) - **Accessibility:** Make domain content accessible to diverse learners ## MuTSE: Multi-use Text Simplification Evaluator ### Human-in-the-Loop Design | Component | Function | Pedagogical Value | |-----------|----------|-------------------| | **LLM generation** | Multiple prompting strategies for simplification | Compare approaches for different learner needs | | **Human evaluation** | Educator/expert quality ratings | Ensure pedagogical (not just linguistic) quality | | **Meta-evaluation** | Framework for comparing simplification approaches | Systematic improvement of content adaptation | ## Connection to LLMs in Education As **LLMs become prevalent in ITS** (cf. [[ai-tutor-effectiveness-review]]), text simplification faces: - **Prompting strategy variability:** Same LLM, different prompts → different simplifications - **Architecture differences:** GPT vs. Claude vs. specialized models - **Evaluation challenge:** Linguistic metrics (BLEU, SARI) don't capture *pedagogical* quality MuTSE fills this gap: **pedagogical evaluation** of simplification, not just linguistic metrics. ## Implications for Adaptive Learning ### For ITS Design - **Content adaptation layer:** Dynamic text simplification as part of [[adaptive-learning]] - **Learner model integration:** Simplify based on real-time reading level assessment - **Multi-modal extension:** Could extend to diagram/math notation simplification ### For Educator Workflows - **Human-in-the-loop:** Teachers validate automated simplifications (cf. [[human-in-the-loop-ai]]) - **Quality assurance:** Prevent oversimplification (losing key concepts) or undersimplification (frustrating learners) ## Connected Concepts - [[zone-of-proximal-development]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] ## Connected Articles - [[ai-tutor-effectiveness-review]] ## Citation al, A.R.R.G.P.A.D.E. (2026). [*Text Simplification for Intelligent Tutoring*](https://arxiv.org/abs/2604.08947) --- ## [The Scaffolded AI literacy (SAIL) framework: Results of a Delphi study for equitable AI literacy framework design in education](https://edtechdev.github.io/aied/articles/the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl/) > **Synthesis:** MacCallum, Parsons, and Mohaghegh (2026) report on a three-round Delphi study that created the Scaffolded AI Literacy (SAIL) framework — a broadly applicable, age-agnostic framework for developing equitable AI literacy across all stages of education. Unlike most existing frameworks, which aggregate older literature or focus on non-generalizable contexts, SAIL provides a scaffolded competency pathway across levels and explicitly addresses second- and third-level digital divides. Grounded in Aotearoa New Zealand education while incorporating international and Indigenous perspectives, the framework organizes AI literacy into three domains (AI Concepts; Application of AI and Technical Skills; AI Digital Citizenship) and six categories, progressing through four levels (Understand and Explore AI → Apply and Integrate AI → Evaluate and Create AI → AI++, beyond AI literacy). ## Key Findings - The SAIL framework is organized into **three domains and six categories**: (1) *Concepts of AI* (The Impacts of AI; What AI Is and How It Works), (2) *Application of AI and Technical Skills* (Cognitive Skills; Applied Skills), and (3) *AI Digital Citizenship* (Social, Cultural, and Ethical Issues; Risks and Mitigations). Each category is detailed across four levels. - The framework defines **four scaffolded levels** (three plus one): *Level 1 - Understand and Explore AI*, *Level 2 - Apply and Integrate AI*, *Level 3 - Evaluate and Create AI*, and *AI++ - Transform and Develop AI (Beyond AI Literacy)* — the last marking the move beyond literacy into expert practice. - SAIL is **age-agnostic**: Level 1 concepts are considered essential for all ages, while higher levels are introduced at appropriate ages and stages, with educators adapting delivery to context. - The framework explicitly addresses the **three-level digital divide** (van Deursen & van Dijk): access/infrastructure, skills and competencies, and outcomes/benefits — arguing that equity requires more than closing the device gap; it requires building skills to use AI effectively and critically so benefits are distributed fairly. - It incorporates **Indigenous perspectives** (Māori data sovereignty, cultural misrecognition, equitable participation) as a core competency strand ("Indigenous Perspectives on AI"), reflecting Aotearoa New Zealand's international leadership in bicultural education and Indigenous data sovereignty. - The study reframes AI literacy to include critical awareness of bias (data-related, algorithmic, user-interaction bias), equity, governance, and cultural diversity — distinct from general digital literacies. ## Study Design & Method The authors conducted a **three-round Delphi study** (an expert-panel consensus method) between May 2023 and May 2024. An expert panel comprising educators (tertiary, high school, primary, and support roles) and industry professionals (data scientists, AI leads, founders) was recruited from Aotearoa New Zealand and internationally (Australia, Canada, South Africa, the UK, USA), with deliberate attention to Indigenous voices. Round 1 (17 fully completed responses) asked experts to identify the knowledge, skills, and understandings needed at each level of an initial four-level maturity model (Informed → Empowered → Engaged → Active participant); responses were thematically coded. Round 2 asked experts to rank competencies by importance and flag misplaced, irrelevant, or missing items, leading to rationalization into three domains and six categories. Round 3 validated the framework structure and refined competency wording. Following the Delphi, the draft was opened to wider review through communities of practice and an online feedback form, yielding further refinements (e.g., renaming levels, separating "AI++ — beyond AI literacy," renaming the "Issues, Challenges, and Opportunities" domain to "AI Digital Citizenship"). ## Implications for AI in Education The SAIL framework offers a research-informed, empirically grounded alternative to literature-review-based [[ai-literacy]] frameworks, providing educators at all levels with a structured pathway for [[scaffolding|scaffolded]] competency development that is equitable and broadly applicable across contexts. It directly addresses the [[digital-divide]] — arguing AI literacy is a central mechanism for addressing structural barriers in education by linking access, capability, and outcomes to social justice. The framework supports curriculum designers, teacher educators, and policy makers (e.g., UNESCO-style competency frameworks) in structuring AI literacy development from early years through higher education, and its inclusion of Indigenous perspectives and emphasis on ethical, cultural, and risk-mitigation competencies positions AI literacy as inseparable from equity and [[ethics|ethical]] participation in an algorithmically shaped society. ## Limitations The Delphi study had a modest fully-completed response rate in Round 1 (17 respondents) and was deliberately grounded in Aotearoa New Zealand, potentially limiting direct generalizability despite international participation. Under ethical approval, data could not be publicly shared to protect panel anonymity. The framework's broad, age-agnostic design means educators must adapt delivery to their specific contexts, and the authors note the field is rapidly evolving — future work should examine how emerging frameworks align with, extend, or diverge from SAIL and integrate insights across approaches. The introductory framework review was not a systematic literature review but contextual framing. ## Connected Concepts - [[ai-literacy]] - [[curriculum-design]] - [[scaffolding]] - [[digital-divide]] - [[equity]] - [[ethics]] - [[higher-ed]] ## Connected Articles - [[ai-literacy-heptagon-2026]] — The AI Literacy Heptagon - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy - [[genai-literacy-training-teacher-education-dbr-2026]] — Development and Evaluation of AI Literacy Training for Teachers - [[sec-ai-literacy-narrative-review-2026]] — Social-Emotional Competence in AI Literacy - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] — Metacognitive AI Literacy: Going Beyond the Skills Gap Agenda ## Citation MacCallum, K., Parsons, D., & Mohaghegh, M. (2026). [*The Scaffolded AI literacy (SAIL) framework: Results of a Delphi study for equitable AI literacy framework design in education*](https://doi.org/10.1016/j.caeai.2026.100584). *Computers and Education: Artificial Intelligence*. --- ## [The synergy of pedagogical agents and metaphorical design: Reducing psychological distance to enhance video learning](https://edtechdev.github.io/aied/articles/the-synergy-of-pedagogical-agents-and-metaphorical-design-reducing-psychological/) > **Synthesis:** This study examined the effects of pedagogical agents (real vs. virtual) and metaphorical design on learners' performance, attention, comprehension, and psychological distance in a 2x2 between-subjects design with 129 learners. Results showed both virtual pedagogical agents and metaphorical design improved learning performance and reduced psychological distance. Metaphorical design specifically increased cognitive effort and enhanced virtual agents' perceived human-likeness. Spatial and hypothetical distances mediated the effects on learning performance. ## Key Findings This study examined the effects of pedagogical agents (real vs. virtual) and metaphorical design on learners' performance, attention, comprehension, and psychological distance in a 2x2 between-subjects design with 129 learners. Results showed both virtual pedagogical agents and metaphorical design improved learning performance and reduced psychological distance. Metaphorical design specifically increased cognitive effort and enhanced virtual agents' perceived human-likeness. Spatial and hypothetical distances mediated the effects on learning performance. ## Connected Concepts - [[agentic-ai]] ## Connected Articles - [[savvy-student-attention-video-learning]] - [[nuclear-diffusion-text-to-image-learning-2026]] - [[learning-to-prompt-adaptive-tutoring]] - [[icap-cognitive-engagement-llm-agents]] ## Citation## Citation Xu, K., Zhang, Q., Zhang, Y., Xu, H., Pi, Z., & Yang, J. (2026). [*The synergy of pedagogical agents and metaphorical design: Reducing psychological distance to enhance video learning*](https://doi.org/10.1111/bjet.70064). *British Journal of Educational Technology*. --- ## [TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions](https://edtechdev.github.io/aied/articles/tibetcpr-ai-training-feedback/) **Yibo Meng, Ruiqi Chen, Zhiming Liu, Xiaolan Ding** — Accepted at MobileHCI 2026 — cs.HC TibetCPR is a low-cost, self-guided CPR training system that pairs depth-driven electrotactile feedback with rhythm-driven visual cues within a Tibetan-language narrative, designed for regions where instructor-mediated training is fragmented and learners' linguistic/educational backgrounds are heterogeneous. In a randomized study with 40 lay community members aged 19–56, the experimental group showed progressive minute-by-minute stabilization of rhythm and depth across a 10-minute intervention, substantially exceeding unguided-practice control, with gains transferring to an unscaffolded post-test (SUS=84.3). The paper synthesizes three transferable design principles for self-guided embodied training: feedback as a calibration reference rather than an immediate corrector; modality temporal granularity matched to behavior's temporal structure; and autonomous interpretability as a deployment prerequisite. These principles connect to [[formative-assessment]] and [[scaffolding]] in skills training contexts, and the feedback-as-calibration approach has implications for [[intelligent-tutoring]] systems that provide real-time guidance in embodied learning tasks. ## Connected Concepts - [[formative-assessment]] - [[scaffolding]] - [[intelligent-tutoring]] ## Connected Articles - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[slidesqaqa-pedagogical-question-generation]] — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation - [[retrieval-augmented-tutoring-algorithm-kite]] — Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education - [[correct-answer-trap-ai-tutor]] — Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) ## Citation Meng, Y., Chen, R., Liu, Z., & Ding, X. (2026). [*TibetCPR: A Multimodal Tactile Feedback System to Enhance Cardiopulmonary Resuscitation Training in High-Altitude Regions of Tibet*](https://arxiv.org/abs/2606.07765). arXiv:2606.07765. --- ## [A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI](https://edtechdev.github.io/aied/articles/tool-invariant-framework-agentic-ai/) > **Larry Engelhardt (Francis Marion University)** — *arXiv:2607.15518* [physics.ed-ph], submitted 17 Jul 2026. CC BY 4.0. doi:10.48550/arXiv.2607.15518. **Larry Engelhardt (Francis Marion University)** — *arXiv:2607.15518* [physics.ed-ph], submitted 17 Jul 2026. CC BY 4.0. doi:10.48550/arXiv.2607.15518. 📄 arXiv full text (HTML) — open access; supplement tables (task-by-task progression, oral-defense rubric) as [ancillary PDF](https://arxiv.org/src/2607.15518v1/anc/supplement_pdf.pdf). > **Note on type:** This is a *framework / position essay* (first-person, draws on the author's teaching practice), not a controlled empirical study. Its claims are argued and illustrated, with open questions explicitly flagged as untested. It is tagged `confidence: high` for fidelity to the source and internal rigor, but the design prescriptions are the author's and largely await validation (Fall 2026 cohort). ## Summary Agentic AI — systems that *write, run, and revise* simulation code from natural-language specs — is the latest step in a centuries-long migration of mechanical work from human to tool. Engelhardt argues that **what a learner must know is tool-invariant**: across tools (fingers → pencil → calculator → Python → agent), the requirements are stable; only their *content* and *weight* shift. The paper organizes these into **five pillars**, argues that **sensemaking/verification is now the load-bearing skill** (because AI-generated artifacts are *bespoke* and *unvalidated*, unlike socially-validated libraries), and draws the assessment consequence: **when artifacts can be generated on demand, the artifact no longer certifies the student.** The proposed response: **AI-free in-class coding quizzes** (measure white-box residue) + **oral defenses of comment-stripped, AI-assisted work** (measure orchestration), with a *verification gate* that must pass regardless of total score. ## The five pillars (tool-invariant) 1. **Inputs and outputs** — what goes in, what comes out, conditions of validity; problem posing. 2. **Method concept** — a working model of what the method does, including its knobs (step size, tolerance) and characteristic failure modes (instability, divergence, overfitting, aliasing). 3. **Terminology** — precise disciplinary vocabulary; *now an input channel*, since vague natural-language specs silently cede choices to the tool. 4. **Sensemaking** — judging whether outputs make sense and *establishing* correctness (verification/validation). Always present; becomes **load-bearing** under agentic AI. 5. **Operating the tool** — the actuation skill: once penmanship, now *directing an agent* (broad, shallow "interactional expertise" rather than narrow deep syntax). The pillars organize into a workflow: **Specify → Predict → Delegate → Verify → Interpret; Iterate**, threaded by **calibrated reliance** (how much verification is owed, given tool + task). ## Why verification is newly load-bearing (the core argument) Opacity was never the problem — *validation* is. A library routine (FFT, linear algebra) is opaque but **socially validated** (published algorithms, decades of testing, millions of users). An AI-generated simulation is opaque **and bespoke**: a population-of-one artifact from a stochastic process whose competence frontier is "jagged and invisible," whose failures arrive *disguised as successes* (running code, smooth plots, confident prose). The verification burden that ecosystems amortized across a community now lands on **each student, for each artifact, every time.** **Principle of validation authority:** *delegation is safe exactly where the delegator retains the ability to validate the outputs.* A computational-physics course now exists to train validation authority. Corollary: "anyone can build software with AI" holds only where correctness is *observable in use* (dashboards, apps); a physics simulation's correctness must be *established* by disciplinary checks. ## What must remain human (non-delegables) Posing the problem · choosing & owning the physical model/assumptions · the pre-execution prediction · specifying the checks · final epistemic responsibility ("the AI said so" is never a justification). Items 1, 2, 5 are constitutive of doing science, not claims about current AI capability, and do not weaken as models improve. ## Assessment design (the constructive response) - **Proxy collapse:** traditional "write code → submit report" grading died because the artifact no longer certifies the student (Goodhart's law; Kortemeyer's assessment alarm). Supervised formats survive. - **The product** is the student's ability to *explain and defend* artifacts in the discipline's language — certified, as at the Ph.D. level, by **oral defense**. - **Two instruments:** (1) **AI-free in-class coding quizzes** in a lockdown browser (assess the white-box phase / coding residue); (2) **ten-minute oral defenses** of AI-assisted work, with code **comments stripped** beforehand so understanding can't be performed by reading borrowed narration. The defense probes, live and adaptively: walkthrough of uncommented code, plot interpretation, and *verification probes* ("why should I believe this?", "what did the AI decide that you didn't?"). - **Verification gate:** the rubric scores five dimensions (code comprehension, method understanding, physics model/terminology, interpretation, verification); the verification dimension must reach "functional" for the defense to pass, regardless of total. - **Scalability:** honest arithmetic — ~28 contact-hours of defenses/semester for ~15 students (less than grading 15 reports, more informative); degraded modes (spot-defenses, TA-led, paired) named with costs. "It doesn't scale" is "partly the point" — a equity concern for under-resourced institutions. ## Teaching practices - **White-box, then black-box** (Buchberger): study a method transparently (hand-code the 15-line Euler integrator, watch it fail at large dt) *before* delegating. Dissolves the "must code vs. need not code" debate — both true at different rungs. - **Error injection:** give students subtly-wrong agent output (sign error, dt too large, wrong potential); grade the *diagnosis*. Trains reading code one didn't write. - **Motivation over prohibition:** a guardrailed tutor (Fall 2025) helped engaged students but was useless to answer-seekers ("an unguardrailed model is two browser tabs away"). Framing: homework is the gym, not the job; AI is a forklift at the gym. Design for motivation, not bans. - Term "**comprehension debt**" (gap between code a system contains and code its maintainers understand) imported from software engineering as a risk of AI-assisted production. ## Connected Concepts - [[plagiarism-detection]] - [[formative-assessment]] - [[automated-essay-scoring]] - [[student-experience]] - [[ai-ed-evaluation]] - [[agentic-ai]] - [[reducing-ai-misuse]] - [[prompt-engineering]] ## Connected Articles - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: redesigning authentic assessment in an AI-mediated world - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[moral-panic-genai-classroom]] — Navigating the moral panic: encouraging appropriate use of GenAI in the classroom rather than condemning innovation as disruption - [[pchl-he-framework-genai-content-creation-2026]] — From Prompts to Verified Loops: The PCHL-HE Framework for Generative AI-Assisted Educational and Research Content Creation in Higher Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[trust-reliance-ai-education-2026]] — Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators ## Citation Engelhardt, L. (2026). A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI. [arXiv:2607.15518](https://arxiv.org/abs/2607.15518) [physics.ed-ph]. https://doi.org/10.48550/arXiv.2607.15518 --- ## [Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing](https://edtechdev.github.io/aied/articles/tracing-genai-literacy-interaction-patterns/) > Identifies interaction signatures of LLM literacy using Epistemic Network Analysis (ENA) on logs from 162 students. High-literacy students exhibit iterative, strategic refinement and dense cognitive networking, while low-literacy students rely on direct, linear commands. This work emphasizes that [[ai-literacy]] is a developmental capacity requiring structured [[scaffolding]] and [[prompt-engineering]] discipline. It connects to the need for [[curriculum-design]] that targets [[metacognition]] and [[agentic-ai]] rather than just syntax mastery. ## Key Findings - Interaction logs from 162 university students engaged in a GenAI-assisted abstract writing task were analyzed using Epistemic Network Analysis (ENA). - High-literacy students exhibit iterative refinement and strategic questioning, producing distinct interaction signatures in the process data. - Low-literacy students rely on direct generation commands, characterized by transactional, generation-oriented dependence on the tool. - GenAI literacy is not just a static score but a dynamic behavioral capability that shapes the human-AI collaboration process. - The work paves the way for data-driven literacy assessment and real-time interventions based on observed interaction behavior rather than self-report. ## Study Design & Method A total of 162 university students (M_age = 20.1) participated in a GenAI-assisted abstract writing task. Prior to the task, students completed a validated GenAI Literacy Test assessing technical understanding, interaction skills, and related constructs. Using Epistemic Network Analysis, the researchers modeled and compared the questioning strategies of students with varying GenAI literacy levels, examining how literacy manifests in the structure of actual human-AI collaboration. The study was presented at the First International Workshop on Advancing AI Literacy with Learning Analytics (AI-LIT) at LAK26. ## Implications for AI in Education The study demonstrates that process data can characterize GenAI literacy in ways that self-reported questionnaires cannot: whether a student actually prompts iteratively, refines outputs, and manages hallucinations in real time is observable in interaction logs. For [[learning-analytics]] practice, this suggests building dashboards and automated assessments around behavioral markers of literate use — for example, detecting the shift from direct generation commands toward iterative refinement. The authors propose data-driven interventions that help learners move from transactional use to epistemic collaboration, such as prompting students to draft their own summary first and asking the AI for critique to improve learning depth. For [[writing-education]] and [[ai-literacy]] instruction, the findings argue for curricula that treat literacy as a developmental, behaviorally observable capacity shaped by structured [[scaffolding]] and strategic prompting, rather than a fixed trait measured once. ## Connected Concepts - [[ai-literacy]] - [[scaffolding]] - [[prompt-engineering]] - [[curriculum-design]] - [[metacognition]] - [[agentic-ai]] ## Connected Articles - [[ase-26-agentic-software-engineering-curriculum]] — ASE-26: A Curriculum for Agentic Software Engineering as a Discipline - [[guided-llm-scaffolding-independent-learning]] — Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[finkelstein-principled-ai-education-2025]] — Principled AI Education Framework - [[prober-ai-inquiry-writing]] — Prober.ai: Gated Inquiry-Based Feedback via LLM-Constrained Personas for Argumentative Writing ## Citation Angxuan Chen & Jiyou Jia (2026). [Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing](https://arxiv.org/abs/2606.00040). arXiv:2606.00040. --- ## [Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education](https://edtechdev.github.io/aied/articles/trio-ethnography-llm-programming-education/) This experience report introduces trio-ethnography — structured dialogue between two computing educators with differing teaching philosophies and one undergraduate CS student — as a method for surfacing how educators' interpretations of students' AI use evolve. The central finding is that much AI-supported learning is invisible from the classroom: across three conversations, the student's lived-experience narratives revealed learning processes that neither educator could infer from observable behavior, prompting both to revise assumptions about AI use, assessment design, and transparency in [[cs-education]]. The paper complicates the dominant framing of [[student-experience]] research, which typically measures behavior (prompt logs, usage frequency) rather than interpretation. Educators here initially read student AI use through the lens of [[over-reliance]] risk, but dialogue revealed more nuanced self-regulation and [[scaffolding]]-seeking than surveillance-style observation suggested. This aligns with broader calls for [[ai-literacy]] frameworks that treat students as reflective agents rather than compliance subjects. Methodologically, trio-ethnography offers a low-cost reflective instrument for faculty development: rather than instrumenting classrooms with analytics, it uses sustained cross-perspective conversation to update pedagogical beliefs. As an experience report with n=3 it makes no generalizability claims, but it provides a replicable protocol for departments adapting programming instruction in the generative-AI era. ## Connected Concepts - [[cs-education]] - [[student-experience]] - [[over-reliance]] - [[scaffolding]] - [[ai-literacy]] ## Connected Articles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[youtube-frames-chatgpt-education]] — How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata - [[student-misconceptions-conditionals-loops-taxonomy]] — How Students (Mis)understand Conditionals and Loops -- A Taxonomy - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[prompt-problems-nl-programming-mistakes]] — Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis ## Citation Ren, McDowell & Zhou (2026). [Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education](https://arxiv.org/abs/2607.22463). arXiv:2607.22463. arXiv preprint. --- ## [Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators](https://edtechdev.github.io/aied/articles/trust-reliance-ai-education-2026/) > Pitts, Rani & Mildort (2026, AIED) show with 432 undergraduates that **higher trust in an AI assistant is associated with lower appropriate reliance**: students who trusted the assistant more were worse at discriminating correct from misleading AI suggestions during Python problem-solving. The relationship is non-linear and **moderated by AI literacy and need for cognition** — trust is not a safe proxy for appropriate use. ## The experiment - 432 undergraduates solved Python output-prediction problems with recommendations + explanations from an AI chatbot that included **accurate and intentionally misleading suggestions** - **Appropriate reliance** measured behaviourally: accepting correct suggestions, rejecting incorrect ones - Surveys captured trust, AI literacy, need for cognition, programming self-efficacy, programming literacy ## Findings - **Non-linear trust→reliance relationship**: higher trust → lower appropriate reliance (weaker discrimination between correct/incorrect recommendations) - **Moderators: AI literacy and need for cognition** significantly shaped how trust translated into reliance behaviour - Implication: interventions should target *calibration* — instructional and system supports that encourage reflective evaluation of AI assistance during problem-solving ## Connected Concepts - [[ai-literacy]] - [[metacognition]] - [[agentic-ai]] - [[llm]] - [[rag]] ## Connected Articles - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course - [[measuring-llm-tutors-teach-vs-solve]] — Measuring Whether LLM Tutors Teach or Solve: A Diagnostic for Educational Impact - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods in the Age of Agentic AI - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module - [[academiclaw-student-agent-benchmark]] — AcademiClaw: When Students Set Challenges for AI Agents - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adhd-video-segmentation-computing-education]] — Leveling the Playing Field: Temporal Video Segmentation for Individuals with ADHD in Computing Education - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[agent-voice-accents-k12-group-learning]] — Exploring How Agent Voice Accents Shape Human-AI Collaboration in K-12 Group Learning - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-literacy-debt]] — Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named - [[agentic-workflows-education]] — Agentic Workflows in Education - [[agents-that-teach-incidental-learning]] — Agents That Teach: Designing Incidental Learning Back into AI-Assisted Software Development - [[agreement-not-quality-llm-coding-verification]] — Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not G... - [[ai-adoption-training-public-sector]] — The Main Barrier to AI Adoption in the Public Sector is Lack of Training - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-agents-peer-learning-discourse]] — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community - [[ai-assessment-scale-reform]] — A bit of chaos and madness": The AI Assessment Scale and the work of assessment reform - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[ai-assisted-learning-modes-eeg]] — An exploratory behavioral and electroencephalographic study of artificial intelligence-assisted learning modes in hig... - [[ai-availability-student-motivation]] — Why Put in This Much Effort?": How AI Availability Shapes Students’ Motivation in Introductory Programming - [[ai-campus-wellbeing-tools]] — AI-Driven Tools for Enhancing Campus Well-being: Prevention and Intervention - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows ## Citation Pitts, G., Rani, N., & Mildort, W. (2026). [Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators](https://arxiv.org/abs/2604.01114). arXiv:2604.01114. AIED 2026. --- ## [TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics](https://edtechdev.github.io/aied/articles/turtleai-visual-programming-benchmark/) > **Synthesis:** Vision-language models (VLMs) have been explored for visual programming, where they generate code to solve visual tasks. However, most prior work focuses on visual programming for productivity; it remains unclear how well current VLMs perform on education-oriented visual programming and what factors ## Abstract Vision-language models (VLMs) have been explored for visual programming, where they generate code to solve visual tasks. However, most prior work focuses on visual programming for productivity; it remains unclear how well current VLMs perform on education-oriented visual programming and what factors limit their performance. To bridge this gap, we introduce T URTLE AI, a benchmark containing 823 tasks curated based on real-world visual programming tasks in the Turtle Graphics domain. Solving these tasks requires models to perceive geometric patterns, reason about spatial relationships, and synthesize Python code that faithfully reproduces geometric patterns. We evaluate 20+ VLMs, including GPT-5, GPT-4o, and Qwen2-VL-72B, and find that they struggle significantly, with most achieving succes ## Connected Concepts - [[human-in-the-loop-ai]] - [[pedagogical-llm-training]] - [[computational-thinking]] - [[stem-education]] - [[math-education]] - [[educational-policy-ai]] - [[formative-assessment]] - [[affective-tutoring]] ## Connected Articles - [[cstutorbench-slm-tutors]] — CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming - [[privacy-aware-classroom-incident-recognition-2026]] — Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[vocabulary-difficulty-prediction]] — What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[eduzone-llm-safety-k12]] — EduZone: A Framework for Evaluating LLM Safety for K-12 Students and Teachers ## Citation for, T.U.A.B.M.M., Graphics, V.P.I.T., Wen, C., Staub, J., tasks, T.U.A.A.B.C., visual, Y.E.A.L.E.A.B., & Rodriguez, Y.E.A.W.E.A. (2026). [*TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics*](https://arxiv.org/abs/2606.03626). T URTLE AI, each task requires a VLM to generate Python code that reproduces a target image --- ## [The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals](https://edtechdev.github.io/aied/articles/tutoring-effectiveness-index/) > **Authors:** Shim Jaechang, Unggi Lee (2026) — CIKM 2026 # The Tutoring Effectiveness Index (TEI) ## What It Is TEI is a **training-free, judge-free index** that selects the best tutoring response from multiple LLM candidate outputs using only four internal conversation signals — no RL training, no external judge model, no reward model required. ## How It Works TEI combines four signals computed during decoding with fixed weights: - **V (Verify ratio)**: Regex over thinking trace counting Schoenfeld Verification keywords ("let me check", "verify", "double-check") - **M̃ (Math-step density)**: Regex on visible output, min-max normalized within candidate pool - **Q (Ends-question rate)**: Regex detecting if tutor turn ends with a question - **D (Deep-reasoning gate)**: Binary, fires if ≥40% of tokens have JSD below threshold Formula: `TEI(y) = 1.0·V + 0.75·M̃ - 1.0·Q + 0.5·1[DTR ≥ 0.4]` The signs matter more than magnitudes: reward verification and math content, penalize ending with questions, small bonus for deep reasoning. ## Key Results - TEI@8 raises improvement rate on pre-incorrect scenarios from **59.0% to 81.9%** (+22.9 pp) on frozen DeepSeek-R1-8B, with no training - TEI@4 achieves 75.7%, beating both Random@4 (58.6%) and DTR@4 (61.2%) - Feature ablation: dropping Verify costs -0.054 AUC, dropping Math-steps costs -0.036, dropping Deep-reasoning gate costs only -0.009 - TEI@8 costs 4.1× tokens of greedy (16,334 vs 3,984), about half of self-consistency ## The Alignment Tax The paper quantifies severe degradation from pedagogical GRPO fine-tuning: - Thinking length drops from 1,764 to 119 words/turn (−93%) - Content-Knowledge accuracy falls by −71% relative - Pedagogical-Knowledge accuracy falls by −80% relative - Student Δ Solve Rate crosses from +0.180 to **−0.012** — the aligned tutor becomes detrimental ## Why It Matters TEI demonstrates that simple lexical and structural signals can effectively steer a frozen LLM to be a much better math tutor without any training. This is especially valuable when RL fine-tuning is shown to catastrophically degrade tutoring quality. The approach is cost-effective and immediately deployable on frozen models. ## Open Questions - Does TEI generalize to non-math tutoring domains (writing, science, language)? - Can the fixed weights be optimized per-domain without losing the training-free property? - How does TEI interact with different base model architectures and sizes? ## Connected Concepts - [[socratic-ai-dialogue]] - [[automated-question-generation]] - [[ai-ed-evaluation]] - [[open-source]] - [[pedagogical-llm-training]] - [[automated-assessment]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] ## Connected Articles - [[learning-by-chatting-genai-impact]] - [[aaai2026-prompting-literacy-k12]] - [[academiclaw-student-agent-benchmark]] - [[access-not-enough-ai-tutoring-2026]] - [[adaptive-pretesting-retention]] - [[agent-voice-accents-k12-group-learning]] - [[agentic-ai-education-scoping-review]] - [[agentic-ai-pedagogical-best-practice-2026]] - [[agentic-workflows-education]] - [[agents-that-teach-incidental-learning]] ## Citation Shim, J., & Lee, U. (2026). [The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals](https://arxiv.org/abs/2605.30666). arXiv:2605.30666. --- ## [Tutoring-Specific vs. General-Purpose AI in Education](https://edtechdev.github.io/aied/articles/tutoring-specific-vs-general-ai/) > 1. **Desirable difficulties** — General-purpose AI removes productive struggle; tutoring tools preserve it via graduated hints. 2. **Germane load** — Effective learning requires processing that feels effortful. General AI short-circuits this. See [[cognitive-load-theory]]. 3. **Metacognition suppression** — When AI completes reasoning, students lose practice in monitoring their own understanding. > The single most consistent finding in the 2026 Stanford SCALE review: **pedagogically designed, tutoring-specific AI consistently outperforms general-purpose chatbots** on durable learning outcomes.^[[stanford-evidence-base-ai-k12-2026]] ## The Core Distinction | Dimension | General-Purpose AI (e.g., ChatGPT, Gemini) | Tutoring-Specific AI | |---|---|---| | **Interaction model** | Open-ended Q&A; completes tasks on request | Structured hints, Socratic questioning, step-by-step scaffolds | | **Cognitive load** | Reduces *all* load, including germane (productive) load | Reduces extraneous load while preserving productive struggle | | **ZPD targeting** | Often operates outside the [[zone-of-proximal-development|zone of proximal development]] | Explicitly calibrated to learner readiness | | **Metacognitive demand** | Low — AI does the reasoning | High — learner must reason with guidance | | **Transfer evidence** | Mixed to negative when tool is removed | More promising (limited causal data) | ## Evidence from the Causal Literature ### General-Purpose AI: Mixed or Negative Transfer - **Bastani et al. (2025):** High schoolers using a general-purpose chatbot for math practice scored **~17% worse** on closed-book final exams than peers with no AI access, despite higher practice grades.^[[stanford-evidence-base-ai-k12-2026]] - **Lehmann et al. (2025):** General-purpose AI for programming increased topics covered but **harmed understanding** and widened achievement gaps for low-prior-knowledge students. - **Stadler et al. (2024):** General-purpose AI produced **lower-quality reasoning and argumentation** vs. traditional search. - **Kosmyna et al. (2025):** AI essay assistance led to **83% of participants failing to recall a quote** from their own essay, vs. 11% for non-AI users. ### Tutoring-Specific AI: Better Outcomes - **Bastani et al. (2025):** A tutoring-specific chatbot with pedagogical guardrails (**hints, step-by-step reasoning, misconception targeting**) **mitigated the exam score drop** observed with general-purpose GPT. General-purpose GPT Base caused the drop; the tutoring variant prevented it. - **Kreijkes et al. (2026):** Retention improved when AI use was **paired with traditional strategies** like note-taking — suggesting that friction-preserving designs matter. ## Why This Happens: Learning Science Mechanisms 1. **Desirable difficulties** — General-purpose AI removes productive struggle; tutoring tools preserve it via graduated hints. 2. **Germane load** — Effective learning requires processing that feels effortful. General AI short-circuits this. See [[cognitive-load-theory]]. 3. **Metacognition suppression** — When AI completes reasoning, students lose practice in monitoring their own understanding. 4. **Expertise reversal** — Novices need scaffolding, not answers. General AI gives answers; tutoring AI gives scaffolds. ## Important Caveats - The causal comparison base is **tiny** (most studies are single-condition AI-access vs. no-access, not head-to-head tutoring vs. general). - "Tutoring-specific" is not yet a standardized design category — implementations vary widely. - Long-term transfer data (months or years out) is essentially absent. ## Implications for Practitioners - **For tool selection:** Favor products with explicit pedagogical guardrails (hints, Socratic mode, step-by-step requirements) over raw LLM access. - **For policy:** School/district procurement criteria should distinguish between "AI-integrated" tools (tutoring-specific) and "AI-access" tools (general chatbox). - **For research:** Head-to-head RCTs comparing pedagogically designed AI vs. raw LLM access on delayed post-tests are urgently needed. ## Connected Concepts - [[cognitive-load-theory]] - [[zone-of-proximal-development]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] ## Citation Stanford SCALE Initiative. (2026). [*The Evidence Base on AI in K-12: A 2026 Review*](https://scale.stanford.edu/sites/default/files/The%20Evidence%20Base%20on%20AI%20in%20K-12%20Report.pdf). --- ## [Supporting Tutors in the Gig Economy with Automated Feedback: A Case Study on Ringle](https://edtechdev.github.io/aied/articles/tutors-gig-economy-automated-feedback/) Park et al. (2026) explore AI-powered automated feedback for tutors on Ringle, a popular online English tutoring platform in the gig economy. Their research probe analyzed tutors' lessons and provided automated feedback, followed by a survey of 36 tutors. Findings reveal that while tutors perceived automated feedback more negatively than learner feedback, they valued it for self-monitoring and understanding platform expectations. However, discrepancies between automated and learner feedback often caused confusion. The study proposes design considerations for feedback systems on educational gig platforms. This work contributes to [[ai-feedback-quality]] research by highlighting the social and affective dimensions of AI feedback in tutoring contexts, and connects to [[teacher-role]] evolution and [[intelligent-tutoring]] system design in [[language-learning]] and [[higher-ed]] settings. ## Connected Concepts - [[ai-feedback-quality]] - [[teacher-role]] - [[intelligent-tutoring]] - [[language-learning]] - [[higher-ed]] ## Connected Articles - [[teaching-feedback-classification-benchmark]] — A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[care-full-feedback-genai]] — The care-full craft of feedback in an age of generative AI - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[automated-presentation-coaching]] — A Survey of Automated Presentation Coaching: Systems, Methods, and Open Challenges ## Citation Yeon Su Park, Sieun Kim, Keighley Overbay, Seoyoung Kim, Sewook Wee, Daho Jung, Juho Kim (2026). [Supporting Tutors in the Gig Economy with Automated Feedback: A Case Study on Ringle](https://arxiv.org/abs/2606.22609). arXiv:2606.22609. arXiv:2606.22609 (cs.HC) --- ## [Thinking Through AI: Advancing Cognitive and Collaborative Research for AI in Education](https://edtechdev.github.io/aied/articles/tzirides-thinking-through-ai-2025/) > **Synthesis:** Tzirides, Galla, Cope & Kalantzis (2025) introduce the "Thinking Through AI" framework combining cognitive labs, think-aloud protocols, and cyber-social methods. A pilot with 30 students at a rural Alaska school showed how AI-enabled tools can enhance middle school writing instruction, with the framework positioning educators and learners as co-creators in AI tool design. ## Three Core Dimensions - **Student cognitive engagement:** Fostering learning objectives while improving AI systems through interactive feedback. - **Educator pedagogical strategies:** Integrating rubrics and knowledge bases to design AI tools for student needs. - **Concurrent design:** AI tools and learning experiences informed by think-aloud and cognitive labs (Learning Experience Research). ## Alaska Implementation A pilot with **30 students** at a rural Alaska school using the CGScholar AI Helper demonstrated practical application for [[writing-education]] in [[k-12]] middle school contexts. The study addresses [[cognitive-offloading]] concerns by designing for active cognitive engagement rather than passive AI reliance. ## Implications This framework connects to [[collaborative-learning]] and positions [[ai-literacy]] as a co-design process, not just consumption. The think-aloud methodology aligns with Learnlab research on cognitive process analysis during AI-assisted learning. ## Connected Concepts - [[ai-education]] - [[writing-education]] - [[k-12]] - [[ai-literacy]] - [[cognitive-offloading]] - [[collaborative-learning]] ## Connected Articles - [[agency-gap-ai-writing]] - [[cstutorbench-slm-tutors]] - [[code-anchor-multi-view-visualization]] ## Citation Tzirides, A., Galla, M., Cope, B., & Kalantzis, M. (2025). [*Thinking Through AI: Advancing Cognitive and Collaborative Research for AI in Education*](https://doi.org/10.35542/osf.io/s8hqe_v1). EdArXiv. doi:10.35542/osf.io/s8hqe_v1. --- ## [Understanding Student Effort Using Response-Time Propensities During Problem Solving](https://edtechdev.github.io/aied/articles/understanding-student-effort-response-time/) # Student Effort Measurement via Response-Time Propensities ## Key Findings Adaptive learning systems produce substantial learning gains, yet many students engage too briefly or superficially to benefit. This paper addresses the central challenge of **measuring student effort** during multi-step problem solving using response-time propensities. The authors develop models that infer student effort from response-time patterns, finding that common log-based proxies like time-on-task are insufficient. They propose response-time propensity modeling as a more nuanced approach to identifying when students are superficially engaging versus deeply working through problems. ## Connections to AIED This work directly connects to [[learning-analytics]] by providing a validated method for inferring effort from interaction logs. It has implications for [[adaptive-learning]] that could use early effort detection to intervene before students disengage. The approach also relates to [[engagement-assessment-video]] and broader [[assessment]] frameworks, suggesting that response-time analysis could complement other engagement signals in multi-modal effort detection systems. ## Connected Concepts - [[learning-analytics]] - [[adaptive-learning]] - [[assessment]] ## Connected Articles - [[engagement-assessment-video]] ## Citation Borchers, C., Zhang, L., Yang, K., Nagashima, T., & Domingue, B. W. (2026). [Understanding student effort using response-time propensities during problem solving](https://arxiv.org/abs/2605.08943). arXiv:2605.08943. --- ## [The University AI Didn''t Replace: Rethinking Universities in the AI Era](https://edtechdev.github.io/aied/articles/universities-ai-era-rethinking/) > **Synthesis:** Rather than replacing universities, generative AI **redefines their essential functions** — this paper proposes a four-level framework of institutional AI adoption and argues that the central challenge is moving from isolated, individual-driven experimentation to strategic integration, supported by workload and recognition systems. ## Core Argument The paper positions AI not as an existential threat to universities but as a **catalyst for reimagining** what universities do. Drawing on a case study of AI-enabled curriculum initiatives across several units at one institution, the authors contend that most universities remain in the early stages of adoption, where AI innovation occurs **informally and without institutional recognition**. The key institutional challenge is therefore not technological provision but **alignment**: redesigning learning around AI-supported reasoning and aligning policies, workload models, and recognition systems to support educational transformation. This connects to broader discussions in [[ai-higher-ed-bridge-gap]] about the frameworks needed for higher education, and complements the [[ai-higher-ed-workforce-survey]] finding that most staff already use AI for work. ## The Four Levels of AI Adoption The paper's central framework describes four stages of institutional response, ranging from defensive restriction to full integration, noting that many institutions presently operate around **Level 0 or Level 1**: 1. **Level 0 — Defensive Containment:** AI is treated primarily as a threat to academic integrity. Responses focus on restricting use (discouraged or banned in assessment, heavy reliance on invigilated exams) with little or no curriculum redesign — often creating a mismatch between policy and the reality of widespread student AI use. 2. **Level 1 — Informal or Peripheral Adoption:** Experimentation happens largely at the level of individual educators rather than through institutional strategy, with limited or inconsistent guidance for students, no formal recognition in workload or promotion, and uneven adoption across units. Innovation thus depends on motivated individuals and can burden those leading change. 3. **Level 2 — Strategic Integration:** Universities begin embedding AI into teaching policy and curriculum design, with clear institutional principles, curriculum redesign for AI-rich environments, professional AI development for educators, and recognition of innovation in workload or funding. Teaching increasingly focuses on reasoning, interpretation, and responsible use. 4. **Level 3 — AI-Embedded or Transformational Universities:** AI becomes part of core educational infrastructure and the learning ecosystem — integrated into the LMS, assessment, and learning support; curricula centred on AI-supported reasoning rather than content delivery; assessments that evaluate judgment and justification in AI-rich environments; and students working with AI as a cognitive partner in authentic problem-solving. ## Key Strategic Steps To move beyond Level 1, the paper recommends concrete institutional actions: **recognise AI-driven curriculum redesign in workload models** so innovation is not dependent on unrecognised effort; **embed AI-enabled teaching innovation in promotion and teaching-award criteria**; **establish clear institutional principles** for AI use in learning and assessment; **redesign assessment toward reasoning and justification** in AI-rich environments rather than merely tolerating AI; and **create institutional pilots or funded initiatives** that scale successful innovations beyond individual courses. ## Connections to Wiki - Shares the institutional perspective with [[ai-education-global-capacity]] on human/institutional capacity bottlenecks - Contrasts with the [[ai-adult-learning-design]] focus on learner-facing technology guidelines - Extends [[teacher-ai-competency]] from individual educators to institutional competency — institutional readiness is a distinct level above individual educator skill - The workload-and-recognition emphasis speaks to [[faculty-development]] centers navigating GenAI adoption, and the assessment implications echo [[assessment-validity]] concerns about what assessment means when AI can produce university-level work - Relevant to [[regulation]] discussions about higher-education policy and [[principled-ai-education]] frameworks ## Open Questions - How do different national contexts (US, EU, Global South) shape university AI responses? - What is the timeline for meaningful institutional transformation vs. superficial adoption? - How does institutional rethinking interact with [[principled-ai-education]] frameworks? - Which structural incentives most effectively convert informal educator experimentation into scaled, recognized innovation? ## Connected Concepts - [[faculty-development]] - [[student-experience]] - [[assessment-validity]] - [[teacher-ai-competency]] - [[regulation]] ## Connected Articles - [[ai-higher-ed-bridge-gap]] - [[ai-higher-ed-workforce-survey]] - [[authentic-assessment]] - [[ai-education-global-capacity]] - [[ai-adult-learning-design]] - [[principled-ai-education]] ## Citation Binkowski, K. P., & Hopkins, A. (2026). [*The University AI Didn't Replace: Rethinking Universities in the AI Era*](https://arxiv.org/abs/2605.07056). arXiv:2605.07056. --- ## [Unveiling patterns of socially shared regulation in relation to self-regulated learning: The roles of individual profiles and group dynamics in online collaborative learning](https://edtechdev.github.io/aied/articles/unveiling-patterns-of-socially-shared-regulation-in-relation-to-self-regulated-l/) > **Synthesis:** This study employed a three-layer analytical method combining cluster analysis, content analysis and complex network analysis to investigate how socially shared regulation of learning (SSRL) patterns evolve in relation to individual self-regulated learning (SRL) profiles. Data from 60 undergraduates in a 16-week course with over 16,000 trace entries revealed three SRL profiles based on time investment, study regularity and help-seeking behaviours. Groups with higher SSRL interaction displayed more diverse and balanced role composition, and distinct SSRL patterns emerged across SRL profiles over time. ## Key Findings This study employed a three-layer analytical method combining cluster analysis, content analysis and complex network analysis to investigate how socially shared regulation of learning (SSRL) patterns evolve in relation to individual self-regulated learning (SRL) profiles. Data from 60 undergraduates in a 16-week course with over 16,000 trace entries revealed three SRL profiles based on time investment, study regularity and help-seeking behaviours. Groups with higher SSRL interaction displayed more diverse and balanced role composition, and distinct SSRL patterns emerged across SRL profiles over time. ## Connected Concepts - [[self-regulated-learning]] - [[collaborative-learning]] ## Connected Articles - [[learning-to-learn-in-the-age-of-generative-ai-a-scoping-review-and-conceptual-fr]] - [[from-emotion-regulation-to-academic-success-a-self-determination-theory-based-em]] - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] ## Citation## Citation He, T., Wu, X., Li, M., Xia, T., & Cao, X. (2026). [*Unveiling patterns of socially shared regulation in relation to self-regulated learning: The roles of individual profiles and group dynamics in online collaborative learning*](https://doi.org/10.1111/bjet.70082). *British Journal of Educational Technology*. --- ## [Towards Valid Student Simulation with Large Language Models](https://edtechdev.github.io/aied/articles/valid-student-simulation-llm-2026/) > Yuan et al. (2026) present a conceptual and methodological framework for valid LLM-based [[simulating-students|student simulation]]. They identify the **competence paradox** — broadly capable LLMs asked to emulate partially knowledgeable learners produce unrealistic error patterns and learning dynamics — and reframe student simulation as a constrained generation problem governed by an explicit **Epistemic State Specification (ESS)** that defines what a simulated learner can access, how its errors are structured, and how its state evolves over time. The paper argues for **epistemic fidelity over surface realism** as a prerequisite for using simulated students as reliable scientific and pedagogical instruments. ## The competence paradox The core failure mode: LLMs are capable agents, so when asked to play a "student who doesn't know the material," they tend to either answer too well or to err in ways that don't match how a real learner at that level actually struggles. This produces simulated learners whose error patterns and learning trajectories are unrealistic, undermining the validity of any conclusions drawn from them. ## The framework - **Epistemic State Specification (ESS):** student simulation is treated as a constrained generation problem. The ESS defines three things: (1) what knowledge the simulated learner can access, (2) how errors are structured for that learner, and (3) how the learner's state evolves over time (i.e., learning dynamics). - **Goal-by-Environment framework:** situates simulated student systems according to their behavioral objectives and deployment contexts, clarifying what a given simulation is meant to achieve. Rather than proposing a new system or benchmark, the paper synthesizes prior literature, formalizes the key design dimensions of student simulation, and articulates open challenges around **validity, evaluation, and ethical risk**. ## Implications The paper makes the case that **validity (epistemic fidelity) must come before surface realism**: a simulated student that merely sounds plausible but does not reflect the intended learner's genuine knowledge state cannot support reliable conclusions about pedagogy or educational AI. This connects directly to the challenge of validating simulated learners before using them to test tutoring systems, curricula, or teacher training — and to the concern that superficially fluent simulations may mask underlying invalidity. ## Connected Concepts - [[simulating-students]] - [[student-modeling]] - [[llm]] - [[generative-ai]] - [[adaptive-learning]] - [[ai-ed-evaluation]] - [[trust-calibration]] - [[ethics]] ## Connected Articles - [[simulating-students-diverse-cognitive-levels-2025]] — Embracing Imperfection: Simulating Diverse Cognitive Levels - [[simulating-students-llm-review-2026]] — Simulating Students with LLMs: A Review - [[agentschool-multi-agent-simulation-education-2026]] — AgentSchool: Multi-Agent Simulation for Education - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? - [[history-aware-student-simulation]] — History-Aware Profiles for Student Simulation - [[llm-student-simulation-teacher-insights]] — Can LLMs Simulate Human Learners? ## Citation Yuan, Z., Xiao, Y., Li, M., Xuan, W., Tong, R., Diab, M., & Mitchell, T. (2026). [*Towards valid student simulation with large language models*](https://arxiv.org/abs/2601.05473). arXiv:2601.05473. --- ## [VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding](https://edtechdev.github.io/aied/articles/veriforge-narrative-drafting-scaffolding-2026/) > **Synthesis:** Sun et al. (2026) present VeriForge, a mixed-initiative [[generative-ai]] writing system that assumes initiative over domain discovery while the author retains initiative over narrative synthesis, using proactive highlighting, dual-stream querying with source-anchored Knowledge Cards, and a spatial Knowledge Canvas to surface latent knowledge gaps without homogenizing voice. ## Key Findings 1. **The core problem is latent knowledge gaps.** Current AI writing tools require explicit queries authors cannot formulate, generate finished prose that risks homogenizing voice, or assist only within what authors already know. VeriForge targets the discovery and integration of unfamiliar domain knowledge. 2. **Mixed-initiative division of labor.** Grounded in interviews with 9 fiction writers, VeriForge divides cognitive labor: the system takes initiative over domain discovery while the author keeps full initiative over narrative synthesis, preserving writer agency. 3. **Three complementary mechanisms.** Proactive inline highlighting flags potential knowledge gaps as authors draft; dual-stream querying pairs conversational responses with source-anchored Knowledge Cards for direct fact extraction; a spatial Knowledge Canvas lets authors organize and connect discovered knowledge, all powered by a graph-based [[rag]] pipeline. 4. **Promising preliminary evidence.** A within-subjects user study (N=12) showed the paradigm helps authors recognize previously overlooked knowledge gaps, supports creative exploration, and produced passages rated by experts as having stronger domain grounding in a cold-start task. ## Implications VeriForge addresses a central tension in [[writing-education]] with AI: how to leverage [[generative-ai]] for knowledge augmentation without ceding authorship and voice. The mixed-initiative design is a concrete instantiation of [[human-in-the-loop-ai]] collaboration, offering a counterpoint to [[cognitive-offloading]] concerns by making writers aware of what they do not know rather than silently filling gaps. For [[creativity]] research, the system's emphasis on preserving author agency over synthesis speaks directly to worries about [[over-reliance]] and homogenized output. The knowledge-gap-revealing mechanism also connects to [[scaffolding]] theory and [[zone-of-proximal-development]], positioning the AI as a scaffold that surfaces and supports what the writer can learn rather than replacing it. The graph-based [[rag]] pipeline with source-anchored evidence cards also models best practice in [[ai-feedback-quality]] and [[hallucination-risk]]: surfaced knowledge is traceable to sources, mitigating [[hallucination-risk]]. The study offers design guidance for [[llm]]-based writing tools across domains beyond fiction, including academic and expository writing. ## Connected Concepts - [[cognitive-offloading]] - [[ai-feedback-quality]] - [[creativity]] - [[generative-ai]] - [[hallucination-risk]] - [[human-in-the-loop-ai]] - [[knowledge-graph]] - [[over-reliance]] - [[rag]] - [[scaffolding]] - [[hallucination-risk]] - [[writing-education]] - [[zone-of-proximal-development]] ## Connected Articles - [[mindcopilot-llm-co-writing]] — MindCopilot co-writing - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Creative writing with robots - [[foxglove-writing-feedback-experts-llms]] — Foxglove writing feedback - [[cognitive-offloading-llm-synthesis-writing]] — Cognitive offloading in writing - [[self-referential-l2-writing-llm-assessment]] — L2 writing LLM assessment ## Citation Sun, R., Li, J., Tao, W., Zheng, X., Tan, Y., Wei, J., & Ma, Y. (2026). [*VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding*](https://arxiv.org/abs/2608.09698). arXiv:2608.09698 (cs.HC); UIST 2026. --- ## [VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI](https://edtechdev.github.io/aied/articles/vetting-dual-llm-safety-education/) > **Synthesis:** VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI ## Key Findings - VETTING is a dual-LLM architecture that separates response generation from policy verification: a generator LLM produces responses and a separate verifier LLM checks them against safety policies at runtime. - In an in situ deployment with 151 middle school students in a writing activity, VETTING achieved precision of .943, recall of .913 (95% CI [.652, .983]), and an F1 score of .928 for safety violation detection; human audit confirmed 50 true positives and 3 false positives, with an estimated 4.8 undetected violations. - The deployment was associated with an estimated 91.2% reduction in inappropriate content exposure, at the cost of a 19.6% increase in token usage (628,011 tokens, roughly 12,560 tokens per prevented violation); median response latency rose from 2.9 s to 6.6 s on verification-triggered turns (Mann-Whitney U = 46,770.5, p < .001, Cohen's d = 0.66). - Across 1250 student–AI interaction turns, only 51 turns (4.1%) triggered verification, concentrated in 25 conversations (16.6% of all conversations); the presence of risky keywords in prompts raised the odds of verification failure more than fourfold (OR = 4.59, 95% CI [2.04, 10.34], p < .001). - The work documents a taxonomy of student boundary-testing behaviors observed during authentic classroom use: eight violation categories, led by Romantic or Intimate Relationship Themes (30.2% of instances) and Inappropriate or Mature Topics (26.4%), with students frequently rephrasing or escalating prompts after verification failures. - Policy isolation keeps policy specifications hidden during interaction, making circumvention attempts monitorable and auditable rather than embedded in prompts. - An open-source Python implementation of the framework is available. ## Study Design & Method Educational AI systems increasingly rely on large language models to support student writing and inquiry, yet enforcing safety and instructional constraints during open-ended, multi-turn interaction remains challenging. Existing approaches commonly embed such constraints within conversational prompts or rely on static filtering; over time these approaches may become sensitive to user interaction, making it difficult to monitor and audit when students are able to circumvent or otherwise attempt to violate the measures. VETTING instead separates response generation from policy verification and applies explicit policy checks at runtime, illustrated through a grounded instantiation that enforces instructional and safety constraints without exposing policy specifications during interaction. The evaluation ran in a middle school classroom during a structured, timed writing activity: 190 students in grades 6–8 were given 45 minutes to write a 500-word essay on the advantages and disadvantages of AI in education, and 151 of them interacted with the chatbot. Every candidate response was checked by the verification layer before release; failed responses triggered an iterative rewrite loop bounded at three attempts before a fallback response was issued. Evaluation combined analysis of student–AI interaction behavior, human audit of verification outcomes against a thematic codebook, characterization of computational overhead, and a retrospective comparison with a single-LLM embedded-policy baseline. ## Implications for AI in Education Policy-isolated runtime verification supports the analysis and management of educational AI behavior under authentic classroom use. Because safety checks are explicit and separate from generation, instructors and developers gain an auditable record of when and how students attempt to violate constraints — and the documented taxonomy of boundary-testing behaviors can inform both system design and classroom guidance. The measured trade-off (a large reduction in inappropriate content exposure for a modest increase in token usage) gives practitioners a concrete cost-benefit estimate for this class of architecture. A retrospective comparison with a single-LLM baseline in which the safety policy was embedded in the system prompt found that 35.3% of the violations intercepted by VETTING would still have produced student-visible responses under embedded prompting — positioning separated runtime verification as a conservative, higher-control design point for high-risk contexts (minors, developmentally sensitive content, or tasks where direct answer provision would undermine the activity) rather than a necessary architecture for every application. For [[pedagogical-safety]] in [[k-12]] settings, VETTING offers a template that complements human oversight rather than replacing it, and its open-source implementation lowers the barrier to adopting verifier-based safeguards in [[human-in-the-loop-ai]] deployments. ## Limitations The authors caution that the taxonomy is exploratory rather than fully validated: categories were developed through collaborative discussion and subsequently applied by a single annotator, so formal inter-rater reliability metrics could not be computed. Recall estimates were derived from a sample-based audit of passed interactions, yielding wide confidence intervals due to the low base rate of violations. The evaluation was conducted within a single middle school writing activity centered on AI in education, which may have influenced both the content of student prompts and the types of violations observed, and it was not designed as a direct empirical comparison against strengthened prompt-based safeguards — the retrospective baseline does not reproduce full live-interaction dynamics. ## Connected Concepts - [[human-in-the-loop-ai]] - [[pedagogical-safety]] - [[formative-assessment]] - [[k-12]] - [[affective-tutoring]] - [[automated-essay-scoring]] - [[cs-education]] - [[student-experience]] ## Connected Articles - [[hazra-safetutors-pedagogical-safety-2026]] — SafeTutors: Pedagogical Safety in AI Tutoring - [[child-safety-genai]] — Child Safety in Generative AI: An Expert-Guided and Incident-Grounded Evaluation Framework - [[concept-catalyst-engineering-scaffolds]] — Creating Learning Scaffolds for Engineering Design Using Concept Catalyst - [[llm-unlearning-math-privacy]] — Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[llm-cultural-relevance-k12]] — LLMs for Culturally Relevant K-12 Pedagogy ## Citation Li, H., Zhang, S., & Botelho, A. F. (2026). [*VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI*](https://doi.org/10.1016/j.caeai.2026.100646). --- ## [From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs](https://edtechdev.github.io/aied/articles/vibe-coding-programming-process-visualizer/) Describes rapid development of a Thonny log visualizer using AI-assisted 'vibe coding' to make student programming processes visible to teachers. Piloted in a 160-student introductory programming course. Provides interactive timelines, session summaries, code-size graphs, and programming-process replays supporting teacher decision-making and academic-integrity clarification. **Relevance to AI in Education:** This paper contributes to the understanding of [[automated-assessment]], [[personalized-learning]], and [[student-experience]]. The findings have implications for [[adaptive-learning]] systems, [[formative-assessment]] design, and the broader [[edtech-platform]] landscape. Future work should explore how these results generalize across [[stem-education]] and [[higher-ed]] contexts. This research connects to the growing body of work on [[ai-literacy]] and [[teacher-role]], highlighting both the promise and limitations of AI tools in educational settings. ## Connected Concepts - [[automated-assessment]] - [[personalized-learning]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[edtech-platform]] - [[stem-education]] - [[higher-ed]] - [[ai-literacy]] - [[teacher-role]] ## Connected Articles - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[student-math-competence-clustering]] — Archetypes or ability? Clustering for modelling student mathematical competence - [[llm-psychometric-calibration-cdp]] — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[llm-student-misconception-identification]] — What Don't You Understand? Using Large Language Models to Identify and Characterize Student Misconceptions About Challenging Topics ## Citation Heidi Taveter, Marina Lepp (2026). [From Idea to Classroom in Days: Using Vibe Coding to Create a Programming Process Visualizer from IDE Activity Logs](https://arxiv.org/abs/2607.24757). arXiv:2607.24757. FIE 2026 (IEEE Frontiers in Education Conference). --- ## [Vibe Compiler: A Research-Logic Synthesis Tool That Runs without Prompt Engineering -Toward Enhancing Metacognition for Sustaining Agency in the Age of Generative AI-](https://edtechdev.github.io/aied/articles/vibe-compiler-metacognition-genai-agency-2026/) > **Synthesis:** This paper introduces the Synthesis-Analysis Reciprocity Model and the Vibe Compiler tool to preserve human epistemic agency during GenAI-assisted intellectual work. The model frames intellectual construction as a reciprocal interaction between Synthesis (combining components into a whole) and Analysis (decomposing a whole into components), arguing that over-reliance on AI for synthesis erodes metacognitive capacity. Vibe Compiler implements this model by requiring users to externalise their reasoning structure — specifying components, relationships, and constraints — before AI generates any output. In preliminary trials, the tool improved users' ability to articulate, critique, and refine AI-generated reasoning, suggesting that structured metacognitive scaffolding can mitigate the cognitive offloading risks identified in [[over-reliance]] research. Generative AI used as a capable servant has greatly accelerated intellectual work, but it also risks eroding human epistemic agency by encouraging uncritical acceptance of AI-generated reasoning. This creates a need for mechanisms that preserve human agency by augmenting metacognition during AI-assisted intellectual work. To address this, we propose the Synthesis-Analysis Reciprocity Model, which views intellectual construction as a reciprocal interaction between Synthesis, which combines components into an artifact, and Analysis, which critically evaluates them against objective indicators and constrains subsequent synthesis. Grounded in this model, we present the Vibe Compiler, a research-logic compiler that helps researchers transform vague ideas (Vibes) into coherent research logic. Th introduces the Synthesis-Analysis Reciprocity Model and the Vibe Compiler tool to preserve human epistemic agency during GenAI-assisted intellectual work. The model frames intellectual construction as a reciprocal interaction between Synthesis (combining components into a whole) and Analysis (decomposing a whole into components), arguing that over-reliance on AI for synthesis erodes metacognitive capacity. Vibe Compiler implements this model by requiring users to externalise their reasoning structure — specifying components, relationships, and constraints — before AI generates any output. In preliminary trials, the tool improved users' ability to articulate, critique, and refine AI-generated reasoning, suggesting that structured metacognitive scaffolding can mitigate the cognitive offloading risks identified in [[over-reliance]] research. ## Connected Concepts - [[over-reliance]] ## Connected Articles - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Mitigating Latent Knowledge Gaps in Narrative Drafting via Mixed-Initiative Scaffolding - [[genai-performance-vs-learning]] — Distinguishing performance gains from learning when using generative AI - [[ai-making-us-stupid]] — Is AI making us stupid? - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion and the Aware Pass-Through of Non-Understanding in Generative AI Learning - [[learning-by-chatting-genai-impact]] — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning ## Citation Mizoguchi, Riichiro et al. (2026). [Vibe Compiler: A Research-Logic Synthesis Tool That Runs without Prompt Engineering -Toward Enhancing Metacognition for Sustaining Agency in the Age of Generative AI-](https://arxiv.org/abs/2608.05545). arXiv:2608.05545. --- ## [VISMATIC: Secure Containerized Framework for Process-Oriented CS Education Monitoring](https://edtechdev.github.io/aied/articles/vismatic-secure-sandbox-cs-education/) Addresses a critical tension in [[stem-education|CS education]]: the widespread adoption of generative AI makes it impossible to distinguish authentic student effort from AI code synthesis by evaluating final submissions alone. The paper presents **VISMATIC**, a rootless containerized framework that pairs robust environment isolation with explicit user-interaction tracking at the API level. **Key innovation:** VISMATIC captures macro-level behavioral metrics (keystroke patterns, command execution timing, copy-paste behavior) that successfully flag statistical anomalies indicative of automated or off-platform workflows — while preserving student anonymity. Unlike JupyterHub, which exposes the host system to arbitrary shell commands, VISMATIC's rootless containers prevent lateral movement and host compromise. The framework provides a scalable blueprint for [[academic-integrity|preserving pedagogical integrity]] in the AI era, enabling educators to enforce authentic coding discipline through [[formative-assessment|process-oriented monitoring]] rather than final-output evaluation. Open source on GitHub for institutional adoption. ## Connected Concepts - [[stem-education]] - [[academic-integrity]] - [[formative-assessment]] ## Connected Articles - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools for Engineering Education: Insights from Psychological Outcomes - [[vibe-coding-programming-process-visualizer]] — From Idea to Classroom in Days: Using "Vibe Coding" to Create a Programming Process Visualizer from IDE Activity Logs - [[hybrid-e-assessment-semi-automated-grading]] — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations - [[moodle-ai-tutoring-deep-learning]] — From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle - [[tibetcpr-ai-training-feedback]] — TibetCPR: A Multimodal Tactile Feedback System for CPR Training in High-Altitude Regions - [[multi-site-vr-immersive-learning]] — Design and Implementation of a Real-time Multi-site Immersive Learning System Using Photon Fusion ## Citation Arroyo, G., López, L., & Torres, J. C. (2026). Securing the Sandbox: A Rootless Containerized Framework for Process-Oriented Monitoring in Computer Graphics Education. [arXiv:2606.05929](https://arxiv.org/abs/2606.05929). --- ## [Evaluating a Visual Query Tracer and Builder for Learning Declarative Logic Programming](https://edtechdev.github.io/aied/articles/visual-query-tracer-declarative-logic-learning/) Nemo Explain Visualizer (nev) is an interactive visual query tracer and builder for the Datalog reasoner Nemo. Although built for expert users, the authors conducted a qualitative study with 14 participants at varying levels of involvement in a university knowledge-graph course to assess whether such tools help students learn declarative logic programming. Participants rated the tracer positively for building intuition about query evaluation, illustrating how visualisation can act as [[scaffolding]] for otherwise abstract material. The findings sit within [[programming-its]] and [[cs-education]] research on tools that make invisible computation visible, and connect to [[stem-education]] efforts to teach formal methods. They also bear on [[pedagogy-ai-mistakes]] discussions about when automation helps versus hinders conceptual understanding in [[higher-ed]] computing courses. ## Connected Concepts - [[scaffolding]] - [[cs-education]] - [[stem-education]] - [[higher-ed]] ## Connected Articles - [[programming-its]] - [[pedagogy-ai-mistakes]] ## Citation Julián Méndez, Lukas Gerlach, Tobias Wieland, Alex Ivliev, et al. (2026). [Evaluating a Visual Query Tracer and Builder for Learning Declarative Logic Programming](https://arxiv.org/abs/2607.18864). arXiv:2607.18864. arXiv:2607.18864 [cs.HC]. --- ## [What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction](https://edtechdev.github.io/aied/articles/vocabulary-difficulty-prediction/) # What Makes Words Hard? Vocabulary Difficulty Prediction at BEA 2026 🔗 [Code](https://github.com/adno/vocabulary-difficulty) ## Overview This paper presents two complementary approaches to predicting vocabulary difficulty for language learners, achieving state-of-the-art results in the BEA 2026 Shared Task. The work advances both the accuracy and explainability of NLP systems for educational applications. ## Key Findings **Dual-model architecture:** 1. **Black-box LLM** fine-tuned with a soft-target loss function → Pearson r > 0.91, achieving the top shared task result in the open track 2. **Explainable model** → r > 0.77, outperforming fine-tuned encoder baselines while providing interpretable feature-level insights **What makes words hard?** Beyond genuine production difficulty (word frequency, length, concreteness), the analysis reveals that: - **Spelling difficulty** significantly impacts vocabulary item difficulty - **Test item construction** (how items are presented) affects measured difficulty independent of the words themselves - These factors are particularly pronounced in the British Council's Knowledge-based Vocabulary Lists (KVL) ## Educational Applications Vocabulary difficulty prediction directly supports **adaptive language learning systems**, enabling: - Personalized vocabulary sequencing based on predicted difficulty - Diagnostic assessment of learner vocabulary gaps - Generation of appropriately leveled reading materials This connects to [[llm-children-reading-story-generation]], which uses LLMs for generating leveled educational content. The explainable model's insights about item construction also inform the design of [[formative-assessment]] instruments. ## Technical Innovation The **soft-target loss function** for LLM fine-tuning on rating (regression) tasks is notable. Traditional fine-tuning uses hard classification targets; adapting LLMs to predict continuous difficulty scores required a custom loss. This technique may generalize to other educational NLP tasks requiring nuanced ordinal or continuous predictions, relevant to [[self-referential-l2-writing-llm-assessment]] and [[cyberscholar-genai-writing-feedback]]. ## Benchmark Context BEA (Building Educational Applications) is the premier venue for NLP-for-education research. This shared task establishes a new benchmark for vocabulary difficulty prediction with open-source code, encouraging reproducible research in a domain where educational impact depends on reliable, interpretable models. ## Connected Concepts - [[formative-assessment]] ## Connected Articles - [[llm-children-reading-story-generation]] - [[self-referential-l2-writing-llm-assessment]] - [[cyberscholar-genai-writing-feedback]] ## Citation Nohejl, A., Wu, X., Ide, Y., Riera Machin, M. A., Chang, Y.-N., & Yanaka, H. (2026). [What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction.](https://arxiv.org/abs/2605.14257) arXiv:2605.14257. BEA 2026. --- ## [Say What? Examining Text and Voice Input Modalities for Prompt-Based Programming in Computing Education](https://edtechdev.github.io/aied/articles/voice-text-prompt-problems-computing-education/) Nearly all prior research on LLMs in computing education has used text input, yet voice-enabled interfaces are becoming common. This exploratory study investigated how introductory programming students interact with **Prompt Problems** — tasks requiring natural-language prompts to generate correct code — under free choice of text or voice (N = 919). For two of three problems, students who typed were more likely to succeed on the first attempt than those submitting unedited voice prompts; editing transcribed voice prompts before submission erased the gap. Most students tried and preferred text, though some used voice complementarily. Qualitative analysis revealed perceived roles, drawbacks, and advantages of each modality, with implications for multimodal tools and instructional design. - **Modality matters for first-attempt success:** Unedited voice prompts underperformed typed ones on two of three problems; editing transcribed voice closed the gap — extending [[prompt-problems-nl-programming-mistakes]]. - **Student preference for text:** Despite voice availability, most students chose and preferred typing, relevant to [[student-experience]] design. - **Complementary use:** Some students mixed modalities, suggesting non-preferential, context-dependent strategies tied to [[prompt-based-programming-lesson]] practice. - **Instructional design:** Findings inform multimodal [[cs-education]] tooling and [[ai-literacy]] around prompt construction. - **Scales Prompt Problems research:** Large N (919) builds on the [[programming-its]] Prompt Problems line of work. ## Connected Concepts - [[student-experience]] - [[cs-education]] - [[ai-literacy]] ## Connected Articles - [[prompt-problems-nl-programming-mistakes]] - [[prompt-based-programming-lesson]] - [[programming-its]] ## Citation Riegel, K., Hua, Y. C., Denny, P., Pădurean, V.-A., & Leinonen, J. (2026). [Say What? Examining Text and Voice Input Modalities for Prompt-Based Programming in Computing Education](https://arxiv.org/abs/2607.05808). arXiv:2607.05808. --- ## [Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions](https://edtechdev.github.io/aied/articles/white-wu-robotics-ai-education-2026/) > **Synthesis:** White & Wu (2026) critically examine the integration of AI and robotics into education, arguing that while transformative potential exists at all levels, effective integration requires sustained investment, coherent policy, rigorous teacher preparation, and ethical practice. The review finds research remains geographically concentrated, methodologically short-term, and insufficiently attentive to equity. ## Key Findings - **Geographically concentrated research:** Most AI and robotics education studies come from a small number of countries, limiting generalizability. - **Methodologically short-term:** Few longitudinal studies exist, making it difficult to assess lasting impacts on learning-gains and student outcomes. - **Equity gap:** Insufficient attention to [[equity]], human-centred design, and social implications of automation in education. - **Policy gaps:** [[educational-policy-ai|AI education policies]] lag far behind technological deployment. ## Implications This critique of the AIED research landscape connects to broader [[ai-governance-education]] concerns and the [[institutional-change-framework-ai|institutional readiness]] challenges identified in the EPIQ-AI framework. The call for teacher preparation aligns with [[teacher-ai-competency]] and [[faculty-development]] research, while the equity critique echoes findings in [[ai-scoring-language-bias-physics]]. ## Connected Concepts - [[ai-education]] - [[ai-literacy]] - [[ethics]] - [[equity]] - [[teacher-ai-competency]] - [[ai-governance-education]] - [[faculty-development]] - [[learning-analytics]] ## Connected Articles - [[liang-genai-systematic-review-human-ai-2026]] - [[brookings-ai-students-report]] - [[raza-farooq-aied-review-2020-2025]] ## Citation White, A. R., & Wu, Z. (2026). [*Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions*](https://doi.org/10.35542/osf.io/ebyhj_v1). EdArXiv. doi:10.35542/osf.io/ebyhj_v1. --- ## [Will, Skill, Not Tool: Chinese university students' acceptance of generative AI for academic writing in informal English medium instruction settings](https://edtechdev.github.io/aied/articles/will-skill-not-tool-chinese-university-students-acceptance-of-generative-ai-for-/) > **Synthesis:** By adopting the Will, Skill, Tool (WST) model, this study explores how EMI students' intentions to use GenAI for academic writing are shaped by AI-specific variables. Survey data from 512 university students at an EMI university in China found that will-related factors (attitudes, perceived risks, perceived importance of policy) and the skill factor (AI literacy) were significant antecedents of students' intentions, whereas the tool component (facilitating conditions) was not. The model explained 56% of the variance in students' behavioural intentions. ## Key Findings By adopting the Will, Skill, Tool (WST) model, this study explores how EMI students' intentions to use GenAI for academic writing are shaped by AI-specific variables. Survey data from 512 university students at an EMI university in China found that will-related factors (attitudes, perceived risks, perceived importance of policy) and the skill factor (AI literacy) were significant antecedents of students' intentions, whereas the tool component (facilitating conditions) was not. The model explained 56% of the variance in students' behavioural intentions. ## Connected Concepts - [[generative-ai]] - [[ai-literacy]] ## Connected Articles - [[generative-refusal-ai-tools-for-thought]] - [[generative-ai-reduced-study-time-math]] - [[genai-usage-design-students-survey]] - [[genai-linguistic-diversity-academic-writing]] ## Citation## Citation Yang, P., & Huang, W. (2026). [*Will, Skill, Not Tool: Chinese university students' acceptance of generative AI for academic writing in informal English medium instruction settings*](https://doi.org/10.1111/bjet.70075). *British Journal of Educational Technology*. --- ## [Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics](https://edtechdev.github.io/aied/articles/wordstream-glass-learning-analytics/) Revisits WordStream (2009) as a quantitative encoding for qualitative learning analytics; demonstrates how structured coding can surface cohort-level trends while preserving individual narrative context. ## Synthesis Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics investigates revisits wordstream (2009) as a quantitative encoding for qualitative learning analytics; demonstrates how structured coding can surface cohort-level trends while preserving individual narrative context. This work connects to existing research on learning-analytics by demonstrating that Abstract:Data-driven learning analytics can surface trends across a student cohort over time, helping instructors improve the learning environment. WordStream, a visualization idiom for topic evolution, has been instantiated in two platforms toward this goal: the Journal Data Dashboard, for analyzin.... ## Connected Concepts - [[administrator]] - [[affective-computing]] - [[math-education]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] - [[equity-in-ai-education]] - [[teacher-ai-competency]] - [[help-seeking]] ## Connected Articles - [[savvy-student-attention-video-learning]] — SAVVY: Student Attention Visualization for Video-based Learning Analysis - [[physiological-signals-exam-outcomes-ml]] — Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning - [[hypergamification-game-engine-lms]] — Hypergamigication Through Integrating Game Engines and Learning Management Systems: Ender's Game - [[cross-dataset-bloom-question-classification]] — Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs - [[a4l-analytics-pipeline]] — Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System for Middle School: A Longitudinal Study of Learning Outcomes ## Citation Huyen N. Nguyen, Kathleen Bowe, Minh-Huyen Nguyen, Kit Thompson, Caleb M. Trujillo (2026). [Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics](https://arxiv.org/abs/2606.18692). arXiv:2606.18692. --- ## [Explainable Artificial Intelligence in Education (XAI-ED)](https://edtechdev.github.io/aied/articles/xai-education-framework/) 📄 DOI: 10.1016/j.caeai.2022.100074 ## Core Contribution This paper introduces **XAI-ED**, a framework for explainable AI that is purpose-built for education. It argues that while XAI in education shares common ground with broader XAI (accountability, fairness, privacy), education has *distinctive needs*: learning data is noisy at many levels, explanations can directly support metacognition and self-regulated learning, and different stakeholders need fundamentally different kinds of explanations. ## The XAI-ED Framework — Six Aspects ### 1. Stakeholders XAI in education must serve multiple audiences — learners, teachers, administrators, parents, researchers, developers, and policymakers — each with different explainability requirements. A learner needs to understand *why* a hint was given; a teacher needs to understand *which* students are at risk and *why*; a developer needs to debug model behavior. This connects to [[teacher-role]] and [[student-experience]] research on multi-stakeholder design. ### 2. Benefits Explanations serve multiple purposes in educational contexts: - **Trust** — building confidence in AI-driven recommendations and assessments - **Fairness** — detecting and mitigating algorithmic [[bias-mitigation]] - **Debugging** — helping developers and researchers identify model flaws - **Usability** — making AI interfaces more transparent and actionable - **Learning** — explanations as *pedagogical tools* that support [[metacognition]] - **Regulatory compliance** — GDPR right to explanation and similar mandates ([[privacy]], [[regulation]]) - **Adoption** — reducing resistance to AI tools among educators ### 3. Approaches for Presenting Explanations The paper catalogs multiple explanation modalities: visual (heatmaps, decision trees), textual (natural language), example-based (counterfactuals, nearest neighbors), feature importance rankings, rule extraction, and model simplification. The key insight is that the optimal approach depends on the stakeholder and the pedagogical context — an insight directly relevant to [[scaffolding]] design and [[intelligent-tutoring]] interface research. ### 4. Classes of AI Models The framework maps explanation approaches to model types: - **White-box** (decision trees, linear models, rule-based systems) — inherently interpretable - **Black-box** (neural networks, ensemble methods) — require post-hoc explanation methods - **Glass-box** — newer approaches that balance accuracy with transparency This taxonomy connects to the [[adaptive-learning]] literature's ongoing tension between model complexity and interpretability. ### 5. Human-Centred Design Explanations are not purely technical artifacts — they are communication acts. The interface must be designed for the specific stakeholder's cognitive needs, not just optimized for technical accuracy. This aligns with [[ai-literacy]] research showing that technical transparency without pedagogical framing often fails to support actual understanding. ### 6. Potential Pitfalls The framework identifies risks specific to educational XAI: - **Explanation overload** — too much information overwhelms users, undermining the benefit - **Misleading explanations** — post-hoc explanations may not reflect actual model reasoning - **Confirmation bias** — users may selectively attend to explanations that confirm existing beliefs - **Over-trust** — explanations can create false confidence in flawed systems, connecting to [[over-reliance]] - **Gaming the system** — students may exploit explanations to circumvent learning, a known risk in [[intelligent-tutoring]] ## Four Case Studies ### OnTask — Instructor-Facing Learning Analytics A learning analytics platform that provides instructors with interpretable student risk indicators. XAI-ED applied to surface the rules and features driving each risk flag, enabling instructors to make informed interventions. Demonstrates the [[teacher-role]] shift toward data-informed decision-making. ### OATutor — Adaptive Tutoring System An open-source ITS where XAI explains mastery predictions and hint selections to both students and instructors. Connects to [[intelligent-tutoring]] and [[adaptive-learning]] research on transparency in automated instruction. ### SRES — Student-Facing Recommender A learning resource recommender that explains *why* specific resources are suggested based on the learner's knowledge state and goals. Supports [[self-regulated-learning]] by making the recommendation logic visible and actionable. ### CLA (Connected Learning Analytics) — Multimodal Collaboration Analytics for collaborative learning environments that surface group dynamics and individual contributions. XAI helps students and instructors understand team interaction patterns — connecting to [[collaborative-learning]] and [[learning-analytics]] research. ## Significance for AIED The XAI-ED framework has become a foundational reference in AIED, cited across the [[stanford-evidence-base-ai-k12-2026]], [[ai-tutor-safety-harms]], and [[ai-literacy]] literatures. It bridges the gap between technical XAI research (which often ignores pedagogical context) and educational practice (which often treats AI as a black box). The framework's emphasis on *distinctive educational needs* anticipates later work on [[pedagogical-safety]] and [[human-in-the-loop-ai]] design. ## Connected Concepts - [[teacher-role]] - [[student-experience]] - [[bias-mitigation]] - [[metacognition]] - [[privacy]] - [[regulation]] - [[scaffolding]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[ai-literacy]] - [[over-reliance]] - [[self-regulated-learning]] - [[collaborative-learning]] - [[learning-analytics]] - [[pedagogical-safety]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[ai-tutor-safety-harms]] ## Citation Khosravi, H., Buckingham Shum, S., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable Artificial Intelligence in education. [*Computers and Education: Artificial Intelligence, 3*](https://doi.org/10.1016/j.caeai.2022.100074), 100074 --- ## [HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis](https://edtechdev.github.io/aied/articles/xie-hillm-cd-2026/) > **Synthesis:** Xie, Yang, Zhang, Li, Wang, Yang & Gao (2026) propose HiLLM-CD, a tree-structured framework for cognitive diagnosis that represents student proficiency as node-wise values on a concept tree, enabling coarse-to-fine diagnosis. A multi-agent LLM pipeline eliminates the need for expert annotations by automatically generating concept trees and exercise-concept links from educational text. Evaluated on MOOCCubeX across four subjects, HiLLM-CD improves response prediction by 11.1% AUC and proficiency reliability by 0.597 Kendall's τ under fully unannotated settings. ## Key Contributions 1. **Hierarchical proficiency modeling:** Represents each student's knowledge as node-wise proficiencies on a concept tree, with a tree-structured encoder-decoder where the encoder infers proficiencies from response history and the decoder predicts responses via a GCN-based cognitive decoder. This enables intuitive coarse-to-fine diagnosis — first assessing broad competence, then drilling into specific concept weaknesses. 2. **Automated concept tree construction:** A multi-agent LLM pipeline (Concept Elaborator, Concept Tagger, Coarse Concept Generator) automatically builds concept trees and exercise-concept links from educational text, eliminating costly expert annotation. This makes the entire diagnosis pipeline fully automated. 3. **LLM-guided proficiency learning:** An LLM teacher provides order-based distillation — supervising pairwise proficiency ordering rather than absolute values — yielding more reliable and semantically coherent proficiency estimates than reconstruction-only training. ## Approach HiLLM-CD departs from traditional [[cognitive-diagnosis]] methods that treat concepts as independent skills. Instead, it models proficiency on a **concept tree** where leaves represent atomic concepts and internal nodes summarize coarser concept groups. This aligns with how educators naturally think about student competence — starting broad and narrowing to specific weaknesses. The framework runs in three stages: - **Annotation:** LLM agents tag exercises with concepts and cluster concepts hierarchically - **Encoding:** An attention-based encoder aggregates response history to predict node-wise proficiencies - **Decoding:** A GCN-based decoder propagates proficiency information through the concept tree to predict exercise responses ## Results On MOOCCubeX across Computer Science, Mathematics, Physics, and Biology: - **+11.1% AUC** average improvement in response prediction - **+0.597 Kendall's τ** improvement in proficiency alignment with LLM teacher reference - **+11.4% AUC** on computerized adaptive testing (CAT) - Ablation confirms both the concept tree structure and order-based distillation are essential ## Implications for AI in Education HiLLM-CD advances [[knowledge-tracing]] and [[student-modeling]] by making cognitive diagnosis fully automated and hierarchically interpretable. The concept tree approach connects to [[adaptive-learning]] systems that need structured student models for personalized content sequencing. By eliminating expert annotation requirements, it enables scalable deployment of [[cognitive-diagnosis]] to new domains. The LLM teacher approach for proficiency supervision suggests a broader pattern where LLMs guide learning of interpretable student representations, relevant to [[personalized-learning]] and [[intelligent-tutoring]]. ## Connected Concepts - [[knowledge-tracing]] - [[student-modeling]] - [[cognitive-diagnosis]] - [[adaptive-learning]] - [[personalized-learning]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[generative-ai]] - [[llm]] ## Connected Articles - [[huang-interpretable-knowledge-tracing-2026]] — Interpretable Knowledge Tracing - [[cold-start-knowledge-tracing-safeinsights]] — Cold-Start Knowledge Tracing - [[deeptutor]] — DeepTutor: Towards Agentic Personalized Tutoring - [[zerkouk-comprehensive-review-its-2025]] — Comprehensive ITS Review ## Citation Xie, Y., Yang, W., Zhang, B., Li, Z., Wang, L., Yang, M., & Gao, Y. (2026). [*HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis*](https://doi.org/10.1145/3770855.3817769). In *Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '26)*. ACM. doi:10.1145/3770855.3817769. --- ## [How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata](https://edtechdev.github.io/aied/articles/youtube-frames-chatgpt-education/) Uses epistemic network analysis of multimodal YouTube metadata (transcripts, titles, thumbnails, comments) to show how different creator groups frame ChatGPT use in education, revealing divergent narratives around learning support versus academic-integrity risk. The work connects to broader debates about how [[generative-ai]] systems reshape [[student-experience]] and the conditions under which AI support [[scaffolding|scaffolds]] rather than undermines learning. It has direct implications for [[pedagogy-ai-mistakes]] and the risk of [[over-reliance]] when assistants absorb too much of the cognitive load. Findings also bear on [[ai-literacy]] and [[self-regulated-learning]], and on how institutions should govern [[student-experience]] and [[academic-integrity]]. Practitioners in [[higher-ed]] and [[teacher-role|teachers]] can use the evidence to calibrate when to deploy [[llm]]-based help and how to pair it with [[feedback-loop|feedback]] that preserves learning gains. ## Connected Concepts - [[generative-ai]] - [[student-experience]] - [[scaffolding]] - [[over-reliance]] - [[ai-literacy]] - [[self-regulated-learning]] - [[academic-integrity]] - [[higher-ed]] - [[teacher-role]] - [[llm]] - [[feedback-loop]] ## Connected Articles - [[pedagogy-ai-mistakes]] ## Citation L. Xiao, G. Chen, Y. Zhang et al. (2026). [How YouTube Frames ChatGPT Use in Education: An Epistemic Network Analysis with Supporting Multimodal Metadata](https://arxiv.org/abs/2607.08698). arXiv:2607.08698. --- ## [Comprehensive Review of Intelligent Tutoring Systems](https://edtechdev.github.io/aied/articles/zerkouk-comprehensive-review-its-2025/) > **Comprehensive Review of Intelligent Tutoring Systems** — Journal of Computers in Education (2025). A systematic literature review covering 2010–2025 that analyzes the deployment and effectiveness of Intelligent Tutoring Systems (ITS) in real educational settings. The review examines the full landscape of ITS research — pedagogical strategies, natural language processing, adaptive learning mechanisms, student modeling approaches, and domain-specific applications — and arrives at a nuanced verdict: the evidence for ITS effectiveness is mixed, revealing a complex landscape of genuine advancements alongside persistent challenges in scientific rigor and real-world impact. ## Key Findings This review provides the most comprehensive mapping of the [[intelligent-tutoring]] field since the emergence of LLM-based tutoring approaches transformed the landscape. Spanning a 15-year window (2010–2025), it captures both the pre-LLM era of traditional ITS and the post-LLM era that has fundamentally reshaped what is technically possible. **Mixed effectiveness evidence.** The review's central finding is that ITS effectiveness is neither uniformly positive nor categorically negative. Some systems demonstrate substantial learning-gains, particularly in well-structured domains like mathematics and programming where [[student-modeling]] and [[knowledge-tracing]] techniques are most mature. Other deployments show negligible or context-dependent effects. This mixed picture challenges both the optimistic narrative that AI tutoring is a proven solution and the pessimistic narrative that it is ineffective. Instead, it calls for more nuanced questions: *which* systems, for *which* learners, in *which* contexts, produce *which* outcomes? This aligns with the [[ai-tutor-effectiveness-review]] literature's emphasis on contextual factors. **Pedagogical strategies.** The review catalogs the range of pedagogical approaches embedded in ITS, from [[socratic-method]] and [[scaffolding]] to [[adaptive-learning]] and [[adaptive-learning]] pathways. A key finding is that many ITS implementations lack explicit pedagogical grounding — the tutoring behavior is often driven by technical capabilities (what the system can do) rather than pedagogical principles (what the system should do). This echoes concerns in the [[pedagogical-llm-training]] literature about the gap between technical sophistication and pedagogical intentionality. **NLP and adaptive mechanisms.** The integration of [[educational-nlp]] techniques — including [[automated-question-generation]], short-answer assessment, and dialogue management — has advanced substantially over the review period. However, the review notes that many NLP components are evaluated in isolation rather than as integrated parts of tutoring systems that actually interact with learners. Similarly, [[adaptive-learning]] show promise but often rely on narrow student models that fail to capture the full complexity of learner cognition and affect — a gap that the [[affective-tutoring]] and [[multimodal-ai-tutoring]] communities are beginning to address. **Student modeling challenges.** [[student-modeling]] remains both the foundation and the bottleneck for ITS. While [[knowledge-tracing]] techniques (including Bayesian approaches like [[stanbkt-bayesian-knowledge-tracing]] and deep learning variants) have improved, the review identifies persistent gaps in modeling higher-order cognitive processes, metacognition, and motivational states. This connects to the [[engagement-intensity-learner-modeling]] and [[metacognition]] literatures. **Scientific rigor deficit.** One of the review's most important contributions is its methodological critique. Many ITS studies suffer from weak experimental designs — small sample sizes, absence of control groups, short intervention durations, and inadequate statistical analyses. The authors call for greater scientific rigor, including [[rct]] where feasible, pre-registration of study designs, and transparent reporting aligned with educational research standards. This methodological critique connects to broader concerns in [[ai-ed-evaluation]] about the quality of evidence in AI education research. ## Implications For researchers, this review provides both a taxonomy of the field and a methodological call to action. The mixed effectiveness findings underscore the need for [[rct]]-quality studies that can disentangle the effects of specific ITS features from confounds like novelty effects, instructor quality, and student self-selection. The review's emphasis on real educational settings — as opposed to laboratory studies — pushes the field toward ecologically valid research designs that mirror the [[ai-education]] challenges faced by practitioners. For system designers, the review's catalog of pedagogical gaps suggests concrete priorities: ITS should be built on explicit pedagogical frameworks rather than retrofitted onto technical architectures; domain-modeling should incorporate richer representations of learner cognition and affect; and [[learning-analytics]] should be embedded from the start to enable continuous improvement and evidence generation. The review's identification of persistent student modeling limitations connects directly to emerging work on [[llm-student-modeling-memory]] and [[history-aware-student-simulation]], which offer potential pathways toward more sophisticated learner representations. Similarly, the call for [[adaptive-learning]] that goes beyond simple difficulty adjustment toward truly personalized pedagogical experiences connects to the [[learnmate2-llm-adaptive-learning]] and [[nie-personavlm-long-term-personalization-2026]] research streams. For the [[edtech-platform]] industry and [[educational-policy-ai]] community, this review serves as a sobering reminder that technical capability does not guarantee educational effectiveness. The mixed evidence base should temper claims of AI tutoring as a silver bullet while also validating continued investment in rigorous ITS research and development. The path forward requires [[ai-ed-evaluation]] frameworks that match the sophistication of the systems being evaluated. ## Connected Concepts - [[adaptive-learning]] - [[adaptive-learning]] - [[affective-tutoring]] - [[ai-ed-evaluation]] - [[ai-education]] - [[automated-question-generation]] - [[educational-nlp]] - [[educational-policy-ai]] - [[knowledge-tracing]] - [[learning-analytics]] - [[adaptive-learning]] - [[metacognition]] - [[pedagogical-llm-training]] - [[rct]] - [[scaffolding]] - [[socratic-method]] ## Connected Articles - [[ai-tutor-effectiveness-review]] — AI Tutor Effectiveness Review - [[engagement-intensity-learner-modeling]] — Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction - [[history-aware-student-simulation]] — Who Am I? History-Aware Profiles for Student Simulation in Tutoring Dialogues - [[learnmate2-llm-adaptive-learning]] — LearnMate^2: Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning - [[llm-student-modeling-memory]] — LLM Student Modeling and Long-Term Memory Architecture - [[multimodal-ai-tutoring]] — Multimodal AI Tutoring in STEM - [[nie-personavlm-long-term-personalization-2026]] — PersonaVLM: Long-Term Personalization for AI Tutors - [[stanbkt-bayesian-knowledge-tracing]] — StanBKT: Rethinking Parameter Estimation in Bayesian Knowledge Tracing - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[adapt-adaptive-lesson-plan-transformer]] — AdaPT: Adaptive Lesson Plan Transformer for Cross-Regional and Differentiated Instruction - [[adaptive-pretesting-retention]] — Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a Retention Study - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[agentic-ai-pedagogical-best-practice-2026]] — Agentic AI and Pedagogical Best Practice: The Tension Between Automation and Learning - [[agentic-education-coding]] — Agentic Education with AI Coding Assistants - [[agentic-workflows-education]] — Agentic Workflows in Education - [[ai-adult-learning-design]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[ai-assessment-human-tutors]] — AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice - [[ai-coaching-rl-skill-development]] — AI Coaching for Accelerating Human Skill Development with Reinforcement Learning - [[ai-enabled-serious-games]] — AI-Enabled Serious Games: Integrating Intelligence and Adaptivity in Training Systems - [[ai-fallibility-warning-help-seeking]] — Warning About AI Fallibility Increases Help-Seeking in an Intelligent Tutoring System - [[ai-generated-traces-novice-programmers]] — AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study - [[ai-interlocutor-l2-spoken-dialogue]] — What Changes When the Interlocutor Is an AI? Interactional Fluency and Linguistic Uptake in L2 Spoken Dialogue - [[stanford-evidence-base-ai-k12-2026]] — AI in K-12 Evidence Base - [[ai-learning-companions-framework]] — Building AI Companions that Prioritise Learning over Performance ## Citation Zerkouk, M., Mihoubi, M., & Chikhaoui, B. (2025). [Comprehensive Review of Intelligent Tutoring Systems](https://arxiv.org/abs/2507.18882v1). *Journal of Computers in Education*. --- ## [When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle](https://edtechdev.github.io/aied/articles/zhang-tutormoments-2026/) > **Synthesis:** Zhang et al. (2026) introduce TutorMoments, a replay-based evaluation framework that tests whether LM tutors adapt their pedagogical actions to context — scaffolding when support is needed, pushing for rigor when students are ready, and avoiding over-scaffolding. Evaluating 462 teacher-annotated transcripts from grades 2-7 math tutoring, they find frontier models default toward over-helpfulness at the expense of productive struggle. The paper argues that AI optimized for helpfulness may be misaligned with the pedagogical goal of providing the right help at the right moment. ## Summary TutorMoments evaluates whether LM tutors select instructional actions appropriate to the pedagogical demands of specific learning moments. Expert math teachers annotate key decision points in authentic tutoring transcripts: scaffolding-appropriate moments (student needs support) and rigor-appropriate moments (student is ready for challenge). The framework then replays these moments to test whether LMs select appropriate tutor moves. Findings show minimally prompted LMs frequently over-scaffold and rarely push for rigor; evaluation-aware prompting improves adaptability but behavior concentrates into a narrow set of tutor moves. The dataset includes 462 transcripts from 198 students in grades 2-7 interacting with 173 human tutors, with 1,500+ teacher-annotated key moments. ## Key Findings - Minimally prompted frontier LMs default toward over-scaffolding — being "helpful" rather than pedagogically adaptive - LMs rarely push for rigor even when students are ready for challenge - Evaluation-aware prompts substantially improve adaptability but reveal large cross-model differences - Tutor behavior concentrates into a narrow set of moves, lacking the diversity of human tutor strategies - Framework operationalizes productive struggle, zone of proximal development, and desirable difficulties as evaluation criteria ## Connected Concepts - [[math-education]] - [[k-12]] - [[scaffolding]] - [[teacher-role]] - [[ai-tutoring]] - [[llm]] - [[over-reliance]] - [[zone-of-proximal-development]] ## Connected Articles - [[lak2026-hint-button-unproductive-use]] - [[kar-mathbuddy-affective-math-tutoring-2025]] ## Citation Zhang, A., Ross, A., Patel, K., Bernado, J., Bowie, R., Ribeiro, A. T., Halper, D., Valayaputtur, H., Andreas, J., Loeb, S., Lucy, L., Lo, K., & Knight, R. (2026). [When Help is Unhelpful: Evaluating AI Tutors for Productive Struggle](https://example.com/tutormoments-preview.pdf). --- # Concepts ## [Academic Integrity](https://edtechdev.github.io/aied/concepts/academic-integrity/) > **Academic integrity** — the ethical framework governing honest academic work in the age of AI. The wiki documents how the concept has been reframed by generative AI: from a problem of detecting dishonest output to a design problem of making honest work visible, verifiable, and worth producing. Academic integrity research in this space has evolved from detection-focused approaches toward fundamental assessment redesign, pedagogy-led governance, and teaching students *how* to use AI well rather than merely policing whether they do. The arrival of generative AI has not created the need for academic integrity — it has made weaknesses in existing approaches harder to ignore. A polished, plausible product can now be generated in seconds, so **product resemblance is an increasingly unreliable signal of capability**. This shifts the integrity question from *"can we catch AI use?"* to *"can our assessments still warrant the inferences we draw about student learning?"* ### The evolution from detection to redesign - **Detection skepticism:** [[plagiarism-detection]] research and institutional analyses increasingly find that AI detection tools are unreliable and procedurally unfair. [[detecting-llm-generated-text-latent-prompt|LLM text detection]] faces fundamental limitations. Fully AI-generated submissions can pass through live examination systems largely undetected, and experienced markers do not reliably spot GenAI-authored work. Detection, at best, is a limited, situational tool — not a strategy of first resort. - **Assessment redesign:** [[authentic-assessment]], [[beyond-detection-authentic-assessment-ai-2025|beyond-detection approaches]], and [[ai-assessment-scale-reform|the AI Assessment Scale]] shift the focus from catching AI use to designing assessments where AI use is either irrelevant, transparent, or required to demonstrate a specific capability. - **Validity as the organizing frame:** [[assessment-validity]] reframes integrity as an evidential problem. [[authentic-products-authenticated-processes-2026|Authentic assessment research]] introduces **construct substitution** — an AI-generated product is attributed to the student, so the assessment infers the tool's capability rather than the student's. The evidential question survives any AI policy: whether use is prohibited, permitted, or required, the assessment must still generate evidence warranting the inference being drawn. - **Policy development:** [[genai-policies-higher-ed-computing|Institutional AI policies]] and [[educational-policy-ai]] research examine how universities develop and communicate integrity expectations — and why abstract policy statements so often fail. ### The rationalization problem Students do not generally misuse AI out of malice; they rationalize it. [[student-rationalization-ai-writing|Interview research]] identifies at least **five disconnect sites** where students' interpretation of AI policy diverges from faculty intent, and a taxonomy of **20+ distinct rationalizations** — from "copying AI text is victimless" to "text reflecting my beliefs is my own writing." These rationalizations are ad hoc, post hoc, and internally inconsistent, and they describe a "steep, ethical slippery slope" on which students slide far outside pedagogical goals. This is why [[student-misconceptions-ai|student misconceptions]] about AI are the upstream cause of integrity violations, and why integrity education must address ethical reasoning, not just technical skill. ### Why policy alone fails: the coordination problem [[ethical-ai-higher-ed-game-theory|A coordination-game framework]] provides a mechanism-level account of why policy pronouncements rarely change behavior: students' AI use is a **coordination problem**, where individual choices depend on peer expectations and assessment design. The model's key finding is **non-linear threshold dynamics** — small, well-calibrated changes to reflective-assessment incentives can trigger rapid cohort-wide shifts toward responsible use, while weak or misaligned incentives let opportunistic practice persist. In practical terms, modest redesign (e.g., requiring students to reflect on their AI interactions) can have disproportionate effects where abstract rules have none. ### The socio-emotional dimension Integrity enforcement has a neglected emotional cost. [[shame-guilt-ai-regulation-computing-education|Shame-and-guilt research]] with students shows these emotions regulate when and how AI use becomes visible, producing **hiding behaviors and selective disclosure** — and that they coexist with continued use, creating cycles of reduced agency and moral tension rather than behavior change. Students even describe their AI use in language of addiction. The implication: detection-heavy, surveillance-oriented policy risks **driving misuse underground** rather than addressing it, undermining the candid negotiation that productive use requires. ### Cultural and contextual variation Policy text does not equal policy perception. [[cross-cultural-student-perceptions-genai-computing|Cross-national research]] found that, despite functionally identical institutional policies, students at different universities rated the same AI-assisted practices differently — **culture, not policy wording, drove perceived wrongness**. Policy harmonization does not produce perception harmonization, so culturally diverse cohorts interpret the same rules differently, an equity concern for enforcement and grading that argues for scenario-based clarification over abstract rule statements. ### From policing to pedagogy The wiki documents a paradigm shift: from AI as an integrity threat to be policed, to AI as a tool whose appropriate use must be taught. This is the ethical dimension of [[ai-literacy]] and is embodied in practical design: - **Task-specific AI-use declarations:** [[genai-declaration-frameworks-higher-education|Domain-specific declaration frameworks]] replace generic "I used AI" checkboxes with structured declarations mapping use to cognitive stages (e.g., structural planning vs. content generation), forcing reflection and shifting focus from policing to professional practice. - **Process-transparent assessment:** architectures such as [[credential-cognitive-stewardship-ai-assessment|cognitive stewardship]], staged submissions, oral defences, and the [[coauthorship-integrity-reconceptualising-assessment-validity-for-the-age-of-gene|AI Viva]] (a conversational agent probing whether students understand their submissions) make human judgement, verification, and responsibility visible. - **Reducing misuse:** integrity sits alongside [[ai-misuse-learning-harm]] (the learning cost of misuse) and [[reducing-ai-misuse]] (the interventions that prevent it), tying honesty to genuine learning rather than rule-following. ### Connections Academic integrity connects to [[assessment-validity]], [[ai-literacy]], [[plagiarism-detection]], [[authentic-assessment]], [[assessment]], [[educational-policy-ai]], [[regulation]], [[ethics]], and [[equity]]. It is the ethical dimension of AI in education, inseparable from [[over-reliance]] and the broader question of how [[generative-ai]] reshapes [[higher-ed]] and [[k-12]] learning. ## Connected Concepts - [[assessment-validity]] - [[ai-literacy]] - [[plagiarism-detection]] - [[authentic-assessment]] - [[assessment]] - [[educational-policy-ai]] - [[regulation]] - [[ethics]] - [[equity]] - [[over-reliance]] - [[ai-misuse-learning-harm]] - [[reducing-ai-misuse]] - [[student-misconceptions-ai]] - [[generative-ai]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[student-rationalization-ai-writing]] — It's OK Because… Student Rationalization of AI Use - [[authentic-products-authenticated-processes-2026]] — From Authentic Products to Authenticated Processes - [[ethical-ai-higher-ed-game-theory]] — Coordination Game Framework for Ethical AI Use - [[shame-guilt-ai-regulation-computing-education]] — Shame and Guilt as Social Regulators of AI Use - [[cross-cultural-student-perceptions-genai-computing]] — Did Alice Do Wrong? Cross-Cultural Perceptions of AI Use - [[genai-declaration-frameworks-higher-education]] — Structuring Transparency: GenAI Declaration Frameworks - [[coauthorship-integrity-reconceptualising-assessment-validity-for-the-age-of-gene]] — Coauthorship Integrity and the AI Viva - [[credential-cognitive-stewardship-ai-assessment]] — Cognitive Stewardship for AI-Mediated Assessment - [[ai-assessment-scale-reform]] — The AI Assessment Scale and Assessment Reform - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: Authentic Assessment in an AI-Mediated World - [[genai-policies-higher-ed-computing]] — Institutional AI Policies in Computing Education - [[detecting-llm-generated-text-latent-prompt]] — Detecting LLM-Generated Text - [[genai-student-experiences-uk-he-survey-2026]] — GenAI Student Experiences: UK HE Survey - [[metacognitively-discordant-completion-genai-2026]] — Metacognitively Discordant Completion - [[pchl-he-framework-genai-content-creation-2026]] — PCHL Framework for GenAI Content Creation - [[genai-higher-education-systematic-review-2026]] — GenAI in Higher Education: Systematic Review - [[competency-based-education-genai-production-2026]] — Competency-Based Education and GenAI Production --- ## [Accessible Learning](https://edtechdev.github.io/aied/concepts/accessible-learning/) > **Accessible Learning** — the design and delivery of educational experiences that accommodate diverse learner needs, spanning physical, cognitive, sensory, and situational differences. In AI in education, accessible learning research examines both how AI tools can remove barriers for disabled and neurodivergent learners and how AI systems themselves must be designed to avoid creating new accessibility gaps. Accessible learning sits at the intersection of [[equity]], [[instructional-design]], and [[special-education]]. Unlike narrow accommodations that retrofit access onto existing systems, the accessible learning perspective — grounded in Universal Design for Learning — argues that environments should be designed for the full range of human diversity from the start. The articles in this wiki explore how AI can enable this through automated content transformation, adaptive assessment interfaces, and tools designed with neurodivergent users' lived experience as the starting point. ### Key research themes **AI-powered content accessibility** demonstrates how automated pipelines can reduce barriers. **[[adhd-video-segmentation-computing-education|Pimenova et al.]]** showed that AI-segmented instructional videos with fixed pauses eliminated the performance gap between ADHD and non-ADHD learners — strong evidence for Universal Design for Learning via automated content transformation. The study connects to [[neurodivergent-computing-students]] research on how collaborative learning structures affect neurodivergent comfort. **Inclusive assessment design** grapples with the tension between security and accessibility. **[[behaviorally-adaptive-visual-diversion-assessment-2026|BAVD]]** proposes a theoretical framework for adaptive visual diversion that resists screen-capture cheating while accommodating learners with visual-processing needs — explicitly modeling the trade-off between anti-cheating measures and [[accessible-learning]] principles. This connects to broader [[academic-integrity]] and [[assessment]] concerns. **Neurodivergent learner experiences** center the voices of disabled and neurodivergent students. **[[neurodivergent-computing-students|Zastudil et al.]]** found that neurodivergent computing students need structured assignments, small consistent teams, and explicit role definitions — preferences that AI tutoring and collaboration tools must accommodate. **[[dyslexlens-dyslexic-learners-ai|DysLexLens]]** analyzed dyslexic learners' forum discussions, revealing that while they value AI for literacy support, they face significant accessibility barriers from inconsistent output quality and lack of equitable accommodations. Both connect to [[special-education]] and [[student-experience]]. **Disability-centered AI critique** examines how AI systems can marginalize rather than include. **[[genai-minoritized-knowledges-disability|Tali-Otmani]]** argues that generative AI systems in higher education actively marginalize disability-centered ways of knowing due to Anglophone, Western-centric training data — connecting to [[equity-in-ai-education]] concerns about epistemic justice. **Accessible tools in practice** shows how AI can expand participation. **[[suacode-african-students-motivations|SuaCode]]** demonstrated that smartphone-based coding courses reach students in low-resource African contexts where fewer than 1% have coding skills. **[[embodied-string-learning-blindness-low-vision-musicians|Pimenova et al.]]** worked with blind and low-vision musicians to develop non-visual learning strategies, centering disability-led embodied design. **[[ludia-udl-ai-thought-partner-2026|LUDIA]]** provides a no-cost, private, multilingual AI thought partner connecting educators with UDL principles. **[[special-r1-rl-special-education|Special-R1]]** extends reinforcement learning to model cognitive and communicative diversity across disability profiles. ### Connections to related concepts Accessible learning is deeply connected to [[equity]] — accessibility is not merely a technical concern but a question of who gets to participate in learning. It connects to [[universal-design-for-learning]] as its theoretical foundation, to [[special-education]] for disability-specific approaches, to [[instructional-design]] for how courses and tools are structured, and to [[neurodiversity]] as the lens that reframes difference as diversity rather than deficit. The [[ai-education]] and [[generative-ai]] connections highlight both the promise (automated content adaptation) and peril (AI systems that reproduce exclusion). ## Connected Concepts - [[equity]] - [[equity-in-ai-education]] - [[special-education]] - [[instructional-design]] - [[universal-design-for-learning]] - [[neurodiversity]] - [[student-experience]] - [[ai-literacy]] - [[higher-ed]] - [[k-12]] - [[cs-education]] - [[assessment]] - [[academic-integrity]] - [[privacy]] - [[generative-ai]] - [[ai-education]] ## Connected Articles - [[adhd-video-segmentation-computing-education]] - [[behaviorally-adaptive-visual-diversion-assessment-2026]] - [[dyslexlens-dyslexic-learners-ai]] - [[neurodivergent-computing-students]] - [[genai-minoritized-knowledges-disability]] - [[embodied-string-learning-blindness-low-vision-musicians]] - [[suacode-african-students-motivations]] - [[ludia-udl-ai-thought-partner-2026]] - [[special-r1-rl-special-education]] --- ## [Active Learning](https://edtechdev.github.io/aied/concepts/active-learning/) > **Active Learning** — instructional approaches that engage students in doing things and thinking about what they are doing, rather than passively receiving information. In AI in education, active learning research examines both how AI tools can support active learning pedagogies and how active engagement with AI tools — rather than passive consumption — affects learning outcomes. Active learning is a foundational principle in education research, grounded in constructivist theories that position learners as active constructors of knowledge. In the context of AI in education, the concept takes on dual significance: AI tools can enable active learning at scale (through interactive tutoring, simulations, and adaptive feedback), but poorly designed AI tools can also undermine it by doing the cognitive work for students. The tension between AI assistance and active cognitive engagement — explored in articles like [[lak2026-hint-button-unproductive-use]] on premature hint use and [[efficiency-gain-illusion-ai-overreliance]] on over-reliance — is a central concern. AI-enabled active learning manifests across multiple forms in this wiki: [[intelligent-tutoring]] systems that engage students in problem-solving rather than answer-giving, [[genai-mindtool-generative-learning]] approaches where students use AI as a thinking tool rather than a substitute, [[test-driven-ai-assisted-learning]] where students drive AI interaction rather than follow it, and [[curiobot-llm-tutoring-exploratory-learning]] exploratory learning environments. The [[scaffolding]] concept is tightly coupled — effective active learning requires calibrated support that fades as competence grows, which AI tutors must learn to provide. ## Connected Concepts - [[scaffolding]] - [[constructivist]] - [[instructional-design]] - [[intelligent-tutoring]] - [[student-experience]] - [[higher-ed]] - [[k-12]] - [[stem-education]] - [[generative-ai]] - [[feedback-loop]] - [[over-reliance]] - [[collaborative-learning]] ## Connected Articles - [[lak2026-hint-button-unproductive-use]] - [[efficiency-gain-illusion-ai-overreliance]] - [[neurodivergent-computing-students]] - [[genai-mindtool-generative-learning]] - [[test-driven-ai-assisted-learning]] - [[curiobot-llm-tutoring-exploratory-learning]] --- ## [Adaptive Learning](https://edtechdev.github.io/aied/concepts/adaptive-learning/) > **Adaptive learning** — AI-driven educational systems that adjust content, pacing, and instructional strategies based on individual learner characteristics and performance. Adaptive learning is the operational goal of much AI in education research: using student models to personalize instruction. ### Core mechanisms - **Measure-model-adapt loop:** [[knowledge-tracing]] estimates what the student knows, [[student-modeling]] represents the learner, and the system adapts difficulty, content, and feedback accordingly. - **Personalization at scale:** [[personalized-learning]] systems use adaptive algorithms to serve unique learning paths for each student. [[deeptutor|DeepTutor]] and [[ai-powered-personalized-learning-elementary-fractions-2026|elementary fraction tutors]] demonstrate adaptive personalization in practice. - **Content sequencing:** [[adaptive-pretesting-retention|Adaptive pretesting]] and [[adapt-adaptive-lesson-plan-transformer|lesson plan transformers]] optimize the order and type of content presented. - **ITS integration:** [[intelligent-tutoring|Intelligent tutoring systems]] are the canonical adaptive learning platform, combining diagnosis with adaptation. ### Effectiveness evidence The wiki documents mixed evidence: adaptive systems improve outcomes when adaptation is grounded in reliable [[student-modeling|student models]], but poorly-calibrated adaptation can harm learning. [[personalized-learning|Personalization research]] distinguishes effective adaptation from superficial customization. ### Connections Adaptive learning connects to [[knowledge-tracing]] (the diagnostic engine), [[personalized-learning]] (the goal), [[intelligent-tutoring]] (the platform), [[cognitive-diagnosis]] (fine-grained assessment), and [[scaffolding]] (adaptation as dynamic scaffolding). ## Connected Concepts - [[knowledge-tracing]] - [[personalized-learning]] - [[intelligent-tutoring]] - [[student-modeling]] - [[scaffolding]] - [[cognitive-diagnosis]] - [[llm]] - [[learning-analytics]] - [[higher-ed]] - [[k-12]] - [[formative-assessment]] ## Connected Articles - [[deeptutor]] - [[ai-powered-personalized-learning-elementary-fractions-2026]] - [[adaptive-pretesting-retention]] - [[adapt-adaptive-lesson-plan-transformer]] - [[zerkouk-comprehensive-review-its-2025]] --- ## [Adaptive Prompt Routing](https://edtechdev.github.io/aied/concepts/adaptive-prompt-routing/) > **Adaptive prompt routing** — dynamically selecting or composing LLM prompts per task and learner — connects the wiki's [[prompt-engineering]] and [[learning-to-prompt-adaptive-tutoring]] threads, where prompt selection becomes part of the tutoring system itself rather than a user skill. ## Connected Concepts - [[prompt-engineering]] ## Connected Articles - [[code-anchor-multi-view-visualization]] - [[confidence-aware-student-drawing-assessment]] - [[learning-to-prompt-adaptive-tutoring]] - [[psyscore-essay-scoring-zpd-feedback]] --- ## [AI from the Administrator Perspective](https://edtechdev.github.io/aied/concepts/administrator/) > Stub — pending source ingestion. AI adoption, strategy, and governance from the institutional administrator and leadership perspective. ## Connected Concepts - [[llm]] - [[generative-ai]] - [[student-experience]] - [[higher-ed]] - [[affective-computing]] - [[edtech-platform]] ## Connected Articles - [[ai-campus-wellbeing-tools]] - [[ai-uk-higher-education-policy-2026]] --- ## [Adult Learning](https://edtechdev.github.io/aied/concepts/adult-learning/) > **Adult Learning** — a key concept in AI in education research. Explored across 1 articles in this wiki. Research themes include: - Drawing on longitudinal deployment data from the National AI Institute for Adult Learning and Online Education (AI-ALOE), this DIS 2026 paper synthesizes 19 empirically grounded design guidelines for AI-powered adult learning technologies. The guidel ## Connected Concepts - [[adult-learning]] - [[higher-ed]] - [[accessible-learning]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[rct]] - [[active-learning]] - [[dot-framework-survey]] ## Connected Articles - [[ai-adult-learning-guidelines-dis2026]] — Guidelines for Designing AI Technologies to Support Adult Learning - [[institutional-governance-ai-universities]] — Policy Fragmentation or Institutional Alignment? Institutional Governance of AI in Universities and Business Schools - [[generative-ai-enhanced-learning-experiences-for-computational-thinking-a-systema]] — Generative AI-enhanced learning experiences for computational thinking: A systematic scoping review and design guidelines - [[unveiling-patterns-of-socially-shared-regulation-in-relation-to-self-regulated-l]] — Unveiling patterns of socially shared regulation in relation to self-regulated learning: The roles of individual profiles and group dynamics in online collaborative learning - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] — Mapping the Emerging Curriculum for AI-Assisted Software Engineering via Syllabus Analysis - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course --- ## [Affective Computing](https://edtechdev.github.io/aied/concepts/affective-computing/) > **Affective computing** in education uses physiological and behavioral signals to sense learner emotion and adapt instruction — see [[affective-text-wearable-student-health]], [[multimodal-affective-its-presentation]], and [[kar-mathbuddy-affective-math-tutoring-2025]]. The wiki also documents emotional risks of AI interaction, including [[sycophantic-ai-social-interaction-2026]] and [[shame-guilt-ai-regulation-computing-education]]. ## Connected Concepts - [[over-reliance]] - [[student-experience]] - [[k-12]] - [[feedback-loop]] - [[ai-tutoring]] - [[instructional-design]] - [[affective-tutoring]] - [[student-modeling]] - [[intelligent-tutoring]] - [[math-education]] - [[open-source]] - [[pedagogical-llm-training]] ## Connected Articles - [[ai-assisted-learning-modes-eeg]] - [[ai-campus-wellbeing-tools]] - [[ai-fatigue-academic-contexts]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[sycophantic-ai-social-interaction-2026]] --- ## [Affective Tutoring](https://edtechdev.github.io/aied/concepts/affective-tutoring/) > Integrating emotional awareness into AI tutoring systems can yield measurable pedagogical gains, but the same affective sophistication risks amplifying harms if learner agency is eroded by empathetic-seeming automation.^[[kar-mathbuddy-affective-math-tutoring-2025]]^[[favero-critical-ai-tutors-empower-enslave-2025]] MathBuddy dynamically models student affect using two modalities: - **Conversational text** — semantic cues for frustration, confusion, confidence - **Facial expressions** — real-time video capture of emotional state Emotions are aggregated from both modalities and mapped to relevant pedagogical strategies before prompting the LLM tutor, yielding emotionally-aware responses. **Results:** - **+23 point win rate** improvement over non-affective baseline - **+3 point DAMR score** gain at overall level - Evaluated across **eight pedagogical dimensions** plus user studies The finding validates a long-standing hypothesis in educational psychology: positive/negative emotional states impact learning capability, and accounting for them improves tutoring outcomes. ## The Risk: Empathy as a Trap Favero et al. (2025) warn that emotional engagement with AI tutors carries underappreciated risks: | Affective tutoring benefit | Corresponding risk | |---|---| | Emotionally-aware responses feel supportive | Students may form **parasocial dependencies** on the tutor | | Empathy reduces anxiety | Reduced anxiety may mask **metacognitive disengagement** | | Affective calibration personalizes pacing | Deep personalization can **reduce transfer** to non-adaptive contexts | | Facial monitoring signals attentiveness | Continuous video capture raises **privacy concerns** | The authors argue that emotional risks are part of a broader pattern of **erosion of self-efficacy, agency, and well-being** when AI use is unchecked. ## Design Principles 1. **Affective data should inform, not replace, learner autonomy** — The tutor adapts its strategy; the student retains control over disclosure 2. **Transparency about affect detection** — Students should know when and how their emotions are being inferred 3. **Affect-as-one-signal-among-many** — Combine with cognitive state (e.g., [[knowledge-tracing-irt]]) and behavioral engagement 4. **Privacy-by-default for multimodal sensors** — Facial/video data requires stronger protections than text-only inference ## Relationship to Broader Safety Affective tutoring intersects with [[ai-tutor-safety-harms|SafeTutors]] in the motivational-affective harm dimension. An affective tutor that is "too supportive" may suppress the frustration that drives productive struggle and self-regulation. See also [[llm-fallacy-misattribution]] — students may attribute emotional support to genuine relationship, reinforcing reliance. ## Connected Concepts - [[pedagogical-llm-training]] - [[intelligent-tutoring]] - [[personalized-learning]] - [[adaptive-learning]] - [[student-modeling]] - [[metacognition]] - [[self-regulated-learning]] - [[ai-tutoring]] - [[collaborative-learning]] - [[human-in-the-loop-ai]] - [[knowledge-tracing]] - [[socratic-method]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[ecnuclaw-k12-personalized-companion]] - [[empathy-coaching-chatbot]] - [[engagement-assessment-video]] - [[epistemic-emotions-collaborative-problem-solving]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[nie-personavlm-long-term-personalization-2026]] - [[zerkouk-comprehensive-review-its-2025]] --- ## [Learner Agency](https://edtechdev.github.io/aied/concepts/agency/) > **Learner agency** — the capacity of learners to act intentionally, make choices, and exercise control over their own learning. In AI in education, agency is a central concern because AI tools can both support and undermine learners' control: well-designed AI preserves and amplifies learner autonomy, while over-reliance or passive acceptance of AI output can erode it. Agency connects to [[self-regulated-learning]], [[motivation]], [[self-efficacy]], and the ethical design of AI systems, and is closely related to the psychological concepts of autonomy and sense of agency. Agency matters because learning is most effective when learners are active, intentional participants rather than passive recipients. AI systems — whether tutoring agents, [[social-robots|robots]], or [[llm|chatbots]] — shape how much control learners retain over their learning process. Preserving agency is therefore a key design principle in responsible AI in education, alongside supporting [[self-efficacy]], building [[trust]], and avoiding [[over-reliance]]. ### How agency appears in the wiki's research - **Robotics and human-robot interaction:** [[roboblockly-conversational-block-robotics-ct-2026|RoboBlockly Studio]] was explicitly designed to preserve learner agency in computational thinking; [[human-autonomy-agency-hri-review-2025|a systematic review]] examines how human-robot interaction affects human autonomy and sense of agency, central to well-being and governance debates. - **Collaborative learning:** [[human-ai-collaboration]] research examines how cognitive tasks are shared between learners and AI, with agency determining whether the human or the AI directs the interaction. - **Critical engagement:** [[cognitive-offloading|Cognitive offloading]] research shows how students who delegate interpretation to AI can lose agency over their own reasoning; critical and [[metacognition|metacognitive]] approaches aim to protect it. - **Design for agency:** Knowledge-based design for [[social-robots|generative social robots]] ([[teachy-mini-generative-social-robot-higher-ed-2026|Teachy Mini]]) addresses risks like overreliance that undermine learner agency. Agency connects to [[self-regulated-learning]], [[motivation]], [[self-efficacy]], [[student-experience]], [[human-ai-collaboration]], [[ethics]], [[over-reliance]], and [[metacognition]]. It is a core consideration in [[educational-robotics|robotics]], [[ai-tutoring|tutoring]], and the design of [[llm|AI learning agents]]. ## Connected Concepts - [[self-regulated-learning]] - [[motivation]] - [[self-efficacy]] - [[student-experience]] - [[human-ai-collaboration]] - [[metacognition]] - [[ethics]] - [[over-reliance]] - [[educational-robotics]] ## Connected Articles - [[human-autonomy-agency-hri-review-2025]] — Human Autonomy and Agency in HRI - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[cognitive-offloading]] — Cognitive Offloading in Student–AI Collaboration - [[teachy-mini-generative-social-robot-higher-ed-2026]] — Teachy Mini - [[knowledge-based-design-generative-social-robots-2026]] — Knowledge-Based Design for Generative Social Robots --- ## [Agentic AI in Education](https://edtechdev.github.io/aied/concepts/agentic-ai/) > **Agentic AI** — AI systems that autonomously plan, execute, and adapt multi-step workflows to achieve learning goals, going beyond single-turn Q&A to act as persistent, goal-directed collaborators: AI tutors that scaffold over extended interactions, multi-agent systems orchestrating instructional designs, and agents that co-regulate learning. The paradigm shift from prompt-responding tool to active collaborator is examined in [[agentic-ai-pedagogical-best-practice-2026]] and [[tool-invariant-framework-agentic-ai]]. Agentic AI refers to artificial intelligence systems that can autonomously plan, execute, and adapt multi-step workflows to achieve learning goals — going beyond single-turn question-answering to act as persistent, goal-directed collaborators in educational contexts. In education, agentic AI manifests as AI tutors that scaffold learning over extended interactions, multi-agent systems that orchestrate complex instructional designs, and autonomous agents that adapt their pedagogical strategies based on learner needs. This emerging paradigm shifts AI from a tool that responds to prompts to a collaborator that actively guides, adapts, and co-regulates learning processes. ## Connected Concepts - [[scaffolding]] - [[intelligent-tutoring]] - [[ai-literacy]] - [[prompt-engineering]] - [[curriculum-design]] - [[metacognition]] - [[rag]] - [[student-experience]] - [[adaptive-learning]] - [[faculty-development]] - [[human-in-the-loop-ai]] - [[ai-education]] ## Connected Articles - [[adversarial-stress-testing-role-playing-agents]] - [[agency-gap-ai-writing]] - [[agentic-ai-education-scoping-review]] - [[ai-adoption-training-public-sector]] - [[ai-agents-constructive-conflict-design-education-2026]] - [[ai-enabled-serious-games]] - [[ai-tpack-teacher-multi-agent-workflow]] - [[antiskillbench-persona-skills-privacy-2026]] - [[ase-26-agentic-software-engineering-curriculum]] - [[code-gen]] - [[deeptutor]] - [[educlaw-bench-pedagogical-llm-agents-2026]] - [[guided-llm-scaffolding-independent-learning]] - [[jeon-isd-agent-bench-2026]] - [[learnmate2-llm-adaptive-learning]] - [[pchl-he-framework-genai-content-creation-2026]] - [[cyberagents-gamified-cybersecurity-learning-2026]] - [[hdr-brachytherapy-agentic-ai-simulation-2026]] --- ## [AI Ed Evaluation](https://edtechdev.github.io/aied/concepts/ai-ed-evaluation/) > **AI-ed evaluation** — the body of methods, benchmarks, and criteria used to assess whether AI education tools (LLM-based tutors, automated graders, feedback systems, agents) actually work — not just on headline accuracy, but on reliability, pedagogical quality, validity, and real learning impact. A recurring theme across the wiki's research is that evaluation must be domain-specific, reliability-aware, and anchored in human judgment and educational outcomes rather than single aggregate accuracy numbers. AI-ed evaluation spans several distinct objects of assessment. It can evaluate the **output** (is the AI's answer, grade, or feedback correct and reliable?), the **process** (does the tool support valid, defensible assessment and learning?), and the **agent** (does an AI tutor or agent teach effectively and behave appropriately?). Each requires different methods and raises different validity questions. ### How AI-ed evaluation appears in the research - **Output reliability and ground truth:** [[ground-truth-reliability-aied|Modernizing ground truth]] argues that reliability problems in AI-ed evaluation often trace back to the reference data itself — the "ground truth" labels systems are judged against — and proposes four shifts toward improving reliability and validity. [[calibrating-trustworthiness-llm-education-2026|Calibrating trustworthiness]] co-designs evaluation metrics and visualizations with stakeholders so that trust in an AI tool rests on demonstrated, interpretable evidence. - **Automated grading and scoring:** [[automatic-short-answer-grading|LLM short-answer grading]], [[cong-confidence-asag-2026|confidence-aware ASAG]], [[cotal-formative-assessment-scoring-2026|CoTAL human-in-the-loop prompt engineering]], and [[llm-cognitive-diagnosis-handwritten-math|cognitive-diagnosis of handwritten math]] show that LLMs can grade and diagnose, but that reliability depends on [[human-in-the-loop-ai|human oversight]], domain-specific grounding, and confidence calibration rather than raw model size. - **Benchmarking and domain specificity:** [[teachbench-llm-teaching-evaluation|TeachBench]] evaluates LLM teaching ability, [[jeon-isd-agent-bench-2026|ISD Agent Benchmark]] evaluates agentic instructional-design agents, [[educational-vlm-evaluation|educational VLM evaluation]] assesses multimodal models, and [[tool-invariant-framework-agentic-ai|a tool-invariant framework]] assesses computational-method competency. These share a warning: generic benchmarks mislead, and evaluation must be tailored to the specific educational task and context. - **Pedagogical quality and alignment:** [[machines-misread-pedagogical-quality|Why machines misread pedagogical quality]] documents human–machine misalignment in judging what makes instruction good, and [[tutoring-effectiveness-index|the Tutoring Effectiveness Index]] predicts tutor quality from teaching behavior. [[responsible-assessment-ai-era-stanford-2026|Responsible assessment in the AI era]] and [[authentic-products-authenticated-processes-2026|authenticated processes]] argue that evaluation must reach beyond correct answers to whether assessment remains authentic, valid, and defensible when AI can produce the "products" of learning. - **Evaluating learning outcomes and agents:** [[zerkouk-comprehensive-review-its-2025|the ITS systematic review]], [[llm-difficulty-calibration-programming-exams-2026|LLM-difficulty calibration]], [[socratic-tests-conversational-assessment|Socratic conversational tests]], and [[valid-student-simulation-llm-2026|valid student simulation]] broaden evaluation to learning gains, test validity, and whether simulated students are a valid proxy for real learners. ### Why evaluation is hard in AI-ed AI-ed evaluation is difficult for several reasons. First, **reliability is not enough** — a system can agree with a rubric yet misjudge pedagogy, as [[machines-misread-pedagogical-quality|human–machine alignment research]] shows. Second, **ground truth is contested** — what counts as a "correct" answer, grade, or teaching move is itself a judgment that varies across disciplines and experts, per [[ground-truth-reliability-aied|ground-truth modernization]]. Third, **educational validity is multidimensional** — [[assessment-validity]], [[formative-assessment]], and [[authentic-assessment]] each impose different criteria that a single accuracy metric cannot capture. Finally, **the target keeps moving** — agentic AI and multimodal models demand evaluation frameworks ([[agentic-ai]], [[tool-invariant-framework-agentic-ai|tool-invariant assessment]]) rather than reuse of text-model benchmarks. ### Connections to related concepts AI-ed evaluation sits at the center of the wiki's methods and risks. It operationalizes [[assessment-validity]], [[educational-measurement]], and [[benchmark]] within [[assessment]] and [[automated-assessment]]. Its call for human oversight connects to [[human-in-the-loop-ai]] and [[teacher-role]], while its focus on reliability connects to [[hallucination-risk]], [[confidence-aware-ai-assessment]], and [[trust-calibration]]. The distinction between evaluating performance and evaluating learning links to [[genai-performance-vs-learning|performance vs. learning]] and to [[student-modeling]]; and evaluation of pedagogical agents connects to [[intelligent-tutoring]], [[pedagogical-llm-training]], and [[pedagogical-safety]]. ## Connected Concepts - [[assessment-validity]] - [[educational-measurement]] - [[benchmark]] - [[automated-assessment]] - [[formative-assessment]] - [[authentic-assessment]] - [[human-in-the-loop-ai]] - [[confidence-aware-ai-assessment]] - [[hallucination-risk]] - [[trust-calibration]] - [[intelligent-tutoring]] - [[agentic-ai]] - [[teacher-role]] - [[assessment]] - [[llm]] ## Connected Articles - [[ground-truth-reliability-aied]] — Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity - [[calibrating-trustworthiness-llm-education-2026]] — Calibrating Trustworthiness: Co-Designing Metrics and Visualizations - [[teachbench-llm-teaching-evaluation]] — TeachBench: Evaluating LLM Teaching Ability - [[machines-misread-pedagogical-quality]] — Why Machines Misread Pedagogical Quality: Human-Machine Alignment - [[automatic-short-answer-grading]] — Automatic Short Answer Grading With LLMs - [[cotal-formative-assessment-scoring-2026]] — CoTAL: Human-in-the-Loop Prompt Engineering for Formative Assessment - [[cong-confidence-asag-2026]] — Confidence-Aware Automatic Short Answer Grading - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking LLMs for Diagnosing Students' Cognitive Skills - [[tutoring-effectiveness-index]] — The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality - [[jeon-isd-agent-bench-2026]] — ISD Agent Benchmark - [[educational-vlm-evaluation]] — Educational VLM Evaluation - [[tool-invariant-framework-agentic-ai]] — A Tool-Invariant Framework for Teaching and Assessing Computational Methods - [[valid-student-simulation-llm-2026]] — Towards Valid Student Simulation With Large Language Models - [[llm-difficulty-calibration-programming-exams-2026]] — From Evaluated Models to Evaluation Aids - [[socratic-tests-conversational-assessment]] — The Theoretical Foundation of Socratic Tests - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era - [[authentic-products-authenticated-processes-2026]] — From Authentic Products to Authenticated Processes - [[zerkouk-comprehensive-review-its-2025]] — Comprehensive Review of Intelligent Tutoring Systems - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Feedback System - [[elbench-education-llm-benchmark-2026]] - [[teaching-monster-pck-benchmark-2026]] --- ## [AI Education](https://edtechdev.github.io/aied/concepts/ai-education/) > **AI Education** — the broad field encompassing both AI in education (using AI to teach) and AI literacy (teaching about AI). As the wiki's umbrella concept, AI education connects instructional technology, learning science, educational policy, and AI development. ### Two dimensions of AI education AI education research in the wiki spans two interconnected dimensions: - **AI for education:** Using AI to improve teaching and learning — [[ai-tutoring|AI tutoring]], [[automated-grading]], [[personalized-learning]], [[formative-assessment]], and [[learning-analytics]]. - **Education about AI:** Developing [[ai-literacy]] — teaching students and educators to understand, use, and critically evaluate AI. [[prompt-engineering]], [[critical-thinking]], and [[ethics]] are core literacy components. ### Cross-cutting themes - **Evidence and evaluation:** [[ai-ed-evaluation]], [[rct|randomized trials]], and [[efficacy-study|meta-analyses]] build the evidence base for what works. - **Policy and governance:** [[educational-policy-ai]], [[regulation]], and [[ai-governance-education]] address the institutional frameworks needed for responsible AI adoption. - **Equity and access:** [[equity]] and [[bias-mitigation]] research examines whether AI education serves all learners. - **Teacher preparation:** [[teacher-ai-competency]] and [[faculty-development]] ensure educators are prepared for AI-augmented classrooms. ### Field maturity The wiki reflects a field in rapid evolution — from early ITS research to LLM-driven tutoring, from detection-focused academic integrity to assessment redesign, from tool-centric studies to sociotechnical frameworks. ### Connections AI education connects to every concept in the wiki — it is the field that all other concepts collectively define. ## Connected Concepts - [[ai-literacy]] - [[ai-tutoring]] - [[generative-ai]] - [[llm]] - [[higher-ed]] - [[k-12]] - [[teacher-role]] - [[educational-policy-ai]] - [[equity]] - [[regulation]] - [[learning-analytics]] ## Connected Articles - [[metacognitively-discordant-completion-genai-2026]] - [[ai-interior-design-malaysia-2026]] - [[raza-farooq-aied-review-2020-2025]] - [[genai-policies-higher-ed-computing]] — Comparative institutional/course GenAI policy in computing - [[liang-genai-systematic-review-human-ai-2026]] - [[brookings-ai-students-report]] - [[white-wu-robotics-ai-education-2026]] - [[institutional-change-framework-ai]] --- ## [AI Feedback Quality](https://edtechdev.github.io/aied/concepts/ai-feedback-quality/) > **AI feedback quality** — the accuracy, usefulness, timeliness, and pedagogical value of feedback generated by AI systems for learners. As AI-generated feedback becomes ubiquitous in education, understanding what makes feedback effective — and when it falls short — is critical to ensuring AI supports rather than undermines learning. AI feedback quality is not simply about correctness. Effective feedback must be timely, specific, actionable, and calibrated to the learner's current understanding. Research in this wiki examines AI feedback quality across multiple dimensions: accuracy (is the feedback correct?), usefulness (does it help the student improve?), and pedagogical alignment (does it promote learning rather than just task completion?). ### How AI feedback quality appears in the research - **Comparability to human feedback:** [[ai-generated-feedback-higher-ed|Studies in higher education]] find that AI-generated feedback is experienced as acceptable and supportive — comparable to teacher feedback. But acceptability does not guarantee learning effectiveness. - **Feedback classification benchmarks:** [[teaching-feedback-classification-benchmark|Cross-language feedback benchmarks]] assess whether feedback quality classification transfers across languages and educational contexts, connecting to [[ai-ed-evaluation]]. - **Collaborative feedback systems:** [[becerra-aicofe-feedback-2026|AICoFE]] implements and deploys AI-based collaborative feedback in higher education, evaluating both system performance and student reception. - **Automated grading feedback:** [[automated-grading]] and [[formative-assessment]] research examine whether AI-scored assessments provide feedback that matches or exceeds human grading quality. - **Essay scoring feedback:** [[cong-confidence-asag-2026|Confidence-aware ASAG]] and [[choi-anchor-aes-prompting-2025|anchor-based AES]] explore how confidence calibration and prompting design affect feedback quality for writing assessment. - **Discretionary feedback provision:** [[ai-assistance-discretionary-feedback|Research on AI-assisted feedback in higher education]] examines whether AI increases the quantity and quality of feedback instructors provide. ### Quality dimensions AI feedback quality spans multiple dimensions captured in the wiki: - **Accuracy:** Does the feedback correctly identify errors and strengths? ([[automated-grading]], [[automated-essay-scoring]]) - **Helpfulness:** Does the feedback guide improvement? ([[feedback-loop]], [[ai-peer-feedback-systems]]) - **Timeliness:** Is feedback delivered when the learner can act on it? ([[formative-assessment]]) - **Bias:** Is feedback equitable across student populations? ([[bias-mitigation]], [[equity]]) - **Calibration:** Does the system know when it's uncertain? ([[confidence-aware-ai-assessment]]) ### Connection to broader concepts AI feedback quality connects fundamentally to [[formative-assessment]] and [[feedback-loop]] — quality feedback closes the gap between current and desired performance. It intersects with [[automated-grading]] (which generates the scores feedback is based on), [[ai-literacy]] (students must evaluate feedback quality critically), and [[over-reliance]] (uncritical acceptance of AI feedback can displace learning). For [[writing-education]], feedback quality is particularly consequential given AI's growing role in writing assessment. ## Connected Concepts - [[formative-assessment]] - [[automated-grading]] - [[feedback-loop]] - [[ai-literacy]] - [[over-reliance]] - [[bias-mitigation]] - [[automated-essay-scoring]] - [[assessment-validity]] - [[writing-education]] - [[higher-ed]] - [[teacher-role]] - [[confidence-aware-ai-assessment]] ## Connected Articles - [[ai-generated-feedback-higher-ed]] — AI-Generated Feedback in Higher Education - [[teaching-feedback-classification-benchmark]] — Teaching Feedback Classification Benchmark - [[becerra-aicofe-feedback-2026]] — AICoFE: AI-Powered Collaborative Feedback - [[automated-grading]] — Automated Short Answer Grading - [[cong-confidence-asag-2026]] — Confidence-Aware Short Answer Grading - [[choi-anchor-aes-prompting-2025]] — Anchor-Based AES Prompting - [[ai-assistance-discretionary-feedback]] — AI Assistance for Discretionary Feedback - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[sequenced-ai-feedback-learning]] — Sequenced AI Feedback and Learning --- ## [AI Governance Education](https://edtechdev.github.io/aied/concepts/ai-governance-education/) > **AI governance in education** spans institutional policy, [[regulation]], and [[privacy]]: [[genai-policies-higher-ed-computing]], [[genai-declaration-frameworks-higher-education]], and [[genai-assessment-governance]] document how universities translate AI capability into acceptable-use frameworks and assessment rules. ## Connected Concepts - [[regulation]] - [[privacy]] ## Connected Articles - [[favero-critical-ai-tutors-empower-enslave-2025]] - [[genai-policies-higher-ed-computing]] - [[raza-farooq-aied-review-2020-2025]] - [[white-wu-robotics-ai-education-2026]] - [[ai-uk-higher-education-policy-2026]] --- ## [AI Literacy](https://edtechdev.github.io/aied/concepts/ai-literacy/) > **AI literacy** — the knowledge, skills, and critical dispositions needed to understand, evaluate, and effectively use AI technologies in educational contexts. AI literacy spans foundational understanding of how AI works, practical competence in using AI tools, critical evaluation of AI outputs, and ethical awareness of AI's societal implications. AI literacy has rapidly emerged as a core competency for learners, educators, and institutions as [[generative-ai|generative AI]] becomes embedded in education. Unlike general digital literacy, AI literacy requires understanding probabilistic systems that can hallucinate, exhibit bias, and shift agency from human to machine — making [[critical-thinking]] and [[over-reliance]] central to the construct. ### Dimensions of AI literacy AI literacy research in this wiki spans four interconnected dimensions: **Foundational knowledge:** Understanding what [[llm|LLMs]] are, how they differ from rule-based systems, and their fundamental limitations. This includes awareness of model capabilities, training data biases, and the distinction between task-specific AI and general-purpose models. Research in [[prompt-engineering]] examines how understanding prompt mechanisms affects effective AI use. **Practical competence:** The ability to use AI tools effectively — from [[prompt-engineering]] to interpreting outputs. Studies of [[genai-usage-design-students-survey|student GenAI usage patterns]] reveal that tool access alone doesn't produce competence; structured practice and [[scaffolding]] are essential. The [[gaide-vibe-coding-k12-teachers|vibe coding framework]] shows how K-12 teachers can develop practical AI literacy through guided tool creation. **Critical evaluation:** The capacity to assess AI outputs for accuracy, bias, and appropriateness. [[ai-literacy-assessment-misalignment|Research on literacy assessment]] shows a 40% gap between self-reported and performance-based AI literacy — people consistently overestimate their evaluation skills. This connects to [[over-reliance]] research showing that students who trust AI uncritically learn less. **Ethical and institutional awareness:** Understanding AI's broader implications — from [[academic-integrity]] to [[equity]] to [[privacy]]. AI literacy at the institutional level involves policy development, [[faculty-development]], and governance frameworks. The [[sangwa-epiq-ai-faculty-readiness-2026|EPIQ-AI framework]] frames institutional AI literacy as a sociotechnical alignment challenge, not just individual training. ### How AI literacy is developed Research points to [[collaborative-learning|collaborative]] and [[active-learning|active]] approaches as most effective. The ICAP framework (Passive → Active → Constructive → Interactive) provides a useful progression: students learn AI literacy best when they co-construct knowledge rather than passively receive information. Practical activities — designing prompts, evaluating outputs in groups, debating AI ethics — outperform lectures. ### Connections across the wiki AI literacy intersects with [[ai-tutoring]] (understanding when and how AI tutors are effective), [[teacher-ai-competency]] (educator preparedness), [[academic-integrity]] (knowing what constitutes appropriate AI use), and [[ai-education]] broadly. It is both a prerequisite for effective AI use and an outcome of well-designed AI integration — students learn AI literacy BY using AI critically, not just by learning ABOUT AI. ## Connected Concepts - [[student-experience]] - [[higher-ed]] - [[over-reliance]] - [[generative-ai]] - [[scaffolding]] - [[teacher-role]] - [[formative-assessment]] - [[faculty-development]] - [[self-regulated-learning]] - [[metacognition]] - [[academic-integrity]] - [[equity]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[ai-acceptance-preservice-science-teachers-2026]] - [[critical-media-literacy-education-2026]] - [[agency-gap-ai-writing]] - [[agent-voice-accents-k12-group-learning]] - [[agentic-literacy-debt]] - [[agents-that-teach-incidental-learning]] - [[ai-adoption-training-public-sector]] - [[ai-assessment-scale-reform]] - [[ai-changing-teaching-workflows]] - [[ai-ethics-education-public-discourse]] - [[ai-fatigue-academic-contexts]] - [[ai-generated-feedback-higher-ed]] - [[ai-generated-slides-student-perception]] - [[ai-higher-ed-workforce-survey]] - [[ai-in-the-wild-college]] - [[ai-learning-assistants-higher-ed-large-scale]] - [[ai-learning-tools-engineering-education-needs]] - [[ai-literacy-career-adaptability-business-2026]] - [[genai-literacy-training-teacher-education-dbr-2026]] - [[hcap-human-centric-ai-pedagogy-framework-2026]] - [[metacognitive-ai-literacy-beyond-skills-gap-2026]] - [[possibility-ai-literacy-critical-editorial]] - [[sec-ai-literacy-narrative-review-2026]] - [[teacher-education-ai-literacy-sdt-2026]] - [[pchl-he-framework-genai-content-creation-2026]] - [[elementary-writing-genai-systematic-review-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [AI Misuse and Learning Harm](https://edtechdev.github.io/aied/concepts/ai-misuse-learning-harm/) > **AI misuse and learning harm** — the causal relationship between students offloading cognitive work to generative AI and reduced durable learning, even when immediate task performance rises. The defining feature is a performance–learning gap: AI inflates assisted performance while degrading unassisted, closed-book, and retention outcomes. AI misuse is distinct from AI use. Use describes employing AI as a complement to learning — feedback, brainstorming, or revision help that keeps the learner's cognitive work in the loop. Misuse describes substitution: delegating to AI the very mental processes (drafting, recall, analysis, revision) that build durable understanding. The harm documented in the wiki's evidence base is not that misuse fails to help; it is that misuse actively degrades later, unassisted achievement. ### The performance–learning gap The core concept, articulated in [[genai-performance-vs-learning]], is that generative AI easily boosts **performance** — immediate efficiency and output quality — while often bypassing the deep cognitive and metacognitive processing required for **learning**. A tool that optimizes for performance can therefore undermine learning. The gap is now causally demonstrated at field scale: a randomized controlled trial found unguarded AI assistance raised practice performance but reduced later unassisted exam scores. ### Mechanisms of harm - **Cognitive surrender** — the term researchers use for students offloading thinking to AI as a passive, unreflective dependency, as opposed to the deliberate, strategic form of [[cognitive-offloading]]. It produces a measurable population-level decline in durable knowledge. - **Answer-copying as a crutch** — misuse is driven less by AI errors misleading students than by students copying answers instead of learning. When engagement analysis shows students mostly "ask for the answer," learning harm follows. - **Motivation erosion** — the perceived availability of an effortless AI shortcut reduces autonomous motivation and persistence, per self-determination theory. Because persistence is what produces deep learning, its erosion compounds the direct harm. - **Learning displacement** — the substitution of AI output for the effortful processes (elaboration, recall, self-explanation) that consolidate knowledge, consistent with [[over-reliance]]. ### The evidence base - **Causal field RCT (≈1,000 high-school math students):** an unguarded ChatGPT-style tutor raised assisted practice performance **+48%** but reduced unassisted, closed-book exam scores **−17%** — students who never had AI access outperformed those who did. A guardrailed "hint-not-answer" tutor eliminated the harm. Notably, students in the harmed arm did not perceive they learned less. - **Population-scale behavioral data (3.2M ALEKS interactions):** study time on AI-susceptible problems fell **−26.9%** cumulatively for college students (high school −31.3%) after ChatGPT's release, with a **−25% decline in odds of a correct response on proctored retention items**. The effect vanished entirely under proctoring, pinning it on off-platform AI use. - **A large null result:** exploiting the seasonal drop in ChatGPT use over summer showed **no net change in high-school standardized test averages** — likely because misuse harm is offset in aggregate by productive AI use. This does not contradict the causal harm to durable learning; it cautions against over-generalizing from aggregate test scores. ### The assessment-dependent nature of harm The most important practical nuance is that the harm is **selective by assessment type**. It shows up on **proctored, closed-book, and unassisted** measures of durable knowledge. On normal graded coursework that cannot distinguish AI-assisted from independent work, misuse can *inflate* immediate grades. This is why the perceived-vs-actual gap is dangerous: students (and sometimes instructors) see short-term performance gains and miss the erosion of learning that only surfaces when the tool is removed. ### Implications and remedies - **Guardrails over raw access:** hint-not-answer prompting and teacher-authored scaffolding neutralize the crutch effect (see [[generative-ai-guardrails-harm-learning]]). - **Assessment design:** AI-resistant and proctored/unassisted assessments are needed to surface — and discourage — misuse. - **Literacy and metacognition:** [[ai-literacy]] and [[self-regulated-learning]] training that helps students recognize reliance patterns and the cost of bypassing their own cognitive work. ## Connected Concepts - [[over-reliance]] - [[cognitive-offloading]] - [[academic-integrity]] - [[assessment]] - [[self-regulated-learning]] - [[motivation]] - [[metacognition]] - [[scaffolding]] - [[generative-ai]] - [[student-experience]] ## Connected Articles - [[generative-ai-guardrails-harm-learning]] — GenAI Without Guardrails Can Harm Learning - [[generative-ai-reduced-study-time-math]] — Generative AI Reduced Study Time on Math - [[genai-performance-vs-learning]] — Distinguishing Performance Gains from Learning - [[chatgpt-impact-high-school-tests]] — Little Impact of ChatGPT on High School Test Scores - [[ai-availability-student-motivation]] — AI Availability and Student Motivation - [[genai-skill-bypass-literacy]] — GenAI Skill Bypass and Literacy - [[cognitive-shift-ai-education]] — Cognitive Shift in AI Education - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration --- ## [AI Tutoring](https://edtechdev.github.io/aied/concepts/ai-tutoring/) > **AI tutoring** — the use of AI (especially [[llm|LLMs]] and [[intelligent-tutoring|intelligent tutoring systems]]) to provide personalized, adaptive, scalable instructional support: conversational tutors, scaffolded feedback systems, adaptive platforms, and agent-based tutors with long-term learner models. Effectiveness hinges on pedagogical design ([[scaffolding]], feedback quality, autonomy balance) rather than the model alone — see [[measuring-llm-tutors-teach-vs-solve]] and [[socratic-method]]. AI tutoring encompasses the use of artificial intelligence — particularly large language models and intelligent tutoring systems — to provide personalized, adaptive, and scalable instructional support to learners. AI tutors can take many forms: conversational tutors that engage in Socratic dialogue, scaffolded feedback systems that guide problem-solving, adaptive learning platforms that personalize content sequencing, and agent-based tutors that maintain long-term learner models. The effectiveness of AI tutoring depends critically on pedagogical design choices — scaffolding, feedback quality, and the balance between autonomy and guidance — rather than on the underlying model alone. ## Connected Concepts - [[intelligent-tutoring]] - [[pedagogical-safety]] - [[llm]] - [[student-modeling]] - [[k-12]] - [[scaffolding]] - [[pedagogical-llm-training]] - [[affective-tutoring]] - [[over-reliance]] - [[assessment-validity]] - [[stem-education]] - [[item-response-theory]] ## Connected Articles - [[adversarial-stress-testing-role-playing-agents]] - [[hazra-safetutors-pedagogical-safety-2026]] - [[huang-interpretable-knowledge-tracing-2026]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[nie-personavlm-long-term-personalization-2026]] - [[oecd-digital-education-outlook-2026]] - [[representation-robustness-llm-math-problem-solving]] - [[stanford-evidence-base-ai-k12-2026]] - [[tact-pedagogically-adaptive-esl-tutoring]]- [[ai-tutoring-quality-k12-methodologies-2026]] - [[hdr-brachytherapy-agentic-ai-simulation-2026]] - [[residencyrl-clinical-rl-training-2026]] --- ## [Assessment Validity in AI Education](https://edtechdev.github.io/aied/concepts/assessment-validity/) > **Assessment validity** — whether assessments measure what they claim to measure. AI in education raises fundamental validity questions: do AI-graded assessments assess student learning or AI prompting skill? Does AI use invalidate traditional assessment assumptions? ### Validity challenges - **Construct validity:** When students use AI on assessments, does the score reflect student knowledge or AI capability? [[genai-performance-vs-learning|Performance vs. learning]] research addresses this directly. - **Consequential validity:** Do AI-mediated assessments have fair consequences? [[ai-scoring-language-bias-physics|Language bias studies]] show that AI scoring can disadvantage non-native speakers. - **Authentic assessment:** [[authentic-assessment]] and [[ai-assessment-scale-reform|the AI Assessment Scale]] propose validity-preserving assessment redesigns. - **Confidence and calibration:** [[confidence-aware-ai-assessment|Confidence-aware systems]] improve validity by flagging uncertain assessments. ### Redesign over detection The wiki argues that maintaining assessment validity requires redesigning assessments for AI-capable students, not detecting AI use. [[beyond-detection-authentic-assessment-ai-2025|Beyond detection approaches]] and [[assessment]] represent validity-forward thinking. ### Connections Assessment validity connects to [[authentic-assessment]], [[automated-grading]], [[confidence-aware-ai-assessment]], [[formative-assessment]], [[academic-integrity]], and [[rct]] (which relies on valid outcome measures). ## Connected Concepts - [[authentic-assessment]] - [[automated-grading]] - [[confidence-aware-ai-assessment]] - [[formative-assessment]] - [[academic-integrity]] - [[rct]] - [[bias-mitigation]] - [[equity]] - [[ai-ed-evaluation]] - [[educational-measurement]] - [[llm]] ## Connected Articles - [[competency-based-education-genai-production-2026]] - [[genai-performance-vs-learning]] - [[ai-scoring-language-bias-physics]] - [[ai-assessment-scale-reform]] - [[beyond-detection-authentic-assessment-ai-2025]] - [[confidence-aware-student-drawing-assessment]] - [[cong-confidence-asag-2026]] --- ## [Assessment](https://edtechdev.github.io/aied/concepts/assessment/) > **Assessment** is a central concept in AI in education research, connected to 8 articles in this wiki. ## Connected Concepts - [[higher-ed]] - [[learning-analytics]] - [[adaptive-learning]] - [[academic-integrity]] - [[llm]] - [[automated-grading]] - [[personalized-learning]] - [[generative-ai]] - [[privacy]] - [[knowledge-tracing]] - [[formative-assessment]] - [[student-modeling]] ## Connected Articles - [[competency-based-education-genai-production-2026]] - [[llm-difficulty-calibration-programming-exams-2026]] - [[elementary-writing-genai-systematic-review-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Automated Assessment](https://edtechdev.github.io/aied/concepts/automated-assessment/) > **Automated assessment** — the use of AI to evaluate student work, from formative quizzes to high-stakes exams. Automated assessment spans multiple modalities — multiple-choice, short answer, essay, code, and performance-based evaluation. ### Assessment modalities - **Short answer and essay:** [[automated-grading]], [[automated-essay-scoring]], and [[cong-confidence-asag-2026|confidence-aware approaches]] handle free-text evaluation. - **Code assessment:** [[automated-grading-linux-bash-examinations-large-language-models|Bash grading]] and [[code-review-genai-cs1|code review]] demonstrate programming assessment. - **Formative assessment:** [[automated-formative-assessments-a-level-sciences|A-level science automation]] and [[cotal-formative-assessment-scoring-2026|CoTAL]] focus on formative rather than summative use. - **Performance assessment:** [[engagement-assessment-video|Video engagement assessment]] and [[confidence-aware-student-drawing-assessment|drawing assessment]] extend automation beyond text. ### Quality and fairness Automated assessment quality depends on [[assessment-validity]] and [[bias-mitigation]]. [[ai-scoring-language-bias-physics|Language bias]] research shows that automated scoring can systematically disadvantage certain student populations. ### Connections Automated assessment connects to [[automated-grading]] (the scoring mechanism), [[assessment-validity]] (quality assurance), [[formative-assessment]] (use context), [[bias-mitigation]] (fairness), and [[teacher-role]] (how automation changes instructor work). ## Connected Concepts - [[automated-grading]] - [[assessment-validity]] - [[formative-assessment]] - [[bias-mitigation]] - [[teacher-role]] - [[automated-essay-scoring]] - [[confidence-aware-ai-assessment]] - [[llm]] - [[higher-ed]] - [[ai-ed-evaluation]] ## Connected Articles - [[competency-based-education-genai-production-2026]] - [[multimodal-item-parameter-estimation-2026]] - [[automated-grading]] - [[automated-formative-assessments-a-level-sciences]] - [[cong-confidence-asag-2026]] - [[ai-scoring-language-bias-physics]] - [[llm-difficulty-calibration-programming-exams-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Automated Essay Scoring](https://edtechdev.github.io/aied/concepts/automated-essay-scoring/) > **Automated Essay Scoring (AES)** — the use of AI to evaluate and score written essays, spanning traditional statistical approaches, fine-tuned language models, and increasingly accessible LLM-based prompting strategies. AES research in this wiki covers scoring accuracy, fairness and bias, psychometric validity, and practical accessibility for educators. Automated Essay Scoring has a long history in educational technology, from early statistical models to modern LLM-based approaches that can evaluate essays holistically without large pre-scored datasets. The key tension in AES research is between accuracy and accessibility — while fine-tuned models achieve strong results, they are resource-intensive and impractical for most educators. ### Key research themes **Prompting-based AES** has emerged as the most accessible approach. The **[[choi-anchor-aes-prompting-2025|Choi et al. anchor paper study]]** shows that including exemplar essays in prompts brings LLM-human agreement close to human-human reliability, with GPT-4o mini achieving comparable results to GPT-4o at lower cost. This connects to broader [[prompt-engineering]] research and makes AES feasible for teacher use. **Psychometric and trait-level scoring** moves beyond holistic scores. **[[psyscore-essay-scoring-zpd-feedback|PsyScore]]** provides a psychometrically-aware framework for trait-adaptive scoring with ZPD-grounded feedback. **[[icle-plus-plus-essay-scoring|ICLE++]]** models fine-grained traits for holistic essay scoring, advancing the precision of automated evaluation. **Bias and fairness** is a critical concern. **[[ai-scoring-language-bias-physics|Feser & Tschisgale]]** found that AI scoring systematically underestimates students from linguistically diverse backgrounds, highlighting the need for [[bias-mitigation]] and [[equity]] considerations in AES deployment. **L2 and self-referential assessment** explores non-native writing contexts. **[[self-referential-l2-writing-llm-assessment|Profile-based L2 assessment]]** uses a self-referential approach comparing student writing to their own prior work rather than native-speaker norms. ### Connections to related concepts AES sits at the intersection of [[automated-assessment]], [[writing-education]], and [[generative-ai]]. It connects to [[formative-assessment]] when used for feedback rather than grading, to [[feedback-loop]] when integrated into iterative writing processes, and to [[ai-literacy]] when educators understand and calibrate AES tools. The [[assessment-validity]] and [[educational-measurement]] concepts are essential for ensuring AES scores are meaningful and fair. ## Connected Concepts - [[bias-mitigation]] - [[equity-in-ai-education]] - [[equity]] - [[automated-grading]] - [[language-learning]] - [[educational-measurement]] - [[k-12]] - [[prompt-engineering]] - [[writing-education]] - [[automated-assessment]] - [[ai-literacy]] - [[assessment-validity]] ## Connected Articles - [[ai-scoring-language-bias-physics]] - [[choi-anchor-aes-prompting-2025]] - [[icle-plus-plus-essay-scoring]] - [[psyscore-essay-scoring-zpd-feedback]] - [[self-referential-l2-writing-llm-assessment]] - [[aiawe-automated-writing-evaluation]] --- ## [Automated Grading](https://edtechdev.github.io/aied/concepts/automated-grading/) > **Automated grading** — AI systems that evaluate student work, from multiple-choice scoring to essay assessment and code review. Automated grading is one of the most mature and widely-deployed AI in education applications. ### Grading modalities - **Short answer grading:** [[automated-grading]] and [[cong-confidence-asag-2026|confidence-aware ASAG]] evaluate free-text responses. [[confidence-aware-ai-assessment|Confidence calibration]] is critical — systems must know when grading is reliable. - **Essay scoring:** [[automated-essay-scoring]] systems like [[choi-anchor-aes-prompting-2025|anchor-based AES]] use prompting strategies to approach human-level reliability. [[aiawe-automated-writing-evaluation|AIAWE]] extends automated evaluation to broader writing assessment. - **Code review:** [[automated-grading-linux-bash-examinations-large-language-models|Linux Bash grading]] and [[code-review-genai-cs1|CS1 code review]] demonstrate automated assessment in computing education. - **Formative assessment integration:** [[automated-formative-assessments-a-level-sciences|A-level science automation]] and [[cotal-formative-assessment-scoring-2026|CoTAL]] show how automated grading feeds into [[formative-assessment]] cycles. - **Bias and fairness:** [[ai-scoring-language-bias-physics|Language bias in physics scoring]] documents how automated grading can disadvantage non-native speakers — connecting to [[bias-mitigation]] and [[equity]]. ### Connections Automated grading connects to [[assessment-validity]] (do automated scores measure what they claim?), [[teacher-role]] (how does automation change instructor work?), and [[ai-feedback-quality]] (grading without useful feedback has limited educational value). ## Connected Concepts - [[assessment-validity]] - [[formative-assessment]] - [[automated-essay-scoring]] - [[ai-feedback-quality]] - [[bias-mitigation]] - [[confidence-aware-ai-assessment]] - [[equity]] - [[teacher-role]] - [[llm]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[automated-grading]] - [[cong-confidence-asag-2026]] - [[choi-anchor-aes-prompting-2025]] - [[ai-scoring-language-bias-physics]] - [[cotal-formative-assessment-scoring-2026]] - [[automated-grading-linux-bash-examinations-large-language-models]] --- ## [Automated Question Generation](https://edtechdev.github.io/aied/concepts/automated-question-generation/) Automated question generation leverages NLP and LLMs to create educational assessments at scale. Wei & Stamper (2025) introduced the **generate-then-validate** paradigm, reducing hallucination by 62% compared to direct generation and achieving 89% accuracy on STEM datasets. ### Generate-Then-Validate Paradigm 1. **Generation Phase:** LLM produces candidate questions from source material 2. **Validation Phase:** Separate verification step filters invalid/low-quality items 3. **Refinement Loop:** Failed items trigger re-generation with corrective prompts ### Advantages Over Direct Generation - **Reduced Hallucination:** Validation catches factually incorrect questions - **Higher Relevance:** 23% improvement on relevance metrics vs. baseline LLMs - **Scalability:** Enables rapid creation of formative assessments across domains ### Educational Applications - **Formative Assessment:** Just-in-time questions for adaptive learning systems - **STEM Education:** Validated on mathematics and science problem generation - **Differentiated Instruction:** Generating multiple difficulty levels automatically ## References Wei, Y., Stamper, J., et al. (2025). *Generate-Then-Validate: A Novel Question Generation Approach*. arXiv:2512.10110. ## Source - https://arxiv.org/abs/2512.10110 ## Connected Concepts - [[personalized-learning]] - [[human-in-the-loop-ai]] - [[edtech-platform]] - [[learning-analytics]] - [[llm]] - [[intelligent-tutoring]] - [[scaffolding]] - [[pedagogical-llm-training]] - [[formative-assessment]] - [[adaptive-learning]] - [[automated-assessment]] - [[rag]] ## Connected Articles - [[code-gen]] - [[generate-then-validate-question-gen]] - [[kt4eqg-personalized-question-generation]] - [[nsmq-riddles-science-math-benchmark]] - [[self-referential-l2-writing-llm-assessment]] - [[slidesqaqa-pedagogical-question-generation]] - [[taklif-ai-interest-based-personalized-assignments]] - [[zerkouk-comprehensive-review-its-2025]] --- ## [Benchmark](https://edtechdev.github.io/aied/concepts/benchmark/) > **Benchmark** — standardized test suites and evaluation frameworks used to measure AI model performance on educational tasks. Benchmarks enable reproducible comparison across models and approaches, and are essential for evaluating the reliability, fairness, and pedagogical quality of AI in education systems. Benchmarks serve as the evidentiary foundation of AI in education research. They provide standardized datasets, tasks, and metrics that allow researchers to compare models, track progress, and identify failure modes. In the wiki's research, benchmarks appear across multiple domains: - **[[educlaw-bench-pedagogical-llm-agents-2026|EduClaw-Bench]]** introduces a long-horizon benchmark for pedagogical LLM agents using simulated learners grounded in [[knowledge-tracing]]. - **[[cstutorbench-slm-tutors|CSTutorBench]]** evaluates small language models for CS tutoring tasks. - **[[anvil-ai-educational-animations|ANVIL]]** benchmarks AI-generated educational animations against human-created alternatives. - **[[teaching-feedback-classification-benchmark|Teaching feedback benchmarks]]** assess cross-language transfer of feedback quality classification. ### Why benchmarks matter in AIED Benchmarks connect to [[ai-ed-evaluation]] and [[assessment-validity]] — without rigorous benchmarks, claims about AI tutoring effectiveness are unverifiable. They also intersect with [[bias-mitigation]], as benchmark design can encode or amplify biases. The tension between benchmark performance and real-world utility is explored across multiple articles, connecting to [[transfer-of-learning]] concerns in [[generative-ai]] applications. ## Connected Concepts - [[ai-ed-evaluation]] - [[bias-mitigation]] - [[benchmark]] - [[human-in-the-loop-ai]] - [[formative-assessment]] - [[knowledge-tracing]] - [[generative-ai]] - [[automated-essay-scoring]] ## Connected Articles - [[authentic-products-authenticated-processes-2026]] — From authentic products to authenticated processes: authentic assessment in AI-rich higher education - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work - [[educlaw-bench-pedagogical-llm-agents-2026]] — EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners - [[responsible-assessment-ai-era-stanford-2026]] — Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[icle-plus-plus-essay-scoring]] — ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring - [[elbench-education-llm-benchmark-2026]] - [[teaching-monster-pck-benchmark-2026]] --- ## [Bias Mitigation](https://edtechdev.github.io/aied/concepts/bias-mitigation/) > **Bias mitigation** in educational AI requires auditing models across the pipeline: [[gender-bias-transfer-llm-writing]], [[ai-scoring-language-bias-physics]], [[llm-cultural-relevance-k12]], and [[equity]] (merged into [[equity]]) document bias sources and mitigation strategies from data curation to prompt design. Bias mitigation in AIED concerns identifying and reducing unfair, identity-patterned behavior in AI tutors and educational systems (e.g., differential refusals or softening of answers for marginalized students). Surfaced by recent auditing work on LLM history tutors. ## Connected Concepts - [[student-experience]] - [[equity]] - [[ai-literacy]] - [[higher-ed]] - [[over-reliance]] - [[teacher-role]] - [[generative-ai]] - [[scaffolding]] - [[human-in-the-loop-ai]] - [[privacy]] - [[automated-grading]] - [[llm]] ## Connected Articles - [[ai-changing-teaching-workflows]] - [[ai-scoring-language-bias-physics]] - [[antiskillbench-persona-skills-privacy-2026]] - [[data-annotations-pedagogical-hints]] - [[genai-minoritized-knowledges-disability]] - [[paternalistic-filter-llm-history-education]] - [[xai-education-framework]] - [[genai-higher-education-systematic-review-2026]] --- ## [Block-Based Programming](https://edtechdev.github.io/aied/concepts/block-programming/) > **Block-based programming** — a visual programming paradigm in which learners build programs by snapping together graphical blocks (e.g., Scratch, Blockly) rather than typing text. Block-based environments lower the barrier to [[programming-education|programming]] by eliminating syntax errors and making program structure visible, which is especially valuable for beginners and younger learners. They are widely used to teach [[computational-thinking]] and to control [[educational-robotics|educational robots]]. Block-based programming is closely related to [[programming-education]], [[computational-thinking]], and [[cs-education]]. In the AI era, block-based environments are increasingly combined with conversational AI agents and embodied robot execution to help learners connect abstract program logic to meaningful, observable outcomes. ## Connected Concepts - [[programming-education]] - [[computational-thinking]] - [[cs-education]] - [[educational-robotics]] - [[k-12]] ## Connected Articles - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education --- ## [Cognitive Diagnosis](https://edtechdev.github.io/aied/concepts/cognitive-diagnosis/) > **Cognitive diagnosis** — the inference of a learner's latent knowledge state — the specific concepts, skills, and misconceptions they have or lack — from their responses or behavior. It is the assessment-side counterpart to [[knowledge-tracing]], focused on characterizing *what* a student knows rather than only predicting their next performance. Whereas knowledge tracing typically estimates a scalar mastery over time, cognitive diagnosis produces a more granular profile: which knowledge components are mastered, which are fragile, and which misconceptions are present. This profile is the substrate for [[personalized-learning]], [[intelligent-tutoring]], and [[adaptive-learning]]. ### How cognitive diagnosis works - **Diagnostic models:** psychometric models (often under [[item-response-theory]] and [[educational-measurement]]) infer latent skill states from patterns of correct and incorrect responses, sometimes via [[cognitive-diagnosis]] models that map items to multiple knowledge components. - **Response data:** diagnosis draws on responses to assessments, hints, help-seeking, and time-on-task — richer signals than raw scores. - **LLM-based diagnosis:** newer approaches use large language models to diagnose from open-ended or handwritten work, and to identify the specific misconceptions behind an error (e.g., the "correct answer trap" where a right answer conceals flawed reasoning). ### Why it matters Accurate diagnosis lets instruction target the actual gaps rather than a global "ability" score — enabling [[automated-assessment]] that explains *why* a student erred and [[feedback-loop]] systems that remediate specific [[student-modeling|knowledge states]]. Poor diagnosis produces the inverse: instruction aimed at the wrong concepts. This is why [[psychometrically-aware-ai]] emphasizes diagnostic validity alongside prediction accuracy. ### Connections Cognitive diagnosis connects to [[knowledge-tracing]], [[student-modeling]], [[educational-measurement]], and [[assessment]]. Its insights feed [[intelligent-tutoring]] and [[adaptive-learning]], and LLM-era work links it to misconception identification in [[ai-tutoring]]. ## Connected Concepts - [[knowledge-tracing]] - [[student-modeling]] - [[educational-measurement]] - [[item-response-theory]] - [[assessment]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[personalized-learning]] - [[automated-assessment]] - [[learning-analytics]] ## Connected Articles - [[llm-cognitive-diagnosis-handwritten-math]] — Benchmarking LLMs for Diagnosing Cognitive Skills from Handwritten Math - [[correct-answer-trap-misconceptions]] — The Correct Answer Trap - [[llm-misconception-difficulty-easy-trap]] — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty - [[student-math-competence-clustering]] — Clustering for Modelling Student Mathematical Competence - [[cognitive-agent-compilation]] — Cognitive Agent Compilation for Explicit Problem Solver Modeling --- ## [Cognitive Load Theory](https://edtechdev.github.io/aied/concepts/cognitive-load-theory/) > **Cognitive Load Theory** is a central concept in AI in education research, connected to 7 articles in this wiki. ## Connected Concepts - [[cognitive-offloading]] - [[scaffolding]] - [[over-reliance]] - [[zone-of-proximal-development]] - [[metacognition]] - [[prompt-engineering]] - [[intelligent-tutoring]] - [[teacher-role]] - [[privacy]] - [[critical-thinking]] - [[pedagogical-safety]] - [[student-experience]] ## Connected Articles - [[favero-critical-ai-tutors-empower-enslave-2025]] - [[misiejuk-cognitive-offloading-prompting-2026]] - [[socraticode-k12-programming-tutor]] - [[stanford-evidence-base-ai-k12-2026]] - [[tutoring-specific-vs-general-ai]] --- ## [Cognitive Offloading](https://edtechdev.github.io/aied/concepts/cognitive-offloading/) > **Cognitive offloading** — the use of external tools (including AI) to reduce internal cognitive demand, shifting mental work from the learner to the system. In AI in education, cognitive offloading is the central mechanism through which AI tools can either support or undermine learning: appropriate offloading frees cognitive resources for higher-order thinking, while excessive offloading bypasses the processing required for durable learning. Cognitive offloading is not inherently harmful — humans have always used external tools (notebooks, calculators, search engines) to reduce cognitive load. What makes AI-mediated offloading different is its comprehensiveness: LLMs can generate complete solutions, explanations, and analyses, potentially eliminating the need for the very cognitive processes that produce learning. ### How cognitive offloading manifests in AIED research The wiki's articles document cognitive offloading across multiple dimensions: - **Prompt patterns as offloading traces:** [[misiejuk-cognitive-offloading-prompting-2026|Misiejuk et al. (2026)]] use Co-Occurrence Network Analysis to show that reactive prompts (disagreement without domain context) indicate higher offloading, while context-rich prompting with integrated instruction reflects engaged cognition. The *how* of AI use — not just whether it's used — determines the degree of offloading. - **The speedup illusion:** [[cognitive-offloading-speedup-illusion|Research on the speedup illusion]] demonstrates that AI-assisted work *feels* faster and easier, creating a misleading impression of productivity that masks reduced learning. Students conflate task completion speed with learning, a metacognitive blind spot. - **Learning losses from unguided AI:** [[generative-ai-guardrails-harm-learning|High school math RCTs]] show that GenAI without guardrails produces worse learning outcomes than traditional instruction. [[generative-ai-reduced-study-time-math|Reduced study time]] correlates with reduced learning — students complete tasks faster but retain less. - **Critical engagement vs. offloading:** [[favero-critical-ai-tutors-empower-enslave-2025|Favero et al.]] frame AI tutors as either empowering (supporting active cognition) or enslaving (enabling passive offloading), connecting to [[over-reliance]] and [[critical-thinking]] research. - **Metacognitive awareness:** [[metacognitive-awareness-experiential-vs-instructional|Studies on metacognitive awareness]] examine whether students recognize when they're offloading versus learning — and whether instructional interventions can improve this calibration. - **Fatigue and cognitive burden:** [[ai-fatigue-academic-contexts|AI fatigue research]] documents how constant AI interaction creates its own cognitive burden, a paradox where offloading one task increases cognitive load from managing AI outputs. ### The CLT framework [[cognitive-load-theory|Cognitive Load Theory]] provides the theoretical foundation: intrinsic load (task complexity), extraneous load (presentation friction), and germane load (schema-building effort). Well-designed AI should reduce extraneous load while preserving germane processing. Poorly integrated AI reduces all three, leaving students with completed tasks and empty learning. ### Connections to related concepts Cognitive offloading connects fundamentally to [[over-reliance]] (the behavioral manifestation of excessive offloading), [[ai-literacy]] (the metacognitive skill of knowing when to offload), [[scaffolding]] (structured support that reduces load without eliminating cognitive demand), and [[prompt-engineering]] (the primary mechanism through which offloading is enacted in LLM interactions). It also intersects with [[metacognition]] and [[self-regulated-learning]] — effective learners calibrate their offloading decisions. ## Connected Concepts - [[over-reliance]] - [[ai-literacy]] - [[metacognition]] - [[student-experience]] - [[self-regulated-learning]] - [[scaffolding]] - [[formative-assessment]] - [[generative-ai]] - [[academic-integrity]] - [[teacher-role]] - [[critical-thinking]] - [[regulation]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[ai-fatigue-academic-contexts]] - [[ai-making-us-stupid]] - [[buggy-genai-code-student-responses]] - [[code-review-genai-cs1]] - [[cognitive-offloading-speedup-illusion]] - [[critical-thinking-genai-scaffolding]] - [[favero-critical-ai-tutors-empower-enslave-2025]] - [[genai-availability-grades-satisfaction]] - [[generative-ai-guardrails-harm-learning]] - [[generative-ai-reduced-study-time-math]] - [[llm-cognitive-diagnosis-handwritten-math]] - [[metacognitive-awareness-experiential-vs-instructional]] - [[misiejuk-cognitive-offloading-prompting-2026]] - [[tzirides-thinking-through-ai-2025]] - [[veriforge-narrative-drafting-scaffolding-2026]] --- ## [Collaborative Learning](https://edtechdev.github.io/aied/concepts/collaborative-learning/) > **Collaborative Learning** — instructional approaches where students work together to solve problems, complete tasks, or construct knowledge, supported or mediated by AI tools. In AI in education, collaborative learning research spans AI as a collaboration partner, AI as a mediator of human collaboration, and the design of collaborative AI tutoring systems. Collaborative learning is grounded in sociocultural theories of learning that position knowledge construction as fundamentally social. AI introduces new dynamics: AI can serve as a peer, a facilitator, or a participant in collaborative processes. The articles in this wiki explore how AI-mediated collaboration affects learning outcomes, epistemic engagement, and equity — and how collaborative structures must be designed to accommodate diverse learners. **AI as collaborative partner** explores AI's role in group learning. **[[polished-artifacts-fragile-engagement-2026|Kimmerle]]** conceptualizes the risk of reduced epistemic effort when learners use AI to produce polished knowledge artifacts, advocating for AI structured as an argumentative partner that preserves cognitive conflict. **[[epistemic-emotions-collaborative-problem-solving]]** examines how emotions shape collaborative problem-solving with AI. **[[hingle-collaborative-ai-literacy-2025]]** explores collaborative approaches to AI literacy development. **AI-mediated peer collaboration** examines how AI scaffolds human-to-human collaboration. **[[collaborative-ai-tutoring]]** and **[[agent-voice-accents-k12-group-learning]]** explore how AI agent characteristics affect group dynamics. **[[ai-agents-peer-learning-discourse]]** documents how AI agents teaching each other produce discourse patterns resembling human peer learning. **Neurodivergent perspectives on collaboration** reveal critical design requirements. **[[neurodivergent-computing-students|Zastudil et al.]]** found that neurodivergent students need structured assignments, small consistent teams with explicitly defined roles, and predictable interaction patterns — requirements that AI collaboration tools must accommodate. This connects collaborative learning to [[accessible-learning]] and [[neurodiversity]]. **Teacher-AI collaboration** examines how teachers and AI work together. **[[teacher-student-agency-orchestration]]** and **[[teacher-ai-teaming-five-levels]]** explore frameworks for human-AI collaborative teaching, connecting to [[teacher-role]] and [[human-in-the-loop-ai]]. ## Connected Concepts - [[collaborative-learning]] - [[scaffolding]] - [[teacher-role]] - [[human-in-the-loop-ai]] - [[student-experience]] - [[equity]] - [[equity-in-ai-education]] - [[ai-literacy]] - [[k-12]] - [[higher-ed]] - [[cs-education]] - [[accessible-learning]] - [[neurodiversity]] ## Connected Articles - [[polished-artifacts-fragile-engagement-2026]] - [[epistemic-emotions-collaborative-problem-solving]] - [[hingle-collaborative-ai-literacy-2025]] - [[neurodivergent-computing-students]] - [[golrang-propact-pair-programming-2026]] - [[teacher-student-agency-orchestration]] - [[collaborative-ai-tutoring]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] --- ## [Computational Thinking](https://edtechdev.github.io/aied/concepts/computational-thinking/) > **Computational thinking** — a problem-solving approach involving decomposition, pattern recognition, abstraction, and algorithmic design. In AI education, computational thinking is both a prerequisite for understanding AI systems and a skill that AI tools can help develop. ### CT and AI education - **CT as AI literacy foundation:** Understanding how AI systems decompose problems and apply algorithms is foundational to [[ai-literacy]]. [[computational-thinking-ai-agent-creation|Agent creation studies]] show how building AI agents develops CT skills. - **Programming education:** [[cs-education]] and [[prompt-problems-nl-programming-mistakes|prompting studies]] connect CT to programming instruction. - **K-12 integration:** [[k-12-ai-education]] and [[stem-education]] curricula increasingly embed CT as a core competency. ### Connections Computational thinking connects to [[cs-education]], [[stem-education]], [[ai-literacy]], [[k-12]], [[active-learning|project-based learning]], and [[prompt-engineering]]. It bridges general problem-solving skills and specific AI competencies. ## Connected Concepts - [[cs-education]] - [[stem-education]] - [[ai-literacy]] - [[k-12]] - [[prompt-engineering]] - [[adaptive-learning]] - [[llm]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[computational-thinking-ai-agent-creation]] - [[reshaping-cs-education-genai]] --- ## [Confidence Aware AI Assessment](https://edtechdev.github.io/aied/concepts/confidence-aware-ai-assessment/) > **Confidence-aware AI assessment** — models that report uncertainty alongside scores — is examined in [[confidence-aware-student-drawing-assessment]], [[cong-confidence-asag-2026]], and [[llm-psychometric-calibration-cdp]]: calibrated confidence improves trust and enables appropriate delegation (trust-calibration, [[ai-ed-evaluation]]). ## Connected Concepts - [[adaptive-prompt-routing]] - [[human-in-the-loop-ai]] - [[rag]] - [[automated-grading]] ## Connected Articles - [[code-anchor-multi-view-visualization]] - [[confidence-aware-student-drawing-assessment]] - [[cong-confidence-asag-2026]] - [[learning-to-prompt-adaptive-tutoring]] - [[psyscore-essay-scoring-zpd-feedback]] - [[llm-difficulty-calibration-programming-exams-2026]] --- ## [Constructivist](https://edtechdev.github.io/aied/concepts/constructivist/) > **Constructivist** learning theory — knowledge built through active experience — underpins wiki analyses of AI as [[genai-mindtool-generative-learning]] and [[icap-cognitive-engagement-llm-agents]]: AI tools support construction only when learners generate, not merely consume ([[active-learning]], educational-theory). ## Connected Concepts - [[active-learning]] ## Connected Articles - [[genai-architecture-education]] --- ## [Creativity](https://edtechdev.github.io/aied/concepts/creativity/) > **Creativity** — the capacity to generate novel and valuable ideas, solutions, or artifacts. In the AI era, creativity is a central educational stake: generative AI can both amplify creative work (as a divergent-thinking partner) and undermine it (by homogenizing output and replacing the generative process). Creativity spans the divergent-thinking end of the cognitive spectrum — generating multiple possibilities — in contrast to convergent thinking, which arrives at a single correct solution. AI systems are especially relevant to creativity because they are statistical generators: they can propose many options (supporting ideation) but also tend toward the average, producing the idea-level homogenization documented when many students rely on the same model. ### Creativity and generative AI - **Amplification:** AI can act as a divergent-thinking partner — brainstorming alternatives, generating counterarguments, and offering perspectives the learner might not consider. Role-specialized multi-agent configurations can restore ideational diversity. - **Homogenization risk:** single-model assistance can reduce the diversity of ideas across students, so the same model produces convergent outputs. This is a direct threat to creativity in [[writing-education]] and design education. - **Protecting creative agency:** keeping the learner's generative process in the loop — draft-first routines, requiring original synthesis, and using AI to challenge rather than replace — preserves the creative work that produces durable learning. ### Connections Creativity connects to [[critical-thinking]] and to [[constructivist]] learning. It is protected by the same [[reducing-ai-misuse]] scaffolds that preserve learning, and by [[authentic-assessment]] designs that reward original reasoning over polished products. ## Connected Concepts - [[critical-thinking]] - [[constructivist]] - [[writing-education]] - [[student-experience]] - [[generative-ai]] - [[reducing-ai-misuse]] - [[authentic-assessment]] - [[collaborative-learning]] ## Connected Articles - [[chatgpt-critical-creative-thinking-review]] — ChatGPT and Critical and Creative Thinking: Systematic Review - [[ai-collaborative-learning-skills-impacts]] — AI and Collaborative Learning: Impacts on Creativity - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Robot-LLM Integration and Creative Writing - [[multi-agent-llm-social-learning]] — Beyond the AI Tutor: Social Learning with LLM Agents - [[genai-mindtool-generative-learning]] — GenAI as a Mindtool for Generative Learning --- ## [Critical Thinking](https://edtechdev.github.io/aied/concepts/critical-thinking/) > **Critical thinking** — the ability to analyze, evaluate, and synthesize information — is both a skill that AI tools can help develop and a competency that students must apply when using AI. In AI in education research, critical thinking appears in two interrelated forms: as a learning objective (teaching students to think critically) and as a safeguard against uncritical AI reliance. Critical thinking is central to [[ai-literacy]] — students who cannot critically evaluate AI outputs are vulnerable to [[over-reliance]], hallucinated information, and biased recommendations. Research on [[cognitive-offloading]] shows that easy access to AI answers can displace critical engagement, while [[socratic-method|Socratic approaches]] that withhold direct answers preserve the cognitive effort necessary for deeper thinking. ### Critical thinking in AI education research The wiki's articles explore critical thinking through design-based and empirical lenses. [[ai-agents-constructive-conflict-design-education-2026|Adversarial AI agents]] enact constructive conflict to prompt reconsideration in novice designers — a Socratic variant that forces critical re-evaluation. [[genai-can-harm-teaching-rct-2026|RCT research on GenAI in teaching]] raises the question of whether AI tools that optimize for surface-level outcomes may inadvertently suppress the critical thinking that leads to deeper learning. [[chatgpt-critical-creative-thinking-review|Reviews of ChatGPT's impact on thinking]] document mixed findings: AI can scaffold critical analysis when used deliberately (e.g., asking students to critique AI-generated arguments), but it can also short-circuit thinking when used as an answer engine. This tension connects to [[ai-literacy-assessment-misalignment]] research showing that self-reported AI competence far exceeds actual critical evaluation ability. ### Connections to other concepts Critical thinking intersects with [[scaffolding]] (designing AI support that maintains cognitive demand), [[prompt-engineering]] (formulating questions that elicit critical analysis), and [[over-reliance]] (knowing when to trust and when to question AI). It is foundational to [[academic-integrity]] and serves as a key dimension of [[ai-literacy]] frameworks across both [[k-12]] and [[higher-ed]] contexts. ## Connected Concepts - [[student-experience]] - [[metacognition]] - [[faculty-development]] - [[cognitive-offloading]] - [[intelligent-tutoring]] - [[over-reliance]] - [[teacher-role]] - [[privacy]] - [[pedagogical-safety]] - [[regulation]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[critical-media-literacy-education-2026]] --- ## [CS Education and AI](https://edtechdev.github.io/aied/concepts/cs-education/) > **CS Education** — computer science education is the most-researched STEM subfield in the wiki, benefiting from natural alignment between AI tools and programming tasks. Code generation, debugging assistance, and automated code review are its primary AI applications. ### AI in CS education - **Code generation and completion:** [[code-review-genai-cs1|CS1 code review]], [[dura-llm-cs2|DURA for CS2]], and [[prompt-problems-nl-programming-mistakes|NL programming mistakes]] examine how students use AI for code generation and what they learn from it. - **Debugging support:** [[debugtracker-classroom-debugging|Debugging tools]] and [[chat-debugging-human-ai-collaboration-circuits|human-AI debugging collaboration]] leverage AI for error identification and repair. - **Automated assessment:** [[automated-grading-linux-bash-examinations-large-language-models|Linux Bash grading]] and [[llm-intervention-design-cs-review|LLM intervention review]] evaluate automated code assessment. - **Curriculum design:** [[ase-26-agentic-software-engineering-curriculum|Agentic SE curricula]] and [[reshaping-cs-education-genai|reshaping CS education]] redesign computing programs for the AI era. ### Unique affordances CS education's unique position — students learn to build the very tools they use — creates both opportunities (meta-cognitive awareness of AI limitations) and risks (over-reliance on AI-generated code). [[code-review-genai-cs1|Code review interviews]] and [[critical-engagement-code-completion|critical engagement studies]] address this duality. ### Connections CS education connects to [[computational-thinking]], [[stem-education]], [[automated-grading]], [[prompt-engineering]], [[higher-ed]], and [[k-12]]. It is the domain where AIED tools are both used and built. ## Connected Concepts - [[computational-thinking]] - [[stem-education]] - [[automated-grading]] - [[prompt-engineering]] - [[higher-ed]] - [[k-12]] - [[llm]] - [[generative-ai]] - [[ai-tutoring]] - [[over-reliance]] - [[code-review-genai-cs1]] ## Connected Articles - [[code-review-genai-cs1]] - [[dura-llm-cs2]] - [[debugtracker-classroom-debugging]] - [[llm-intervention-design-cs-review]] - [[ase-26-agentic-software-engineering-curriculum]] - [[reshaping-cs-education-genai]] - [[critical-engagement-code-completion]] --- ## [Culturally Relevant Pedagogy](https://edtechdev.github.io/aied/concepts/culturally-relevant-pedagogy/) Culturally Relevant Pedagogy (CRP), introduced by Gloria Ladson-Billings (1995), centers marginalized students' cultural references in curriculum design. Wang et al. (2025) demonstrate that **LLMs can support K-12 teachers** in implementing CRP: 78% of teachers found AI suggestions helpful for diversifying curriculum materials. ### Three Pillars of CRP 1. **Academic Success:** Maintaining rigorous standards while honoring cultural identity 2. **Cultural Competence:** Developing critical consciousness about culture and power 3. **Sociopolitical Consciousness:** Empowering students to challenge inequitable systems ### AI-Supported CRP Implementation - **Material Diversification:** LLMs suggest texts/authors from underrepresented groups - **Perspective Expansion:** AI generates discussion prompts centering marginalized voices - **Curriculum Auditing:** Automated analysis of representation gaps in existing materials ### Critical Considerations While AI tools can scaffold CRP implementation, they risk **reinforcing dominant narratives** if not carefully prompted. Teachers must retain agency in evaluating AI suggestions for cultural authenticity and community alignment. ## References Wang, J., Xiao, R., et al. (2025). *LLMs to Support K-12 Teachers in Culturally Relevant Pedagogy*. arXiv:2505.08083. Ladson-Billings, G. (1995). *Toward a Theory of Culturally Relevant Pedagogy*. American Educational Research Journal. ## Source - https://arxiv.org/abs/2505.08083 ## Connected Concepts - [[personalized-learning]] - [[edtech-platform]] - [[student-experience]] - [[human-in-the-loop-ai]] - [[equity-in-ai-education]] - [[higher-ed]] - [[generative-ai]] - [[academic-integrity]] - [[automated-question-generation]] ## Connected Articles - [[international-students-conversational-ai-adaptation]] - [[taklif-ai-interest-based-personalized-assignments]] --- ## [Curriculum Design](https://edtechdev.github.io/aied/concepts/curriculum-design/) > **Curriculum Design** — the process of planning and structuring what is taught across courses, programs, and institutions, including learning objectives, content sequencing, assessment strategies, and skill progression. In the AI era, curriculum design must balance foundational knowledge with emerging AI competencies, determining not just what students learn but how they learn to work with and critically evaluate AI tools. Curriculum design addresses the *what* of education at the program level, complementing [[instructional-design]] which addresses the *how* at the course level. The articles in this wiki explore how AI is reshaping curricula across disciplines — from software engineering to architecture to green education — and how educators are designing curricula that embed AI literacy without sacrificing disciplinary fundamentals. ### Key research themes **Redesigning curricula for the AI era** is the central challenge. **[[reshaping-cs-education-genai|Lee et al.]]** synthesized findings from international workshops on reshaping undergraduate CS education, arguing that as GenAI automates implementation-level programming, curricula must shift toward system design, abstraction, and critical evaluation — while de-emphasizing low-level implementation details. **[[ase-26-agentic-software-engineering-curriculum|Gorsky]]** formalized Agentic Software Engineering as a distinct discipline with a 21-module curriculum focused on the "evolution of intent" and practitioner discipline required to manage AI agents. Both connect to [[ai-literacy]] and [[scaffolding]]. **Curriculum mapping and analysis** uses AI to understand existing curricula. **[[ai-assisted-se-curriculum-syllabus-analysis-2026|Geng et al.]]** analyzed 23 syllabi from AI-assisted software engineering courses, identifying common themes — prompt engineering, code review with AI, ethical considerations — and deriving design guidance that emphasizes balancing tool fluency with foundational knowledge. **[[coursegraph-cs-course-comparison-2026|CourseGraph]]** applies computational methods to compare CS course structures across institutions. **AI literacy integration** embeds AI competencies across disciplines. **[[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl|SAIL]]** provides a scaffolded AI literacy framework applicable across all ages and educational stages, addressing second- and third-level digital divides. **[[tracing-genai-literacy-interaction-patterns]]** examines how AI literacy develops through interaction patterns. **[[hingle-collaborative-ai-literacy-2025]]** explores collaborative approaches to AI literacy curriculum development, connecting to [[collaborative-learning]]. **Domain-specific curriculum innovation** applies curriculum design to specific fields. **[[genai-architecture-education]]** explores how generative AI reshapes architectural design pedagogy. **[[talebzadeh-ai-green-education-2026]]** examines AI integration in green education curricula. **[[connected-ai-lesson-planning-vietnam]]** and **[[llm-cultural-relevance-k12]]** address culturally responsive curriculum design. **Institutional frameworks** address curriculum change at scale. **[[finkelstein-principled-ai-education-2025]]** and **[[principled-ai-education]]** provide principles for integrating AI across educational programs. **[[ai-adoption-training-public-sector]]** examines barriers to AI curriculum adoption in public sector education. ### Connections to related concepts Curriculum design connects directly to [[instructional-design]] — curriculum defines what, instruction defines how. It connects to [[ai-literacy]] because embedding AI competencies is a primary curriculum challenge, to [[teacher-role]] and [[faculty-development]] because curriculum change requires educator preparation, and to [[scaffolding]] because well-designed curricula scaffold skill development across courses and years. The [[higher-ed]] and [[k-12]] connections reflect curriculum design's relevance across educational levels. ## Connected Concepts - [[instructional-design]] - [[ai-literacy]] - [[scaffolding]] - [[faculty-development]] - [[teacher-role]] - [[higher-ed]] - [[k-12]] - [[stem-education]] - [[cs-education]] - [[generative-ai]] - [[agentic-ai]] - [[metacognition]] - [[prompt-engineering]] - [[collaborative-learning]] ## Connected Articles - [[ai-interior-design-malaysia-2026]] - [[critical-media-literacy-education-2026]] - [[ai-generated-interactive-fiction-education-2026]] - [[reshaping-cs-education-genai]] - [[ase-26-agentic-software-engineering-curriculum]] - [[ai-assisted-se-curriculum-syllabus-analysis-2026]] - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] - [[curriculum-as-code-instructional-design-2026]] - [[tracing-genai-literacy-interaction-patterns]] - [[finkelstein-principled-ai-education-2025]] - [[hingle-collaborative-ai-literacy-2025]] - [[learnity-graphs-lifelong-learning-framework-2026]] --- ## [Design Thinking](https://edtechdev.github.io/aied/concepts/design-thinking/) > **Design Thinking** — a key concept in AI in education research. Explored across 1 articles in this wiki. Research themes include: - Investigates adversarial AI design agents that enact constructive conflict to prompt reconsideration in novice designers. Between-subjects experiment (N=48) comparing adversarial vs. cooperative AI agent roles. Adversarial agent condition produced si ## Connected Concepts - [[scaffolding]] - [[student-experience]] - [[socratic-method]] - [[agentic-ai]] ## Connected Articles - [[agency-gap-ai-writing]] — The agency gap in AI-supported writing: how reactive and proactive agent designs shape multimodal reasoning - [[absent-cognitive-baseline-2026]] — The Absent Cognitive Baseline: Theorizing a Structural Gap in AI-Native College Students' Academic Self-Assessment - [[genai-architectural-design-studios]] — Development and applications of Generative AI in architectural design studios - [[hybrid-intelligence-feedback-systems-in-design-thinking-development-stage-specif]] — Hybrid intelligence feedback systems in design thinking development: Stage-specific insights on pedagogical effects and characteristics of generative AI and instructors - [[ai-agents-constructive-conflict-design-education-2026]] — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers - [[learner-ai-interaction-patterns-oop]] — Patterns of Learner-AI Interaction and Academic Performance in an Object-Oriented Programming Course --- ## [Desirable Difficulties](https://edtechdev.github.io/aied/concepts/desirable-difficulties/) > **Desirable difficulties** — the finding that harder retrieval conditions improve long-term learning (Bjork) — is the theoretical counterweight to AI smoothing: [[agentic-ai-pedagogical-best-practice-2026]] calls for intentional friction, and [[generative-ai-reduced-study-time-math]] documents the cost of removing productive struggle. In the AI era, the principle warns that tools which eliminate effortful processing (spacing, retrieval, generation) may raise immediate performance while undercutting durable learning. Desirable difficulties connect to the [[over-reliance]] and [[ai-misuse-learning-harm]] findings: an AI that removes desirable difficulty produces the performance–learning gap seen across the wiki's evidence base. It is also the theoretical grounding for [[reducing-ai-misuse]] scaffolds such as draft-first routines and hint-not-answer tutoring, which deliberately preserve productive struggle. ## Connected Concepts - [[cognitive-load-theory]] - [[self-regulated-learning]] - [[metacognition]] - [[transfer-of-learning]] - [[scaffolding]] - [[learning-gains]] - [[over-reliance]] - [[ai-misuse-learning-harm]] - [[reducing-ai-misuse]] ## Connected Articles - [[agentic-ai-pedagogical-best-practice-2026]] - [[finkelstein-principled-ai-education-2025]] - [[stanford-evidence-base-ai-k12-2026]] - [[structured-llm-feedback-programming]] --- ## [Digital Divide](https://edtechdev.github.io/aied/concepts/digital-divide/) > **Digital divide** — the unequal distribution of access to, skills for, and benefits from digital (and increasingly AI) technologies across individuals, communities, and nations. In AI education, the digital divide is a central equity concern: generative AI is rapidly reshaping learning, and the gap between those who can use it effectively and critically and those who cannot threatens to deepen existing educational inequalities. The digital divide is commonly understood as operating across **three levels** (van Deursen & van Dijk, 2014): the *first-level* divide concerns access to technologies and infrastructure (connectivity, devices, supportive environments); the *second-level* divide concerns skills and competencies (the uneven capacity to use tools effectively and meaningfully); and the *third-level* divide concerns outcomes and benefits (who actually benefits from technology use, with AI potentially exacerbating social, cultural, and economic disparities). Framing AI literacy through this lens makes clear that equity requires more than closing the device-and-infrastructure gap — it requires building the skills to use AI effectively and critically so that its benefits are distributed fairly rather than reinforcing existing inequalities. ### How the digital divide appears in the research - **AI literacy as a mechanism for equity:** [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl|The SAIL framework]] was explicitly designed to address second- and third-level divides, providing a scaffolded, age-agnostic pathway for equitable AI literacy across all stages of education, grounded in the argument that AI literacy is inseparable from equity and participation. - **Policy and infrastructure:** [[oecd-digital-education-outlook-2026|OECD Digital Education Outlook 2026]] situates the digital divide within national education policy, examining how access to digital and AI technologies varies and what systems can do to close gaps. - **Responsible-use and prompting literacy:** [[aaai2026-prompting-literacy-k12|K-12 prompting-literacy research]] addresses the second-level divide by teaching students the skills to use AI chatbots responsibly, recognizing that access alone does not confer the ability to use AI well. - **Representation and structural silence:** [[structural-silence-underrepresented-language-ai-2026|Research on underrepresented languages]] highlights how the digital divide extends to *which* communities, languages, and perspectives are represented in and served by AI systems — a cultural and epistemic dimension of inequality. ### AI deepens (and can close) divides AI adds new layers to the equity implications of technology. Algorithmic bias can disproportionately impact learners from marginalized communities, and AI literacy — the ability to understand, critically evaluate, and mitigate AI's biases and risks — is itself a key factor in whether AI widens or narrows gaps. Research shows educators with higher AI literacy are more effective at identifying and mitigating biased outcomes. The digital divide in the AI era is therefore not simply a technical provision problem but a question of justice and participation: who can access AI, who can use it critically, and who benefits. ### Connections to related concepts The digital divide is a core concern of [[equity]] and [[equity-in-ai-education]] research, closely tied to [[ai-literacy]] (which is positioned as a central mechanism for addressing structural barriers), and to [[ethics]] and [[bias-mitigation]] (since algorithmic bias disproportionately affects marginalized groups). It connects to [[ai-education]] and [[higher-ed]] as the settings where access and capability gaps manifest, and relates to [[student-experience]] as it shapes who can participate meaningfully in AI-shaped learning. ## Connected Concepts - [[equity]] - [[equity-in-ai-education]] - [[ai-literacy]] - [[ethics]] - [[bias-mitigation]] - [[ai-education]] - [[higher-ed]] - [[student-experience]] ## Connected Articles - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] — The Scaffolded AI literacy (SAIL) framework - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[aaai2026-prompting-literacy-k12]] — Teaching Responsible Use of AI Chatbots to K-12 Students - [[structural-silence-underrepresented-language-ai-2026]] — Structural Silence and Underrepresented Languages - [[sec-ai-literacy-narrative-review-2026]] — Social-Emotional Competence in AI Literacy --- ## [DOT Framework Survey: Practitioner Beliefs and Behaviors in AI-Enhanced Education](https://edtechdev.github.io/aied/concepts/dot-framework-survey/) A 2026 cross-sectional survey (n=72) by Gibson, Azukas, and Knezek examined how higher education practitioners think about and use AI in teaching, grounded in the **DOT Framework** — a synthesis of [[design-thinking]] and open-systems-theory. ## Core Findings ### Three-Factor Belief Structure Exploratory factor analysis of 19 Likert-scale items yielded a three-factor structure with strong reliability (overall α = .90): 1. **AI Functional Capabilities** (α = .91) — perceived utility of AI tools for teaching tasks 2. **Oversight and Governance** (α = .79) — need for human supervision, policy frameworks, and critical evaluation 3. **Instructor Collaboration and Planning** (α = .85) — value of teamwork in AI integration design This factor structure provides a psychometric anchor for understanding how educators conceptualize AI's role — not as a monolithic "good or bad" judgment but as differentiated beliefs spanning capability, governance, and collaborative practice. ### Practice vs. Theory Gap Practitioners reported frequent use of **iterative prompting** and **content generation**, but far less engagement with **needs assessment** and **feedback loops** — the front and back ends of a full design cycle. This gap between design-oriented theory and current implementation is the study's central diagnostic finding. It echoes broader patterns in [[ai-literacy]] where educators adopt AI for productivity but not yet for systematic instructional redesign. ### AI as "Fallible Intern" The study operationalizes AI as a **co-intelligent collaborator** following four tenets: (1) AI is fallible — always review, (2) AI supports not replaces higher-order thinking, (3) instructors should actively experiment, (4) instructor maintains agency. This framing aligns with ai-co-intelligence and [[human-ai-collaboration]] paradigms where the human remains the epistemic authority. ### Institutional Barriers Widespread lack of policy, training, and infrastructure was reported — consistent with findings across institutional-ai-readiness and [[ai-governance-education]]. Without institutional scaffolding, even motivated practitioners remain in ad-hoc, fragmented adoption patterns. ## The DOT Framework The DOT Framework integrates **Design Thinking stages** (Empathize → Define → Ideate → Prototype → Test) with **Open Systems concepts** (Environment, Input, Process, Structure, Output, Feedback) at both classroom (micro) and institutional (macro) levels. Key intersections: - **Empathize × Environment**: Ground instructional design in contextual understanding - **Test × Feedback**: Enable recursive improvement at both levels This study provides the *first empirical evidence* supporting DOT as a descriptive model — practitioners' beliefs and behaviors partially map to its structure, but the gaps (needs assessment, feedback) reveal where the model is aspirational rather than descriptive of current practice. ## Implications for [[ai-education]] - **Measurement**: The 19-item belief scale offers a validated starting point for [[ai-ed-evaluation]] studies - **Professional Development**: Training should target the full design cycle, not just tool use — especially needs assessment and feedback integration - **Policy**: Institutional AI strategies need to address governance concerns while building on practitioners' existing favorable dispositions - **Future Research**: Confirmatory factor analysis needed; outcome-based studies linking DOT-aligned practices to instructional quality ## Connected Concepts - [[dot-framework-survey]] - [[ai-literacy]] - [[design-thinking]] - [[ai-ed-evaluation]] - [[ai-education]] - [[human-in-the-loop-ai]] - [[ai-governance-education]] - [[human-ai-collaboration]] ## Connected Articles - [[institutional-governance-ai-universities]] — Policy Fragmentation or Institutional Alignment? Institutional Governance of AI in Universities and Business Schools - [[white-wu-robotics-ai-education-2026]] — Robotics and Artificial Intelligence in Education: Transformations, Challenges, and Future Directions - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[agentic-ai-education-scoping-review]] — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm - [[ai-uk-higher-education-policy-2026]] — Artificial Intelligence in UK Higher Educational Policy and Institutional Decision Making - [[chatgpt-critical-creative-thinking-review]] — ChatGPT Critical and Creative Thinking: Systematic Review --- ## [Dual-Process Theory](https://edtechdev.github.io/aied/concepts/dual-process-theory/) > **Dual-process theory** — the account of cognition as operating through two interacting systems: a fast, automatic, intuitive System 1 and a slower, effortful, analytical System 2. In education, dual-process theory frames when learners rely on quick heuristics versus deliberate reasoning, and why they sometimes take cognitive shortcuts. Dual-process theory has strong explanatory value for AI in education. Much of the risk of [[over-reliance]] and [[cognitive-offloading]] can be read as learners defaulting to System 1 (accept the fluent AI answer) instead of engaging System 2 (verify, analyze, reason). Conversely, [[desirable-difficulties]] and desirable difficulties require System 2 engagement. ### Relevance to AI in education - **Explaining over-reliance:** an AI's fluent, plausible output invites System 1 acceptance; unverified trust is a heuristic shortcut. This links to [[cognitive-offloading]] and the "speedup illusion" in human-AI interaction. - **Designing for System 2:** scaffolds that force verification, articulation, and reasoning (draft-first, hint-not-answer, explanation requirements) push learners toward deliberate processing and protect learning. - **Measuring cognition:** dual-process distinctions inform how we interpret learner behavior and performance, and caution against mistaking fluent output (System 1) for understanding (System 2) — a theme in [[educational-measurement]] and [[psychometrically-aware-ai]]. ### Connections Dual-process theory connects to [[cognitive-load-theory]], [[metacognition]], and [[cognitive-offloading]]. It underpins the rationale for [[reducing-ai-misuse]] interventions and for assessment designs that distinguish surface fluency from deep understanding. ## Connected Concepts - [[cognitive-load-theory]] - [[metacognition]] - [[cognitive-offloading]] - [[over-reliance]] - [[educational-measurement]] - [[reducing-ai-misuse]] - [[psychometrically-aware-ai]] ## Connected Articles - [[cognitive-offloading-speedup-illusion]] — Cognitive Offloading and the Speedup Illusion - [[ai-making-us-stupid]] — Is AI Making Us Stupid? - [[efficiency-gain-illusion-ai-overreliance]] — The Efficiency-Gain Illusion - [[llm-reasoning-traces-metacognition]] — LLM Reasoning Traces and Metacognition - [[cognitive-commons-ai-expertise-regeneration]] — The Tragedy of the Cognitive Commons --- ## [Edtech Platform](https://edtechdev.github.io/aied/concepts/edtech-platform/) > **Edtech Platform** is a central concept in AI in education research, connected to 15 articles in this wiki. Research in this wiki explores edtech platform through multiple lenses: - **Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition:** Pilot study on privacy-aware computer vision for classroom incident detection. ## Connected Concepts - [[personalized-learning]] - [[higher-ed]] - [[adaptive-learning]] - [[ai-literacy]] - [[student-experience]] - [[teacher-role]] - [[formative-assessment]] - [[stem-education]] - [[automated-assessment]] - [[generative-ai]] - [[llm]] - [[learning-analytics]] ## Connected Articles - [[a4l-analytics-pipeline]] - [[ai-campus-wellbeing-tools]] - [[ai-learning-tools-engineering-education-needs]] - [[aiawe-automated-writing-evaluation]] - [[calibrating-trustworthiness-llm-education-2026]] - [[community-centered-ai-education-adults]] - [[concept-catalyst-k12-teacher]] - [[llm-misconception-difficulty-easy-trap]] - [[llm-psychometric-calibration-cdp]] - [[multi-site-vr-immersive-learning]] - [[privacy-aware-classroom-incident-recognition-2026]] - [[raza-farooq-aied-review-2020-2025]] - [[singh-eduqwen-pedagogical-rl-2026]] - [[student-math-competence-clustering]] - [[taklif-ai-interest-based-personalized-assignments]] --- ## [Educational Measurement](https://edtechdev.github.io/aied/concepts/educational-measurement/) > **Educational measurement** — psychometric theory applied to learning data — runs through the wiki's [[item-response-theory]], [[knowledge-tracing]], and [[assessment-validity]] pages: LLM-era measurement must reconcile classical psychometrics with new AI-generated response streams (psychometrically-aware-ai, [[educational-nlp]]). ## Connected Concepts - [[generative-ai]] - [[socratic-method]] - [[learning-analytics]] - [[intelligent-tutoring]] - [[stem-education]] - [[physics-education]] - [[equity]] - [[personalized-learning]] - [[faculty-development]] - [[equity-in-ai-education]] - [[equity]] - [[language-learning]] ## Connected Articles - [[multimodal-item-parameter-estimation-2026]] - [[ai-scoring-language-bias-physics]] - [[genai-architecture-education]] - [[hashmi-socratic-physics-chatbot-2025]] - [[llm-difficulty-calibration-programming-exams-2026]] --- ## [Educational NLP](https://edtechdev.github.io/aied/concepts/educational-nlp/) > **Educational NLP** applies language technologies to learning: [[llm-item-difficulty-prediction]], [[teaching-feedback-classification-benchmark]], [[llm-sentiment-analysis-education-research]], and [[vocabulary-difficulty-prediction]] show LLMs advancing analysis of student language at scale ([[educational-measurement]], [[educational-nlp]]). ## Connected Concepts - [[intelligent-tutoring]] - [[student-modeling]] - [[knowledge-tracing]] - [[socratic-method]] - [[scaffolding]] - [[adaptive-learning]] - [[pedagogical-llm-training]] - [[metacognition]] - [[rct]] - [[learning-analytics]] - [[educational-policy-ai]] - [[adaptive-learning]] ## Connected Articles - [[ai-generated-interactive-fiction-education-2026]] - [[zerkouk-comprehensive-review-its-2025]] --- ## [Educational AI Policy](https://edtechdev.github.io/aied/concepts/educational-policy-ai/) > **Educational AI policy** — the formal and informal rules governing AI use in educational institutions, from national legislation to classroom guidelines. Policy research in the wiki spans institutional governance, curriculum mandates, and teacher preparation requirements. ### Policy levels - **Institutional policy:** [[genai-policies-higher-ed-computing|Institutional policy analysis]] compares how universities develop AI policies. [[institutional-change-framework-ai|Institutional change frameworks]] provide models for policy development. - **Government policy:** [[state-policy-teacher-ai|State teacher AI policy]] and [[ai-lifelong-learning-policy|lifelong learning policy]] examine regulatory approaches at national and regional levels. - **K-12 policy:** [[brookings-ai-students-report|The Brookings AI students report]] and [[stanford-evidence-base-ai-k12-2026|Stanford evidence reviews]] inform K-12 AI policy. - **Assessment policy:** [[ai-assessment-scale-reform|Assessment reform policies]] and [[authentic-assessment]] frameworks represent policy-level responses to AI-enabled cheating. ### Policy maturity gap The wiki documents that institutional AI policies [[genai-policies-higher-ed-computing|lag behind actual AI use]]. [[faculty-development]] programs, [[teacher-ai-competency]] frameworks, and [[regulation]] all require coherent policy foundations. ### Connections Educational AI policy connects to [[regulation]] (legal framework), [[ai-governance-education]] (institutional implementation), [[faculty-development]] (policy implementation through training), [[equity]] (policy impacts on access), and [[higher-ed]] / [[k-12]] (context-specific policy). ## Connected Concepts - [[regulation]] - [[ai-governance-education]] - [[faculty-development]] - [[equity]] - [[higher-ed]] - [[k-12]] - [[academic-integrity]] - [[ethics]] - [[teacher-ai-competency]] - [[institutional-change-framework-ai]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[ai-acceptance-preservice-science-teachers-2026]] - [[ai-interior-design-malaysia-2026]] - [[critical-media-literacy-education-2026]] - [[genai-policies-higher-ed-computing]] - [[institutional-change-framework-ai]] - [[state-policy-teacher-ai]] - [[brookings-ai-students-report]] - [[ai-assessment-scale-reform]] - [[stanford-evidence-base-ai-k12-2026]] - [[ai-uk-higher-education-policy-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Robots in Education](https://edtechdev.github.io/aied/concepts/educational-robotics/) > **Robots in education (educational robotics)** — the use of physical or simulated robots as tools for teaching and learning. Educational robotics spans a wide spectrum: from programmable kits that teach computational thinking and programming, to socially assistive and humanoid robots that tutor, tell stories, model sign language, or rehearse social skills. It is valued for fostering problem solving, critical thinking, creativity, and STEAM engagement, and for making abstract computing concepts tangible through embodied interaction. The wiki's robotics corpus spans curriculum-integrated programming, LLM-powered conversational tutors, socially assistive storytelling robots, and role-play for social-emotional learning. Educational robotics is a distinct but closely related application of AI in education. Unlike software-only [[intelligent-tutoring|intelligent tutoring]] or [[llm|LLM]] chatbots, robots add an **embodied** and often **social** presence — a physical agent that learners can see, manipulate, and (increasingly) converse with. This embodiment is central to their pedagogical value: it grounds abstract program logic in observable behaviour, and it can support relationship-building and emotional engagement that disembodied systems cannot. The field overlaps with [[human-robot-interaction]] (the study of how people and robots interact) and [[social-robots]] (robots designed to engage people socially). ### How robots are used in education - **Computational thinking and programming:** Programmable robots (e.g., LEGO, block-based platforms) help learners connect code to real outcomes. [[computational-thinking-educational-robotics-secondary-2026|Valls i Pou]] links computational thinking to secondary STEAM curricula, and [[roboblockly-conversational-block-robotics-ct-2026|RoboBlockly Studio]] combines block programming with a conversational AI agent and embodied robot feedback. [[edusim-llm-robotic-simulation-education-2026|EduSim-LLM]] lets beginners control simulated robots with natural language. - **Tutoring and knowledge delivery:** [[knowledge-based-design-generative-social-robots-2026|Knowledge-based design research]] and [[teachy-mini-generative-social-robot-higher-ed-2026|Teachy Mini]] develop LLM-powered generative social robots that tutor higher-education students, addressing risks like misinformation and overreliance. [[task-context-trust-educational-hri-2026|Research on trust]] shows that what a robot does (task context) shapes learner trust more than its appearance. - **Storytelling and engagement:** [[motibo-digital-storytelling-robots-motivation-2026|MotiBo]] and [[robobuddy-llm-social-robots-classroom-2025|RoboBuddy]] use interactive, LLM-powered social robots for storytelling to boost motivation and engagement, while [[icub-humanoid-storytelling-llm-hri-2025|the iCub narrative HRI study]] explores co-creative storytelling between humans and humanoids. - **Social-emotional learning and inclusion:** [[remind-robot-mediated-roleplay-antibullying-2026|REMind]] uses robot-mediated role-play to rehearse anti-bullying bystander intervention, and [[pepper-robot-sign-language-lis-2025|work with the Pepper robot]] explores robot sign-language communication to support Deaf learners. - **Project-based and game-based approaches:** [[bots-blocks-project-based-robotics-education-2026|Bots and Blocks]] presents a project-based robotics course, and [[game-based-gamified-robotics-education-review-2026|a systematic review]] compares game-based learning and gamification in robotics education. ### Embodiment and pedagogy A defining theme is that robots are effective when they support genuine learning goals — not as isolated technical exercises. The value of a robot depends on the pedagogical context: teaching computational thinking ([[computational-thinking]]), supporting [[stem-education|STEAM]], building [[programming-education|programming]] skills, motivating learners ([[motivation]], [[student-engagement|engagement]]), or supporting [[social-emotional-learning]] and [[equity-in-ai-education|inclusion]]. Robotics also connects to [[project-based-learning]], [[game-based-learning]], and [[experiential-learning]]. Key design considerations include preserving learner [[agency]], building [[trust]], supporting [[self-efficacy]], and grounding learning in [[embodied-learning|embodied interaction]]. ## Connected Concepts - [[human-robot-interaction]] - [[social-robots]] - [[computational-thinking]] - [[programming-education]] - [[stem-education]] - [[embodied-learning]] - [[project-based-learning]] - [[game-based-learning]] - [[llm]] - [[motivation]] - [[student-engagement]] - [[social-emotional-learning]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[computational-thinking-educational-robotics-secondary-2026]] — Computational Thinking and Educational Robotics - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM - [[teachy-mini-generative-social-robot-higher-ed-2026]] — Teachy Mini - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo - [[robobuddy-llm-social-robots-classroom-2025]] — RoboBuddy - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks --- ## [Efficacy Study](https://edtechdev.github.io/aied/concepts/efficacy-study/) > **Efficacy Study** — a key concept in AI in education research. Explored across 6 articles in this wiki. Research themes include: - ### 6. Six Dimensions of Agentic Capability (the Review's Analytical Framework) - The review identifies four design elements that distinguish effective interventions: - 2. **Practice Question Engine**: Automatically generates programming exercises at three difficulty levels (beginner, intermediate, advanced), dynamically tailored to student competency. ## Connected Concepts - [[higher-ed]] - [[rct]] - [[design-thinking]] - [[edtech-platform]] - [[pedagogical-llm-training]] - [[multimodal]] - [[efficacy-study]] - [[engagement-metrics]] ## Connected Articles - [[genai-pd-ai-pck-learning-gain-2026]] --- ## [Embodied Learning](https://edtechdev.github.io/aied/concepts/embodied-learning/) > **Embodied learning** — the pedagogical principle that learning is grounded in bodily experience, physical interaction, and the sensory-motor context of the learner. Embodied approaches hold that cognition is not purely abstract but shaped by the body and its interaction with the environment. In AI in education, embodiment is realized through [[educational-robotics|educational robots]] and [[social-robots|social robots]], whose physical presence grounds abstract concepts (such as program logic or social skills) in observable, manipulable behaviour. Embodied learning is closely related to [[active-learning]], [[experiential-learning]], and situated/constructivist theories. The key claim is that a physical, manipulable agent helps learners connect abstract ideas to concrete outcomes — a program that makes a robot move, or a role-play with a physical robot — in ways that pure screen-based interaction may not. Robotics is the clearest embodiment of AI in education, giving learners something to see, touch, and observe. ### How embodied learning appears in the wiki's research - **Grounded programming:** [[roboblockly-conversational-block-robotics-ct-2026|RoboBlockly Studio]] grounds block programming in embodied robot execution, creating a tight loop of authoring, running, observing, and revising so learners see their code become behaviour. - **Social-robotic interaction:** [[social-robots|Social robots]] used for storytelling ([[motibo-digital-storytelling-robots-motivation-2026|MotiBo]], [[robobuddy-llm-social-robots-classroom-2025|RoboBuddy]]), role-play ([[remind-robot-mediated-roleplay-antibullying-2026|REMind]]), and sign language ([[pepper-robot-sign-language-lis-2025|Pepper]]) provide embodied social interaction that supports relational and [[social-emotional-learning|emotional]] learning. - **Embodiment and creative writing:** [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen|Research on robot-LLM integration in creative writing]] examines how embodiment affects learners' interaction and outcomes. - **Human-robot interaction:** [[human-robot-interaction|HRI]] research ([[task-context-trust-educational-hri-2026|trust]], [[human-autonomy-agency-hri-review-2025|agency]]) examines how physical embodiment shapes trust, engagement, and autonomy. Embodied learning connects to [[educational-robotics]], [[social-robots]], [[human-robot-interaction]], [[active-learning]], [[experiential-learning]], [[computational-thinking]], and [[social-emotional-learning]]. ## Connected Concepts - [[educational-robotics]] - [[social-robots]] - [[human-robot-interaction]] - [[active-learning]] - [[experiential-learning]] - [[computational-thinking]] - [[social-emotional-learning]] ## Connected Articles - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[pepper-robot-sign-language-lis-2025]] — Pepper and Sign Language - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Robot-LLM Integration in Creative Writing - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education --- ## [Engagement Metrics](https://edtechdev.github.io/aied/concepts/engagement-metrics/) > **Engagement metrics** — the range of observable signals and measurement approaches researchers and systems use to operationalize [[student-engagement|student engagement]] in AI-supported learning: behavioral (time-on-task, activity counts, interaction frequency), cognitive (depth of processing, critical engagement, discourse analysis), affective (emotion, motivation), and contextual (multitasking, attention). In AI-education research, the choice of engagement metric fundamentally shapes what counts as "engagement" and, therefore, what conclusions are drawn about whether and how AI tools help or harm learning. Engagement is a multidimensional construct, and no single metric captures it. Behavioral metrics measure what learners *do* (clicks, time, persistence); cognitive metrics measure how learners *think* (elaboration, critical analysis, self-regulation); affective metrics measure how learners *feel* (interest, anxiety, motivation). AI-education research increasingly combines these — and treats engagement as a mediating mechanism between AI tool design and learning outcomes, rather than an outcome in itself. ### How engagement metrics appear in the research - **Motivation and engagement as outcomes:** [[genai-motivation-engagement-2026|GenAI and student motivation research]] models perceived autonomy, competence, relatedness, and value as drivers of student motivation, which then emerges as the strongest predictor of engagement in generative-AI-supported learning — an SDT-based view where engagement follows need satisfaction. - **Critical engagement vs. passive use:** [[critical-engagement-code-completion|Measuring critical engagement in AI code completion]] and [[icap-cognitive-engagement-llm-agents|cognitive-engagement discourse analysis]] show that *how* students engage matters more than how much: critical, generative engagement with AI output predicts learning, whereas passive acceptance predicts the [[over-reliance]] and learning-displacement documented across the wiki. - **Engagement as a fragile, situation-dependent signal:** [[polished-artifacts-fragile-engagement-2026|Polished artifacts, fragile engagement]] and [[genai-tutor-engagement-patterns|multi-institution engagement patterns]] find that engagement with AI tutoring varies by context, task, and learner — the same tool produces strong engagement for some students and shallow, output-chasing behavior for others. - **Behavioral telemetry from learning platforms:** [[engagement-forecasting-its|Effort and progress forecasting]], [[learning-engagement-assistant-lea|Learning Engagement Assistant]], [[engagement-assessment-video|video engagement assessment]], and [[interactive-learning-dashboards-engagement|learning dashboards]] translate behavioral and physiological signals (attention, activity, persistence) into engagement metrics used for adaptive feedback and instructor intervention. - **Engagement as a learner-modeling signal:** [[engagement-intensity-learner-modeling|Engagement intensity as a learner-modeling signal]] uses engagement strength to inform adaptive AI systems, positioning engagement metrics as inputs to [[student-modeling]] and [[adaptive-learning]] rather than merely evaluation outputs. ### Why the metric choice matters The definitional problem is central to AI-education research. A study that measures engagement as *time-on-task* may conclude an AI tool enhances engagement when students spend more time interacting with it; a study that measures engagement as *critical processing* may reach the opposite conclusion for the same tool. This is why the wiki's research emphasizes distinguishing engagement (participation in the learning process) from learning (actual cognitive gain) — see [[genai-performance-vs-learning|performance vs. learning]] — and why engagement metrics must be validated against what they claim to measure. ### Connections to related concepts Engagement metrics connect to [[learning-analytics]] and [[educational-measurement]], which supply the quantitative tools. They intersect with [[motivation]] and [[self-determination-theory]] as the psychological antecedents of engagement, and with [[student-experience]] as the lived context. The distinction between genuine engagement and superficial use ties directly to [[over-reliance]], [[cognitive-offloading]], and [[self-regulated-learning]], since self-regulated learners engage critically and strategically with AI. In evaluation terms, engagement metrics feature in [[efficacy-study]] designs and relate to the affordances measured by [[engagement-metrics|behavioral telemetry]] in [[edtech-platform]]s. ## Connected Concepts - [[student-engagement]] - [[learning-analytics]] - [[motivation]] - [[student-experience]] - [[educational-measurement]] - [[self-regulated-learning]] - [[over-reliance]] - [[student-modeling]] - [[adaptive-learning]] - [[efficacy-study]] - [[higher-ed]] ## Connected Articles - [[genai-motivation-engagement-2026]] — Impact of Generative AI on Student Motivation and Engagement - [[critical-engagement-code-completion]] — To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion - [[icap-cognitive-engagement-llm-agents]] — Measuring Cognitive Engagement in Collaborative Discourse - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement - [[engagement-intensity-learner-modeling]] — Engagement Intensity as a Learner-Modeling Signal - [[learning-engagement-assistant-lea]] — Learning Engagement Assistant - [[engagement-assessment-video]] — Engagement Assessment in Video Learning - [[engagement-forecasting-its]] — From Heuristics to Analytics: Forecasting Effort and Progress - [[interactive-learning-dashboards-engagement]] — Interactive Learning Dashboards and Engagement - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning --- ## [Equity in AI Education](https://edtechdev.github.io/aied/concepts/equity-in-ai-education/) Equity in AI Education addresses systemic disparities in access to, representation within, and benefits from AI educational tools. Three critical dimensions emerge: ### 1. Access & Infrastructure Equity - **Digital Divide:** Unequal access to AI-powered learning tools across socioeconomic lines - **Language Barriers:** Most AI tools prioritize English, marginalizing multilingual learners - **Disability Access:** Need for AI tools designed with universal design principles ### 2. Representational Equity Wang et al. (2025) found that **78% of teachers** used LLMs to diversify curriculum materials, yet most AI training data reflects dominant cultural perspectives. Key gaps: - Underrepresentation of BIPOC authors in AI-curated reading lists - Western-centric problem contexts in STEM AI tutors - Gender and racial bias in AI-generated examples ### 3. Outcome Equity - **Differentiated Impact:** AI tools may widen gaps if designed without equity lens - **Teacher Bias Amplification:** AI suggestions can reinforce (not challenge) existing biases - **Student Agency:** Ensuring AI empowers rather than replaces student voice ## References Wang, J., Xiao, R., et al. (2025). *LLMs to Support K-12 Teachers in Culturally Relevant Pedagogy*. arXiv:2505.08083. ## Source - https://arxiv.org/abs/2505.08083 ## Connected Concepts - [[personalized-learning]] - [[student-experience]] - [[teacher-ai-competency]] - [[cs-education]] - [[language-learning]] - [[equity]] - [[special-education]] - [[stem-education]] - [[metacognition]] - [[faculty-development]] - [[over-reliance]] - [[human-in-the-loop-ai]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[ai-acceptance-preservice-science-teachers-2026]] - [[competency-based-education-genai-production-2026]] - [[ai-interior-design-malaysia-2026]] - [[critical-media-literacy-education-2026]] - [[ai-education-global-capacity]] - [[ai-powered-materials-discovery-ai-literacy]] - [[ai-scoring-language-bias-physics]] - [[cost-of-ethics-crisis-cs-ethics-education]] - [[genai-patient-education-transplant-handbooks]] - [[hybrid-human-ai-tutoring-differentiated]] - [[international-students-conversational-ai-adaptation]] - [[llm-children-reading-story-generation]] - [[neurodivergent-computing-students]] - [[suacode-african-students-motivations]] --- ## [Equity in AI Education](https://edtechdev.github.io/aied/concepts/equity/) > **Equity** — the principle that AI in education should serve all learners fairly, without exacerbating existing disparities. Equity research in the wiki examines access gaps, bias in AI systems, culturally responsive design, and the distribution of AI's benefits and harms. ### Equity dimensions in AIED - **Access equity:** [[access-not-enough-ai-tutoring-2026|Access is not enough]] — providing AI tools without addressing structural barriers does not close gaps. [[equity|Digital divide]] research and [[ai-literacy-equity-programming-policy|equity policy analysis]] document who gets left behind. - **Bias in AI systems:** [[bias-mitigation]], [[ai-scoring-language-bias-physics|language bias in scoring]], and [[bias-mitigation|algorithmic bias]] research show that AI can encode and amplify existing biases unless explicitly designed for fairness. - **Culturally responsive AI:** [[culturally-relevant-pedagogy|Culturally relevant pedagogy]] and [[community-centered-ai-education-adults|community-centered AI education]] advocate for AI that reflects diverse cultural contexts. - **Special populations:** [[special-education]], [[neurodivergent-computing-students|neurodivergent learners]], and [[dyslexlens-dyslexic-learners-ai|dyslexic learners]] represent groups whose needs are often overlooked in AI system design. - **Global equity:** [[suacode-african-students-motivations|African student motivations]], [[connected-ai-lesson-planning-vietnam|Vietnamese AI lesson planning]], and [[amponsah-ai-acceptance-science-teachers-2026|Ghanaian teacher acceptance]] provide Global South perspectives. ### Equity as a design principle The wiki argues that equity must be designed into AI systems, not added as an afterthought. This connects to [[bias-mitigation]] (the technical approach), [[institutional-change-framework-ai|institutional change]] (the systemic approach), and [[ai-literacy]] (ensuring all students can use AI effectively). ### Connections Equity connects to [[bias-mitigation]], [[special-education]], [[accessible-learning|accessible learning]], [[culturally-relevant-pedagogy]], and [[educational-policy-ai]] — it is the cross-cutting concern that every AI education deployment must address. ## Connected Concepts - [[bias-mitigation]] - [[special-education]] - [[accessible-learning]] - [[culturally-relevant-pedagogy]] - [[ai-literacy]] - [[educational-policy-ai]] - [[higher-ed]] - [[k-12]] - [[privacy]] - [[regulation]] - [[ethics]] ## Connected Articles - [[access-not-enough-ai-tutoring-2026]] - [[ai-scoring-language-bias-physics]] - [[ai-literacy-equity-programming-policy]] - [[culturally-aware-aied-community-learning]] - [[community-centered-ai-education-adults]] - [[amponsah-ai-acceptance-science-teachers-2026]] - [[neurodivergent-computing-students]] - [[elementary-writing-genai-systematic-review-2026]] - [[ai-uk-higher-education-policy-2026]] - [[genai-policies-higher-ed-computing]] — Who sets GenAI-use rules and how consistently across courses - [[genai-higher-education-systematic-review-2026]] - [[structural-silence-underrepresented-language-ai-2026]] --- ## [Ethics in AI Education](https://edtechdev.github.io/aied/concepts/ethics/) > **Ethics** — the moral principles governing the design, deployment, and use of AI in educational contexts. AI education ethics spans data privacy, algorithmic fairness, transparency, accountability, and the broader question of what AI should and should not do in learning environments. ### Ethical dimensions - **Fairness and bias:** [[bias-mitigation]] and [[equity]] research address whether AI systems treat all learners fairly. [[ai-scoring-language-bias-physics|Language bias]] and [[bias-mitigation]] studies document real-world inequities. - **Privacy and consent:** [[privacy]] research examines data collection, student surveillance, and the power imbalance between institutions and learners. - **Transparency and explainability:** [[xai-education-framework|Explainable AI frameworks]] argue that students and teachers should understand how AI systems make decisions affecting them. - **Autonomy and agency:** [[over-reliance]] and [[cognitive-offloading]] research raise ethical questions about whether AI use diminishes learner agency. - **Safety and harm prevention:** [[pedagogical-safety]] and [[ai-tutor-safety-harms|tutor harm research]] define ethical obligations for AI system developers. ### Ethics in practice The wiki's ethics articles range from theoretical frameworks ([[ethical-ai-higher-ed-game-theory|game theory approaches]]) to practical guidelines ([[cost-of-ethics-crisis-cs-ethics-education|CS ethics education]]). [[ai-ethics-education-public-discourse|Public discourse analysis]] tracks how AI ethics conversations evolve over time. ### Connections Ethics connects to [[equity]], [[privacy]], [[bias-mitigation]], [[regulation]], [[pedagogical-safety]], [[academic-integrity]], and [[ai-governance-education]]. It is the normative foundation for all other AI education concepts. ## Connected Concepts - [[equity]] - [[privacy]] - [[bias-mitigation]] - [[regulation]] - [[pedagogical-safety]] - [[academic-integrity]] - [[ai-governance-education]] - [[ai-literacy]] - [[over-reliance]] - [[teacher-role]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[genai-student-experiences-uk-he-survey-2026]] - [[critical-media-literacy-education-2026]] - [[ai-ethics-education-public-discourse]] - [[ethical-ai-higher-ed-game-theory]] - [[cost-of-ethics-crisis-cs-ethics-education]] - [[xai-education-framework]] - [[ai-tutor-safety-harms]] - [[ai-uk-higher-education-policy-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Experiential Learning](https://edtechdev.github.io/aied/concepts/experiential-learning/) > **Experiential learning** — learning through direct experience, reflection, and the application of knowledge in authentic or hands-on contexts ("learning by doing"). Drawing on Kolb's experiential learning cycle (concrete experience, reflective observation, abstract conceptualization, active experimentation), experiential approaches emphasize that learners learn most deeply when they act, observe the results, and reflect. In AI education, experiential learning includes hands-on labs, project-based work, [[educational-robotics|robotics]], simulations, and real-world problem solving. Experiential learning is closely related to [[active-learning]], [[project-based-learning]], [[embodied-learning]], and [[simulation]]. It is particularly relevant to AI, cybersecurity, and robotics education, where students develop skills by working with tools and systems in applied contexts rather than through lectures alone. A key rationale is closing the theory-practice gap in professional preparation. ### How experiential learning appears in the wiki's research - **Cybersecurity labs:** [[genai-cybersecurity-ocr-multimodal-instruction-2025|LLM-assisted cybersecurity instruction]] integrates a generative-AI instructional assistant into a virtual lab platform, supporting hands-on experiential skill building. - **Robotics projects:** [[bots-blocks-project-based-robotics-education-2026|Bots and Blocks]] uses a project-based, hands-on approach to teach robotics, addressing the lack of practical experience in classic programs. - **Simulation and embodied learning:** [[edusim-llm-robotic-simulation-education-2026|EduSim-LLM]] lets beginners experiment with simulated robots, and [[embodied-learning|embodied]] robot interaction grounds learning in direct experience. Experiential learning connects to [[active-learning]], [[project-based-learning]], [[embodied-learning]], [[simulation]], [[educational-robotics]], and [[higher-ed]] professional preparation. ## Connected Concepts - [[active-learning]] - [[project-based-learning]] - [[embodied-learning]] - [[simulation]] - [[educational-robotics]] - [[higher-ed]] ## Connected Articles - [[genai-cybersecurity-ocr-multimodal-instruction-2025]] — GenAI in Cybersecurity Education - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[ai-lms-middle-school-longitudinal]] — AI-Integrated LMS Longitudinal Study --- ## [Faculty Development](https://edtechdev.github.io/aied/concepts/faculty-development/) > **Faculty development** — the processes, programs, and institutional supports that help educators develop the skills and confidence to teach effectively with AI. Faculty development spans individual training, curriculum redesign, and institutional policy change. ### Faculty development in the AI era - **Readiness frameworks:** [[sangwa-epiq-ai-faculty-readiness-2026|The EPIQ-AI framework]] identifies four readiness domains: epistemic, pedagogical, institutional, and quality-and-compliance. Faculty readiness is a sociotechnical alignment problem, not just an individual skills gap. - **Adoption and confidence:** [[teacher-ai-adoption-confidence|Teacher AI adoption research]] identifies concerns, support, confidence, and attitudes as key predictors. [[faculty-development]] programs must address all four. - **Curriculum integration:** [[institutional-change-framework-ai|Institutional change frameworks]] and [[ai-assessment-scale-reform|assessment reform]] require faculty to redesign courses, not just add AI tools. - **Training programs:** [[crewscaler-ai-upskilling-framework|AI upskilling frameworks]] and [[ai-tpack-preservice-math-teachers|TPACK-based preservice training]] provide models for structured faculty AI education. - **Governance and policy:** [[genai-policies-higher-ed-computing|Institutional AI policy analysis]] documents the gap between institutional ambitions and faculty support capacity. ### Connections Faculty development connects to [[teacher-ai-competency]] (the outcome), [[teacher-role]] (how AI changes instructional work), [[ai-literacy]] (faculty must model AI literacy for students), and [[educational-policy-ai]] (institutional policies that enable or constrain development). ## Connected Concepts - [[teacher-ai-competency]] - [[teacher-role]] - [[ai-literacy]] - [[educational-policy-ai]] - [[higher-ed]] - [[k-12]] - [[instructional-design]] - [[curriculum-design]] - [[institutional-change-framework-ai]] - [[professional-training]] ## Connected Articles - [[ai-acceptance-preservice-science-teachers-2026]] - [[sangwa-epiq-ai-faculty-readiness-2026]] - [[teacher-ai-adoption-confidence]] - [[institutional-change-framework-ai]] - [[genai-policies-higher-ed-computing]] - [[ai-tpack-preservice-math-teachers]] - [[crewscaler-ai-upskilling-framework]] - [[ai-assessment-scale-reform]] - [[pchl-he-framework-genai-content-creation-2026]] - [[genai-pd-ai-pck-learning-gain-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Feedback Loop](https://edtechdev.github.io/aied/concepts/feedback-loop/) > **Feedback loop** — the cyclical process where AI systems assess student work, deliver feedback, observe the student's response, and adapt subsequent instruction. Effective feedback loops close the gap between current and desired performance. ### The feedback cycle AI-mediated feedback loops operate at multiple timescales: - **Immediate feedback:** [[automated-grading|Automated grading systems]] and [[ai-tutoring|AI tutors]] provide real-time correction during problem-solving. [[correct-answer-trap-ai-tutor|Correct-answer trap research]] shows that immediate feedback can short-circuit learning if students simply copy corrections. - **Assignment-level feedback:** [[formative-assessment]] systems and [[ai-feedback-quality|AI feedback quality research]] examine whether AI-generated assignment feedback improves subsequent work. [[sequenced-ai-feedback-learning|Sequenced feedback studies]] test whether the order of feedback matters. - **Course-level loops:** [[learning-analytics|Learning analytics dashboards]] and [[edtech-platform|educational platforms]] aggregate feedback across assignments to identify patterns and recommend interventions. ### Quality matters The effectiveness of a feedback loop depends on [[ai-feedback-quality|feedback quality]] — accuracy, specificity, timeliness, and actionability. [[ai-peer-feedback-systems|AI peer feedback systems]] add a social dimension to the loop. [[confidence-aware-ai-assessment|Confidence-aware systems]] calibrate feedback strength to model certainty. ### Connections Feedback loops connect to [[formative-assessment]] (the assessment philosophy that feedback loops operationalize), [[self-regulated-learning]] (learners use feedback to adjust strategies), and [[scaffolding]] (feedback is a form of just-in-time scaffolding). ## Connected Concepts - [[formative-assessment]] - [[ai-feedback-quality]] - [[automated-grading]] - [[scaffolding]] - [[self-regulated-learning]] - [[ai-tutoring]] - [[learning-analytics]] - [[confidence-aware-ai-assessment]] - [[metacognition]] - [[student-experience]] - [[personalized-learning]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] - [[sequenced-ai-feedback-learning]] - [[correct-answer-trap-ai-tutor]] - [[ai-feedback-quality]] - [[ai-peer-feedback-systems]] - [[automated-formative-assessments-a-level-sciences]] - [[ai-feedback-enactment-workflow-2026]] --- ## [Formative Assessment in AI Education](https://edtechdev.github.io/aied/concepts/formative-assessment/) Assessment designed to inform ongoing instruction and learning, as opposed to summative evaluation. AI systems can generate, validate, and adapt formative assessment items at scale, though quality varies dramatically across assessment types. ## AI-Generated Formative Items ### Multiple-Choice Questions (CODE-GEN) Duan et al. (2026) demonstrate that agentic AI can reliably generate MCQs for coding comprehension when validated across seven pedagogical dimensions. Success rates reach **98.6%** for concept alignment and **79.9%** for feedback quality—suggesting that AI is strongest on verifiable dimensions and weakest on instructional-judgment dimensions. ### Automated Essay Scoring (MASS) Kamalov et al. (2026) implement a multi-agent framework (MASS) for essay scoring. Preliminary results show improved consistency over stand-alone LLMs, though interpretability of multi-agent scoring decisions remains an open challenge. ### Curriculum-Grounded Feedback (LearnLens) Zhao et al. (2025) present **LearnLens**, a modular LLM system for science education feedback that addresses three persistent problems in AI formative assessment: 1. **Error-aware assessment** — captures nuanced reasoning errors rather than surface mistakes 2. **Topic-linked memory chains** — replaces noisy similarity-based RAG with structured curriculum-grounded retrieval 3. **Educator-in-the-loop** — teacher customisation and oversight, not full automation **Key differentiator:** LearnLens uses a **structured, topic-linked memory chain** rather than traditional RAG similarity search, improving relevance and reducing noise. This connects to the broader tension in [[human-in-the-loop-ai]]: scalable automation with expert validation. ## Design Trade-offs | Dimension | AI Suitability | Human Requirement | |-----------|----------------|-------------------| | Factual correctness | High | Low | | Concept alignment | High | Medium | | Distractor quality | Low | High | | Feedback depth | Low | High | | Rubric consistency | Medium | Medium | ## Risk: Assessment as Surveillance Formative assessment systems can shift from learning-support tools to behavior-monitoring infrastructure. The same data streams that enable adaptive tutoring can enable punitive tracking if governance is weak. ## Connected Concepts - [[ai-literacy]] - [[higher-ed]] - [[automated-grading]] - [[student-experience]] - [[scaffolding]] - [[stem-education]] - [[personalized-learning]] - [[teacher-role]] - [[adaptive-learning]] - [[feedback-loop]] - [[generative-ai]] - [[intelligent-tutoring]] ## Connected Articles - [[ai-changing-teaching-workflows]] - [[ai-coaching-rl-skill-development]] - [[ai-generated-feedback-higher-ed]] - [[ai-learning-tools-engineering-education-needs]] - [[assessment-team-problem-solving-computing-education]] - [[authentic-assessment]] - [[automated-formative-assessments-a-level-sciences]] - [[automated-grading-linux-bash-examinations-large-language-models]] - [[becerra-aicofe-feedback-2026]] - [[buggy-genai-code-student-responses]] - [[code-review-genai-cs1]] - [[cognitive-offloading-llm-synthesis-writing]] - [[correct-answer-trap-ai-tutor]] - [[correct-answer-trap-misconceptions]] - [[critical-engagement-code-completion]] --- ## [Game-Based Learning](https://edtechdev.github.io/aied/concepts/game-based-learning/) > **Game-based learning (GBL)** — the use of games themselves (digital or physical) as the medium and context for learning, where the game's mechanics, challenges, and progression carry educational content. Learners learn *through* playing. GBL is distinct from [[gamification]], which adds game elements (points, badges, leaderboards) to non-game activities. In AI and robotics education, GBL is used to make technical content engaging and motivating, and is more prevalent in informal settings. GBL is grounded in [[motivation]], [[student-engagement]], and [[active-learning]] theories: games provide intrinsic motivation, immediate feedback, and authentic problem contexts. It overlaps with [[gamification]], [[simulation]], and [[project-based-learning]]. GBL is particularly relevant to [[educational-robotics]], [[computational-thinking]], and [[programming-education]], where games can make abstract technical concepts concrete and fun. ### How GBL appears in the wiki's research - **Robotics education:** [[game-based-gamified-robotics-education-review-2026|A comparative systematic review]] of GBL and gamification in robotics education found GBL more prevalent in informal settings, while gamification dominated formal classrooms and favored project-based learning. - **Robot-mediated games:** [[remind-robot-mediated-roleplay-antibullying-2026|REMind]] is a robot-mediated role-play game for anti-bullying intervention, and [[motibo-digital-storytelling-robots-motivation-2026|MotiBo]] uses interactive digital storytelling to boost motivation. GBL connects to [[gamification]], [[educational-robotics]], [[student-engagement]], [[motivation]], [[active-learning]], [[simulation]], and [[project-based-learning]]. ## Connected Concepts - [[gamification]] - [[educational-robotics]] - [[student-engagement]] - [[motivation]] - [[active-learning]] - [[simulation]] ## Connected Articles - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks --- ## [Gamification](https://edtechdev.github.io/aied/concepts/gamification/) > **Gamification** — the application of game-design elements (points, badges, levels, leaderboards, challenges, progress bars) to non-game contexts to motivate and engage users. Unlike [[game-based-learning]] (where learning happens *through* a game), gamification layers game mechanics onto an existing learning activity without turning it into a full game. In education, gamification is used to boost [[motivation]], [[student-engagement]], and persistence, and is widely applied in formal classroom settings. Gamification is grounded in motivational theory, particularly [[self-determination-theory]] (supporting autonomy, competence, and relatedness) and behaviour-change frameworks. It is closely related to [[game-based-learning]], [[active-learning]], and [[educational-robotics|educational technology]], and has shown particular synergy with [[project-based-learning]] in applied domains like robotics. ### How gamification appears in the wiki's research - **Robotics education:** [[game-based-gamified-robotics-education-review-2026|A comparative systematic review]] found gamification dominated formal classrooms in robotics education (p < .001) and strongly favored project-based learning (p = .009), while game-based learning was more common in informal settings. - **Engagement and motivation:** Gamification is used across the wiki to increase learner engagement and motivation in AI, programming, and STEM learning contexts. Gamification connects to [[game-based-learning]], [[student-engagement]], [[motivation]], [[self-determination-theory]], [[active-learning]], [[project-based-learning]], and [[educational-robotics]]. ## Connected Concepts - [[game-based-learning]] - [[student-engagement]] - [[motivation]] - [[self-determination-theory]] - [[active-learning]] - [[project-based-learning]] - [[educational-robotics]] ## Connected Articles - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[genai-motivation-engagement-2026]] — Generative AI, Motivation, and Engagement --- ## [Generative AI](https://edtechdev.github.io/aied/concepts/generative-ai/) > **Generative AI** — AI systems capable of producing text, code, images, and other content, most prominently large language models like GPT-4 and Claude. Generative AI is the technology driving the current wave of AI in education research. ### What makes generative AI different for education Unlike earlier rule-based or retrieval-based systems, generative AI produces fluent, contextually appropriate content on demand. This creates both unprecedented opportunities and novel risks: - **Content generation:** LLMs can create instructional materials, examples, and explanations. [[book-level-synthetic-textbook-organization|Synthetic textbooks]], [[courseblueprint-adaptive-video-generation|adaptive videos]], and [[ai-generated-instructional-videos-computing-ed|instructional videos]] show the range of educational content generation. - **Tutoring and dialogue:** [[ai-tutoring|AI tutoring systems]] use generative AI for conversational instruction. [[socratic-method|Socratic dialogue]] and [[collaborative-ai-tutoring|collaborative tutoring]] exploit generative capabilities for pedagogical interaction. - **Assessment:** [[automated-essay-scoring|Essay scoring]], [[automated-grading|automated grading]], and [[formative-assessment]] increasingly rely on generative models. - **Risks:** [[hallucination-risk|Hallucination]], [[over-reliance]], [[cognitive-offloading]], and [[academic-integrity]] concerns arise specifically from generative AI's fluency and accessibility. ### The wiki's generative AI coverage With 80+ articles, generative AI is the wiki's largest technology thread. Research spans effectiveness studies ([[genai-meta-analysis-programming-learning|meta-analyses]]), safety concerns ([[ai-tutor-safety-harms|tutor harms]], [[eduguard-safe-rag-llm-tutor|guardrailing]]), and design principles ([[instructional-guidance-genai-learning|instructional guidance]]). ### Connections Generative AI connects to [[llm]] (the model class), [[prompt-engineering]] (how outputs are shaped), [[rag]] (retrieval-augmented grounding), and [[ai-literacy]] (the competency needed to use it effectively). ## Connected Concepts - [[llm]] - [[prompt-engineering]] - [[rag]] - [[ai-literacy]] - [[ai-tutoring]] - [[over-reliance]] - [[hallucination-risk]] - [[cognitive-offloading]] - [[academic-integrity]] - [[automated-grading]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[competency-based-education-genai-production-2026]] - [[ai-interior-design-malaysia-2026]] - [[ai-generated-interactive-fiction-education-2026]] - [[genai-meta-analysis-programming-learning]] - [[ai-tutor-safety-harms]] - [[eduguard-safe-rag-llm-tutor]] - [[instructional-guidance-genai-learning]] - [[genai-performance-vs-learning]] - [[liang-genai-systematic-review-human-ai-2026]] - [[learnity-graphs-lifelong-learning-framework-2026]] - [[veriforge-narrative-drafting-scaffolding-2026]] - [[pchl-he-framework-genai-content-creation-2026]] - [[elementary-writing-genai-systematic-review-2026]] - [[genai-pd-ai-pck-learning-gain-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [AI Governance](https://edtechdev.github.io/aied/concepts/governance/) > **AI governance** — the frameworks, policies, institutional structures, and norms that guide the responsible design, deployment, and use of artificial intelligence in education. Governance spans formal institutional mechanisms (AI steering groups, policies on academic integrity and acceptable use, ethical review) and informal norms (faculty guidelines, professional development, cultures of responsible AI use). In the AI era, effective governance is a prerequisite for ethical, equitable, and sustainable adoption of GenAI — it determines whether AI is integrated transparently, with accountability, or adopted reactively in ways that deepen inequities. AI governance in education is increasingly urgent because generative AI introduces new epistemic, ethical, and organizational challenges: it destabilizes assumptions about knowledge production, learner agency, assessment validity, and the role of educators as epistemic authorities. Governance addresses questions of academic integrity (what counts as acceptable AI use), data privacy and security, algorithmic bias and fairness, transparency and accountability, and the alignment of AI adoption with institutional mission and values. A recurring finding across the wiki's research is that **institutional governance is often lagging** — many institutions lack clear, unified AI policies, leaving students and faculty to negotiate acceptable use on their own. ### How AI governance appears in the research - **Institutional adoption at scale:** [[new-systems-of-learning-for-distance-learning-institutions-a-six-study-review-of|The AIDA study at the Open University]] shows how an institution designed, implemented, and evaluated a GenAI assistant, identifying that responsible system-level deployment requires governance structures (AI Steering Group), senior leadership sponsorship, and alignment with institutional strategy — not just technical capability. - **Leadership and systemic change:** [[leveraging-complex-systems-leading-for-transformative-change|SPARK]] frames governance within Complexity Leadership Theory, arguing leaders must balance administrative stability with emergent innovation, embedding governance mechanisms (policies, assessment regimes, accountability frameworks) so adaptive-space innovations can be sustained and scaled. - **Policy ambiguity and student experience:** [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination|Students' engagement with GenAI]] found 12/23 students noted the lack of explicit institutional AI policies ("University doesn't have a clear and unified policy yet"), arguing governance ambiguity shapes students' practices, norms, and self-regulation — supporting a shift toward transparent institutional guidance. - **Academic integrity and assessment:** Governance is central to how institutions handle AI-related [[academic-integrity]] concerns and redesign [[assessment|assessment]] — moving from prohibition/policing toward guidance, AI literacy, and process-oriented designs, as seen in research on [[student-rationalization-ai-writing|student rationalization]] and [[beyond-detection-authentic-assessment-ai-2025|authentic assessment redesign]]. - **Ethics, privacy, and bias:** Governance mechanisms operationalize the ethical principles ([[ethics]], [[privacy]], [[bias-mitigation]]) that are often recognized but not enforced, connecting to responsible AI and regulatory debates in education. ### Governance across levels AI governance operates at multiple levels — from **national/regulatory** (government policy, the OECD framework, state AI guidelines) to **institutional** (university policies, AI steering groups, ethical review boards) to **classroom** (instructor guidelines, syllabus statements, assignment design). Effective governance aligns these levels: national frameworks set expectations, institutions translate them into policies and support structures, and educators implement them in ways that build students' AI literacy and agency. The wiki's research emphasizes that governance is not merely about restriction but about creating the conditions for responsible, equitable, and learning-supportive AI integration — including faculty development, transparent guidance, and ongoing evaluation. ### Connections to related concepts AI governance connects to [[ethics]] (the principles it operationalizes), [[higher-ed]] (the institutional context), [[privacy]] and [[bias-mitigation]] (specific governance concerns), and [[academic-integrity]] (a primary governance arena). It is central to institutional change and responsible AI, and intersects with [[ai-literacy]] (governance supports the development of critical, informed use). It also connects to [[learning-analytics]] (data governance) and [[student-experience]] (governance shapes how students navigate acceptable use). ## Connected Concepts - [[ethics]] - [[higher-ed]] - [[privacy]] - [[bias-mitigation]] - [[academic-integrity]] - [[ai-literacy]] - [[learning-analytics]] - [[student-experience]] ## Connected Articles - [[new-systems-of-learning-for-distance-learning-institutions-a-six-study-review-of]] — Implementing AIDA at the Open University - [[leveraging-complex-systems-leading-for-transformative-change]] — SPARK: Leading for Transformative Change - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] — Students' Engagement With GenAI (SDT) - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[state-policy-teacher-ai]] — State Policy and Teacher AI - [[stanford-evidence-base-ai-k12-2026]] — The Stanford Evidence Base for AI in K-12 - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: Authentic Assessment Redesign - [[ethical-ai-higher-ed-game-theory]] — Ethical AI in Higher Education --- ## [Hallucination Risk](https://edtechdev.github.io/aied/concepts/hallucination-risk/) > **Hallucination Risk** — the danger that AI systems generate plausible but factually incorrect or fabricated content in educational contexts, where such errors can mislead learners, undermine trust, and produce invalid assessments. Hallucination is particularly consequential in education because students may lack the domain knowledge to detect AI errors, and teachers may rely on AI-generated diagnoses or feedback that appears authoritative but is unfounded. Hallucination in educational AI takes several forms documented in this wiki's articles: fabricated evidence in student assessment, over-confident misdiagnosis of learner knowledge, and plausible-sounding but incorrect explanations that students accept as truth. The risk is amplified in education because the asymmetry of knowledge between AI and learner means the learner is poorly positioned to verify AI outputs. **Assessment hallucination** is particularly damaging. **[[llm-cognitive-diagnosis-handwritten-math|MathCog]]** found that LLMs fabricate evidence quotes not present in student handwriting when diagnosing cognitive skills, with 58.5% of incorrect diagnoses accompanied by false claims of evidential confidence. **[[llm-fallacy-misattribution]]** documented systematic over-attribution of evidence in LLM reasoning — models claim evidential support where none exists. Both connect to [[ai-ed-evaluation]] and [[knowledge-tracing]] concerns about assessment validity. **Tutoring hallucination** affects learning directly. **[[llm-tutoring-feedback-diagnosis-gap]]** found LLMs over-validated incorrect solutions while over-rejecting valid-but-suboptimal reasoning — systemic failures that would mislead both students and teachers. **[[eduframetrap-llm-sycophancy-educational-safety]]** and **[[eduguard-safe-rag-llm-tutor]]** address safety mechanisms for educational LLMs. These risks connect to [[pedagogical-safety]] and [[human-in-the-loop-ai]] requirements. **Mitigation approaches** include [[human-in-the-loop-ai]] designs where AI supports rather than replaces teacher judgment, evidence-aware architectures that calibrate confidence based on evidential quality (as advocated by MathCog), and [[rag]]-based grounding that constrains LLM outputs to verified sources. The [[over-reliance]] concept is closely related — hallucination is most dangerous when users trust AI outputs uncritically. ## Connected Concepts - [[over-reliance]] - [[human-in-the-loop-ai]] - [[ai-ed-evaluation]] - [[pedagogical-safety]] - [[knowledge-tracing]] - [[rag]] - [[academic-integrity]] - [[teacher-role]] - [[cognitive-offloading]] - [[multimodal]] - [[generative-ai]] - [[llm]] ## Connected Articles - [[llm-cognitive-diagnosis-handwritten-math]] - [[llm-fallacy-misattribution]] - [[llm-tutoring-feedback-diagnosis-gap]] - [[eduframetrap-llm-sycophancy-educational-safety]] - [[eduguard-safe-rag-llm-tutor]] - [[prompt-injection-defenses-educational-llm-tutors]] - [[veriforge-narrative-drafting-scaffolding-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Help-Seeking](https://edtechdev.github.io/aied/concepts/help-seeking/) > **Help-Seeking** — a key concept in AI in education research. Explored across 4 articles in this wiki. Research themes include: - **Alexandra Neagu, Jeffrey T. H. Wong, Marcus Messer, Rhodri Nelson, Peter B. Johnson** (2026). Pluralistic Alignment Workshop @ ICML 2026 - A three-semester, 999-student analysis of hint usage in a K-12 mathematics ITS finds that two simple, interpretable indicators—premature hint requests and superficial hint reading—are consistently associated with reduced learning gains, even after co - Recent work in Technology-Enhanced Learning and HumanComputer Interaction highlights the importance of transparency and trust calibration in AI-supported learning environments as they pose a risk of hallucinations. In this study, we investigate wheth ## Connected Concepts - [[help-seeking]] - [[affective-tutoring]] - [[metacognition]] - [[agentic-ai]] - [[ai-tutoring]] - [[teacher-ai-competency]] - [[socratic-ai-dialogue]] - [[pedagogical-agent]] ## Connected Articles - [[edumirror-educational-social-dynamics]] — EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation - [[adaptive-virtual-patient-psychotherapy-training]] — The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training - [[bridging-instructional-design-framework-math]] — WIP: Bridging the Gap Between Instructional Design and Pedagogical Use: A Framework for Mathematics Educators - [[genai-academic-search-workshop]] — Report on CHIIR 2026 Workshop on Generative AI and Academic Search (GAI&AS) - [[cognitive-offloading-llm-synthesis-writing]] — Profiling cognitive offloading in LLM-mediated synthesis writing: Volume vs. content - [[persistent-ai-agents-academic-research]] — Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study --- ## [AI in Higher Education](https://edtechdev.github.io/aied/concepts/higher-ed/) > **AI in Higher Education** — the integration of artificial intelligence into university teaching, learning, assessment, and administration. Higher education is the most-studied context in the wiki, with over 100 articles examining how AI transforms college-level instruction, institutional policy, and student experience. AI in higher education research spans every function of the university: from AI tutoring and automated grading to faculty development, academic integrity, institutional governance, and student support. The wiki's higher education articles cluster around several key themes. ### Institutional transformation [[institutional-change-framework-ai|Institutional change frameworks]] analyze how universities adapt to AI — not just at the classroom level but across policy, governance, and organizational structure. [[sangwa-epiq-ai-faculty-readiness-2026|The EPIQ-AI framework]] reframes faculty readiness as a sociotechnical alignment challenge involving epistemic, pedagogical, institutional, and quality domains. [[universities-ai-era-rethinking|Rethinking universities in the AI era]] examines whether current institutional models can accommodate AI-driven education. ### Student experience at scale Large-scale studies of [[ai-in-the-wild-college|authentic student AI use]] and [[genai-availability-grades-satisfaction|GenAI availability and satisfaction]] document how students actually use AI — revealing gaps between institutional policy and everyday practice. [[ai-higher-ed-workforce-survey|Workforce preparation surveys]] connect AI use in higher education to employment outcomes. ### Faculty and teaching [[faculty-development]] research examines how instructors adopt, resist, or adapt to AI. [[teacher-ai-adoption-confidence|Teacher AI adoption studies]] identify confidence, support, and attitude as key predictors. [[ai-assistance-discretionary-feedback|AI-assisted discretionary feedback]] research explores whether AI increases instructor feedback quality and quantity. ### Assessment and integrity [[academic-integrity]] and [[ai-assessment-scale-reform|AI assessment reform]] research grapple with how universities should redesign evaluation for an AI-capable student body. Detection-centered approaches are giving way to [[authentic-assessment]] and process-based evaluation. ## Connected Concepts - [[generative-ai]] - [[llm]] - [[student-experience]] - [[faculty-development]] - [[academic-integrity]] - [[ai-literacy]] - [[assessment-validity]] - [[teacher-role]] - [[scaffolding]] - [[educational-policy-ai]] - [[regulation]] - [[professional-training]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[ai-acceptance-preservice-science-teachers-2026]] - [[competency-based-education-genai-production-2026]] - [[ai-interior-design-malaysia-2026]] - [[critical-media-literacy-education-2026]] - [[ai-generated-interactive-fiction-education-2026]] - [[sangwa-epiq-ai-faculty-readiness-2026]] — EPIQ-AI Faculty Readiness Framework - [[genai-policies-higher-ed-computing]] — Institutional vs course GenAI policy in computing - [[ai-in-the-wild-college]] — AI in the Wild: College Student AI Use - [[institutional-change-framework-ai]] — Institutional Change in the Age of AI - [[genai-availability-grades-satisfaction]] — GenAI Availability and Student Satisfaction - [[ai-assessment-scale-reform]] — AI Assessment Scale and Reform - [[ai-higher-ed-workforce-survey]] — AI in Higher Education Workforce Survey - [[universities-ai-era-rethinking]] — Rethinking Universities in the AI Era - [[teacher-ai-adoption-confidence]] — AI Adoption Among Teachers - [[learnity-graphs-lifelong-learning-framework-2026]] - [[pchl-he-framework-genai-content-creation-2026]] - [[ai-uk-higher-education-policy-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Human AI Collaboration](https://edtechdev.github.io/aied/concepts/human-ai-collaboration/) > **Human-AI collaboration** — the division of cognitive labor between people and models — is the wiki's core interaction theme: [[human-ai-collaboration-trust-expectations]], [[humanlike-ai-collaborative-writing]], [[genai-mindtool-generative-learning]], and [[teacher-student-agency-orchestration]] examine trust, agency, and complementary roles ([[human-in-the-loop-ai]], [[agentic-ai]]). ## Connected Concepts - [[generative-ai]] - [[ai-literacy]] - [[llm]] - [[scaffolding]] - [[intelligent-tutoring]] - [[teacher-role]] - [[higher-ed]] - [[k-12]] - [[over-reliance]] - [[student-experience]] - [[metacognition]] - [[self-regulated-learning]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[agent-voice-accents-k12-group-learning]] - [[chat-debugging-human-ai-collaboration-circuits]] - [[generativism-learning-theory]] - [[student-mental-models-genai]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] - [[genai-higher-education-systematic-review-2026]] - [[ai-feedback-enactment-workflow-2026]] --- ## [Human-in-the-Loop AI for Education](https://edtechdev.github.io/aied/concepts/human-in-the-loop-ai/) Educational AI systems that strategically interleave automated generation with human expert judgment, preserving pedagogical quality while scaling production. Two recent implementations illustrate distinct architectures: ## CODE-GEN: Human-in-the-Loop MCQ Generation Duan et al. (2026) built a RAG-based agentic system with two agents: - **Generator Agent** — Produces multiple-choice coding questions aligned with course learning objectives - **Validator Agent** — Assesses quality across seven pedagogical dimensions **Evaluation:** 6 SMEs judged 288 AI-generated questions. Human-validated success rates: **79.9%–98.6%** across dimensions. **AI-Strong Dimensions (low human burden):** - Question clarity, code validity, concept alignment, correct-answer validity **Human-Required Dimensions (high human burden):** - Pedagogically meaningful distractor design - High-quality explanatory feedback Strategic insight: Human effort should be concentrated where instructional judgment is irreplaceable; computational verification can be fully automated. ## MAIC: Human-in-the-Loop Script Generation Yu et al. (2024) deployed a multi-agent classroom (Teacher Agent, TA Agent, classmate archetypes) at Tsinghua University with >500 students and >100,000 learning records. Human instructors participate in script generation and oversight, ensuring that mass-scale AI augmentation does not displace pedagogical expertise. ## Synthesis Human-in-the-loop design is not merely a safety measure—it is a **resource-allitution strategy**. The frontier question is not *whether* to include humans, but *where* in the pipeline their judgment has highest marginal value. ## Connected Concepts - [[formative-assessment]] - [[automated-grading]] - [[scaffolding]] - [[teacher-role]] - [[ai-literacy]] - [[intelligent-tutoring]] - [[feedback-loop]] - [[student-experience]] - [[self-regulated-learning]] - [[metacognition]] - [[faculty-development]] - [[generative-ai]] ## Connected Articles - [[agentic-ai-education-scoping-review]] - [[ai-changing-teaching-workflows]] - [[ai-literacy-legal-translation-2026]] - [[ai-tutor-effectiveness-review]] - [[aicode-collaborative-feedback-system]] - [[becerra-aicofe-feedback-2026]] - [[calibrating-trustworthiness-llm-education-2026]] - [[chatgpt-critical-creative-thinking-review]] - [[civic-education-ai-lesson-plans]] - [[code-gen]] - [[concept-catalyst-engineering-scaffolds]] - [[cong-confidence-asag-2026]] - [[correct-answer-trap-ai-tutor]] - [[cyberscholar-genai-writing-feedback]] - [[eduagentbench-agent-teaching-benchmark]] - [[llm-difficulty-calibration-programming-exams-2026]] - [[veriforge-narrative-drafting-scaffolding-2026]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] - [[pchl-he-framework-genai-content-creation-2026]] --- ## [Human-Robot Interaction](https://edtechdev.github.io/aied/concepts/human-robot-interaction/) > **Human-robot interaction (HRI)** — the interdisciplinary study of how people and robots interact, encompassing perception, communication, collaboration, and the social, cognitive, and ethical dynamics of that interaction. In education, HRI underlies how learners perceive, trust, and learn with robots — whether programming a robot, conversing with a tutoring robot, or rehearsing social scenarios. HRI research examines how robot appearance, behaviour, task context, and embodiment shape user experience, trust, agency, and learning. HRI sits at the intersection of [[social-robots|robotics]], cognitive science, psychology, and [[human-ai-collaboration]]. It is the empirical and theoretical foundation for [[educational-robotics]], examining what makes human-robot interaction effective, trustworthy, and beneficial for learning and well-being. Key concerns in educational HRI include preserving human [[agency]], building [[trust]], supporting [[self-efficacy]], and ensuring that interaction with robots supports rather than undermines autonomy and social learning. ### How HRI appears in the wiki's research - **Trust and task context:** [[task-context-trust-educational-hri-2026|Research on trust in educational HRI]] shows that task context shapes trust in social robots more than appearance, with the highest trust during instructional tasks. - **Autonomy and sense of agency:** [[human-autonomy-agency-hri-review-2025|A systematic review]] synthesizes how HRI affects human autonomy and sense of agency, bridging design frameworks with regulatory demands (EU AI Act, IEEE Ethically Aligned Design). - **Natural language and perception:** [[icub-humanoid-storytelling-llm-hri-2025|The iCub narrative HRI study]] integrates perceptual capabilities with generative models (ChatGPT) for contextually appropriate social behaviour in co-creative storytelling. - **Language-driven control:** [[edusim-llm-robotic-simulation-education-2026|EduSim-LLM]] translates natural-language instructions into robot behaviour sequences, and [[roboblockly-conversational-block-robotics-ct-2026|RoboBlockly Studio]] uses a conversational AI agent alongside embodied robot execution. - **Sign-language interaction:** [[pepper-robot-sign-language-lis-2025|Pepper sign-language research]] explores robot communication with Deaf users in LIS. HRI connects to [[social-robots]], [[educational-robotics]], [[embodied-learning]], [[human-ai-collaboration]], [[agency]], [[trust]], [[well-being]], and [[ethics]]. It informs responsible design of educational robots and their evaluation. ## Connected Concepts - [[social-robots]] - [[educational-robotics]] - [[embodied-learning]] - [[human-ai-collaboration]] - [[agency]] - [[trust]] - [[well-being]] - [[ethics]] ## Connected Articles - [[task-context-trust-educational-hri-2026]] — Task Context and Trust in Educational HRI - [[human-autonomy-agency-hri-review-2025]] — Human Autonomy and Agency in HRI - [[icub-humanoid-storytelling-llm-hri-2025]] — iCub Narrative HRI - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM - [[pepper-robot-sign-language-lis-2025]] — Pepper and Sign Language - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education --- ## [Instructional Design with AI](https://edtechdev.github.io/aied/concepts/instructional-design/) > **Instructional Design** — the systematic process of creating effective learning experiences through the analysis of learning needs and the design, development, implementation, and evaluation of instructional materials and activities. AI is transforming instructional design by automating content creation, enabling adaptive learning paths, supporting data-driven iteration, and augmenting — rather than replacing — the instructional designer's role. Instructional design bridges AI capabilities and effective pedagogy. Where [[curriculum-design]] addresses *what* to teach at the program level, instructional design addresses *how* to teach it at the course and lesson level. The articles in this wiki explore both AI as a tool for instructional designers and instructional design principles for building effective AI tutoring systems. ### Key research themes **AI-assisted content creation** is the most directly transformative application. **[[curriculum-as-code-instructional-design-2026|Curriculum as Code]]** presents a six-phase architecture integrating Generative AI with LaTeX and Python to automate STEM materials creation, validated across 8 modules and 28 project contexts with student quality ratings of 8.5-9.9/10. **[[instructional-agents-multi-agent-course-gen|Instructional Agents]]** uses a multi-agent framework structured around the ADDIE model, with role-based agents (Teaching Faculty, Instructional Designer, Course Coordinator) collaborating to generate complete course materials. **[[courseblueprint-adaptive-video-generation|CourseBlueprint]]** provides a structured pipeline for adaptive pedagogical video generation grounded in course corpora, demonstrating that explicit pedagogical structure — not just AI fluency — is essential for educational content generation. **Pedagogically grounded AI tutoring** applies instructional design principles to AI system design. **[[didactical-teacher-assistant-dimensional-modeling|Brisson et al.]]** built a didactically-driven LLM teacher assistant where tutoring strategy is encoded in an explicit external layer — making content selection and didactic structuring traceable and reproducible, directly addressing opacity concerns in [[rethinking-scaffolding-llm-tutors]]. **[[instructional-guidance-genai-learning|Hou et al.]]** demonstrated that a five-step prompting framework grounded in Generative Learning Theory significantly improved higher-order cognitive outcomes, showing that instructional guidance — not just AI access — determines learning effectiveness. Both connect to [[scaffolding]] and [[intelligent-tutoring]]. **Frameworks and evaluation** provide structured approaches. **[[bridging-instructional-design-framework-math]]** and **[[cotal-formative-assessment-scoring-2026|CoTAL]]** demonstrate human-in-the-loop design principles. **[[genai-mindtool-generative-learning]]** positions AI as a "mindtool" — a cognitive partner that extends rather than replaces learner thinking — directly applying instructional design theory to AI integration. **[[ludia-udl-ai-thought-partner-2026|LUDIA]]** applies Universal Design for Learning principles to create an accessible AI thought partner for educators, connecting instructional design to [[accessible-learning]]. ### Connections to related concepts Instructional design is the bridge discipline of AI in education — it connects [[curriculum-design]] (what to teach) with [[scaffolding]] (how to support learners), [[faculty-development]] (how to prepare educators), and [[generative-ai]] (the tools themselves). It is tightly coupled with [[teacher-role]] because AI tools reshape what instructional designers and teachers do, and with [[ai-literacy]] because effective AI integration requires educators to understand AI capabilities and limitations. ## Connected Concepts - [[curriculum-design]] - [[scaffolding]] - [[faculty-development]] - [[teacher-role]] - [[ai-literacy]] - [[generative-ai]] - [[intelligent-tutoring]] - [[personalized-learning]] - [[adaptive-learning]] - [[formative-assessment]] - [[higher-ed]] - [[k-12]] - [[accessible-learning]] - [[universal-design-for-learning]] ## Connected Articles - [[curriculum-as-code-instructional-design-2026]] - [[instructional-agents-multi-agent-course-gen]] - [[didactical-teacher-assistant-dimensional-modeling]] - [[instructional-guidance-genai-learning]] - [[courseblueprint-adaptive-video-generation]] - [[bridging-instructional-design-framework-math]] - [[cotal-formative-assessment-scoring-2026]] - [[genai-mindtool-generative-learning]] - [[ludia-udl-ai-thought-partner-2026]] - [[learnity-graphs-lifelong-learning-framework-2026]] - [[pchl-he-framework-genai-content-creation-2026]] --- ## [Intelligent Tutoring](https://edtechdev.github.io/aied/concepts/intelligent-tutoring/) > **Intelligent Tutoring Systems (ITS)** — a well-established subfield of AI in education that uses AI to model student knowledge, adapt instruction, and provide personalized feedback, typically through model-tracing, knowledge tracing, and scaffolded problem-solving. ITS research predates the LLM era but has been transformed by generative AI, creating hybrid systems that combine structured domain models with flexible dialogue. Intelligent Tutoring Systems represent one of the oldest and most researched areas of AI in education. Unlike general-purpose [[ai-tutoring|AI tutoring]] systems that rely primarily on LLM capabilities, ITS traditionally use structured approaches: domain models (what to teach), student models (what the learner knows), and pedagogical models (how to teach). These components enable fine-grained tracking of student progress, misconception diagnosis, and adaptive sequencing. ### ITS vs. LLM-based tutoring The emergence of [[llm|LLMs]] has created a productive tension in the ITS field. Traditional ITS offer precision and transparency — you know exactly why the system made a particular decision — but lack flexibility. LLM tutors offer natural dialogue and broad knowledge but can hallucinate, over-scaffold, or bypass learning entirely. Modern research increasingly explores **hybrid approaches** that combine structured ITS components with LLM flexibility. Key research in the wiki includes: - **[[educlaw-bench-pedagogical-llm-agents-2026|EduClaw-Bench]]** evaluates pedagogical LLM agents using simulated learners grounded in knowledge tracing, finding that tutoring quality depends on both the base model and adapter design. - **[[codify-socratic-tutoring-programming|Codify]]** applies Socratic ITS principles to programming education, guiding students through incremental questions rather than providing solutions. - **[[lak2026-hint-button-unproductive-use|Hint button research]]** shows that traditional ITS hint design can inadvertently enable bypass strategies, calling for more sophisticated [[scaffolding]] approaches. - **[[deeptutor|DeepTutor]]** provides a fully open-source agentic tutoring framework with citation-grounded tutoring and difficulty-calibrated question generation. - **[[huang-interpretable-knowledge-tracing-2026|Interpretable Knowledge Tracing]]** addresses the opacity problem in ITS by producing interpretable cognitive quantities from LLM logits. ### Key ITS concepts - **[[knowledge-tracing]]** — modeling what a student knows over time (Bayesian, deep learning, IRT-based) - **[[student-modeling]]** — broader learner representation including affect, engagement, and misconceptions - **[[adaptive-learning]]** — systems that personalize content sequencing based on learner state - **[[scaffolding]]** — providing just enough support to enable progress without giving away answers - **productive-struggle** — letting students wrestle with difficulty rather than over-helping - **[[feedback-loop]]** — ITS feedback cycles that diagnose, guide, and verify ### Historical context The ITS field has produced landmark systems (Cognitive Tutors, Andes, AutoTutor) and continues to evolve. The [[zerkouk-comprehensive-review-its-2025|Zerkouk et al. comprehensive ITS review]] catalogs this evolution. The tension between structured ITS and open-ended LLM tutoring is explored in [[correct-answer-trap-ai-tutor|the correct answer trap]] research and [[rethinking-scaffolding-llm-tutors|rethinking scaffolding for LLM tutors]]. ## Connected Concepts - [[scaffolding]] - [[adaptive-learning]] - [[llm]] - [[learning-analytics]] - [[equity]] - [[personalized-learning]] - [[student-experience]] - [[self-regulated-learning]] - [[generative-ai]] - [[ai-education]] - [[metacognition]] - [[over-reliance]] ## Connected Articles - [[adaptive-virtual-patient-psychotherapy-training]] - [[agent-voice-accents-k12-group-learning]] - [[ai-coaching-rl-skill-development]] - [[ai-enabled-serious-games]] - [[ai-interlocutor-l2-spoken-dialogue]] - [[ai-lifelong-learning-policy]] - [[ai-lms-middle-school-longitudinal]] - [[ai-metacognition-stem-review]] - [[ai-pedagogical-accompaniment-amico]] - [[ai-stem-bibliometric-trends]] - [[automated-presentation-coaching]] - [[child-safety-genai]] - [[clara-collaboration-literacy-dashboard]] - [[codify-socratic-programming-tutor]] - [[codify-socratic-tutoring-programming]]- [[ai-tutoring-quality-k12-methodologies-2026]] --- ## [Item Response Theory](https://edtechdev.github.io/aied/concepts/item-response-theory/) > **Item response theory** — psychometric modeling of item difficulty and ability — meets LLMs in [[llm-item-difficulty-prediction]], [[llm-psychometric-calibration-cdp]], and [[knowledge-tracing-irt]]: AI predicts difficulty and calibrates assessment, improving measurement precision ([[educational-measurement]], [[assessment-validity]]). ## Connected Concepts - [[knowledge-tracing]] - [[student-modeling]] - [[ai-tutoring]] ## Connected Articles - [[multimodal-item-parameter-estimation-2026]] - [[huang-interpretable-knowledge-tracing-2026]] - [[llm-difficulty-calibration-programming-exams-2026]] --- ## [K-12 AI Education](https://edtechdev.github.io/aied/concepts/k-12-ai-education/) K-12 AI Education encompasses the integration of artificial intelligence literacy, tools, and pedagogical approaches into primary and secondary education. Recent research reveals three critical pillars: ### 1. Teacher Preparation Gap Teachers systematically overestimate their AI competency (40% gap between self-report and performance in Zhang et al. 2026), yet brief training interventions (4-hour prompting workshops) yield 2.3x higher classroom AI integration (Xiao et al. 2026). ### 2. Cultural Relevance Imperative LLM-supported curriculum design shows promise for diversifying materials — 78% of teachers found AI suggestions helpful for culturally relevant pedagogy (Wang et al. 2025). However, most AI tools center dominant perspectives, requiring deliberate equity-centered design. ### 3. Policy-to-Practice Translation Institutional GenAI policies largely lack implementation guidance. Successful models transform policy documents into actionable teacher training modules, bridging the "what" (policy) and "how" (prompting instruction). ## References Xiao, R., Ye, R., et al. (2026). *Transforming GenAI Policy to Prompting Instruction: An RCT*. arXiv:2602.16033. Zhang, S., Xiao, R., et al. (2026). *How to Assess AI Literacy: Misalignment Between Self-Reported and Performance*. arXiv:2601.06101. Wang, J., Xiao, R., et al. (2025). *LLMs to Support K-12 Teachers in Culturally Relevant Pedagogy*. arXiv:2505.08083. ## Source - https://arxiv.org/abs/2602.16033 - [https://arxiv.org/abs/2601.06101](https://arxiv.org/abs/2601.06101) - [https://arxiv.org/abs/2505.08083](https://arxiv.org/abs/2505.08083) ## Connected Concepts - [[ai-literacy]] - [[zone-of-proximal-development]] - [[metacognition]] - [[rct]] - [[scaffolding]] - [[faculty-development]] - [[instructional-design]] - [[active-learning]] - [[computational-thinking]] - [[cognitive-load-theory]] - [[ai-tutoring]] - [[collaborative-learning]] ## Connected Articles - [[computational-thinking-ai-agent-creation]] - [[hingle-collaborative-ai-literacy-2025]] - [[stanford-evidence-base-ai-k12-2026]] --- ## [K-12 AI Education](https://edtechdev.github.io/aied/concepts/k-12/) > **K-12 AI education** — the use of artificial intelligence in primary and secondary education, spanning AI literacy curricula, AI tutoring, teacher support, and safety considerations unique to younger learners. ### Distinctive K-12 considerations - **Safety and guardrailing:** K-12 AI use demands stronger [[pedagogical-safety]] protections. [[eduzone-llm-safety-k12|EduZone]], [[eduguard-safe-rag-llm-tutor|EduGuard]], and [[ai-tutor-safety-harms|tutor harm research]] specifically address child-safe AI interaction. - **AI literacy curricula:** [[aaai2026-prompting-literacy-k12|K-12 AI literacy modules]] and [[brookings-ai-students-report|policy frameworks]] develop age-appropriate AI understanding. The [[gaide-vibe-coding-k12-teachers|vibe coding framework]] empowers teachers to create their own AI tools. - **Evidence base:** [[stanford-evidence-base-ai-k12-2026|The K-12 AI evidence base]] aggregates what works at scale, with [[transfer-of-learning]] as the critical open question about whether AI-assisted gains persist. - **Tutoring at grade level:** [[ecnuclaw-k12-personalized-companion|K-12 personalized companions]] and [[correct-answer-trap-ai-tutor|correct answer trap research]] examine tutoring effectiveness for younger students. - **Developmental appropriateness:** [[child-safety-genai|Child safety research]] and [[special-education]] considerations address the needs of diverse K-12 populations. ### Scale and equity K-12 AI deployment operates at societal scale — millions of students, compulsory education, and significant equity implications. [[equity]] and [[k-12]] research examines whether AI widens or narrows opportunity gaps. ### Connections K-12 connects to [[pedagogical-safety]] (child protection), [[ai-literacy]] (student competency), [[teacher-role]] (K-12 teacher transformation), [[equity]] (access gaps), and [[special-education]] (diverse learner needs). ## Connected Concepts - [[pedagogical-safety]] - [[ai-literacy]] - [[teacher-role]] - [[equity]] - [[special-education]] - [[higher-ed]] - [[scaffolding]] - [[personalized-learning]] - [[stem-education]] - [[llm]] - [[generative-ai]] ## Connected Articles - [[eduzone-llm-safety-k12]] - [[eduguard-safe-rag-llm-tutor]] - [[ai-tutor-safety-harms]] - [[aaai2026-prompting-literacy-k12]] - [[brookings-ai-students-report]] - [[stanford-evidence-base-ai-k12-2026]] - [[ecnuclaw-k12-personalized-companion]] - [[gaide-vibe-coding-k12-teachers]] - [[elementary-writing-genai-systematic-review-2026]] --- ## [Knowledge Graph](https://edtechdev.github.io/aied/concepts/knowledge-graph/) > **Knowledge graph** — a structured representation of concepts and their relationships used to model domain knowledge, student understanding, and learning dependencies in AI in education systems. Knowledge graphs enable AI systems to reason about what students know, what they need to learn next, and how concepts relate to each other. Knowledge graphs provide the structural backbone for many intelligent education systems. Unlike flat lists of skills or concepts, knowledge graphs capture prerequisite relationships, similarity, and hierarchical organization — essential for [[adaptive-learning]], [[knowledge-tracing]], and [[student-modeling]]. ### How knowledge graphs are used in AIED - **[[knowledge-tracing]]** models use concept graphs to propagate student proficiency estimates across related skills, improving prediction accuracy when data is sparse - **[[student-modeling]]** systems leverage knowledge graphs to represent what learners know in a semantically meaningful way, enabling fine-grained diagnosis - **[[adaptive-learning]]** platforms use prerequisite graphs to sequence content and recommend personalized learning paths - **[[cognitive-diagnosis]]** frameworks like [[xie-hillm-cd-2026|HiLLM-CD]] construct concept trees from educational text using LLMs, eliminating manual annotation ### LLM-driven knowledge graph construction Recent research explores using [[llm|LLMs]] to automatically construct knowledge graphs from educational content. The [[xie-hillm-cd-2026|HiLLM-CD]] framework uses multi-agent LLM pipelines to generate exercise-concept links and hierarchical concept trees, reducing reliance on expert annotation. This connects to broader [[generative-ai]] applications in curriculum design and automated content organization. ### Related concepts Knowledge graphs connect to [[instructional-design]] (defining what to teach), [[curriculum-design]] (how to sequence it), and [[learning-analytics]] (extracting insights from student interaction data). They are foundational to [[intelligent-tutoring]] systems that need structured representations of educational domains. ## Connected Concepts - [[adaptive-learning]] - [[knowledge-tracing]] - [[intelligent-tutoring]] - [[cognitive-diagnosis]] - [[student-modeling]] - [[learning-analytics]] - [[knowledge-graph]] - [[curriculum-design]] ## Connected Articles - [[learnity-graphs-lifelong-learning-framework-2026]] - [[veriforge-narrative-drafting-scaffolding-2026]] --- ## [Knowledge Tracing](https://edtechdev.github.io/aied/concepts/knowledge-tracing/) > **Knowledge tracing** — modeling what learners know over time by tracking their performance on exercises and predicting future mastery. It is the wiki's richest modeling thread, spanning Bayesian, deep learning, and LLM-enhanced approaches to tracking student knowledge as it evolves. Knowledge tracing transforms raw exercise responses into estimates of what a student has mastered and what they still need to learn. Unlike simple correctness tracking, knowledge tracing models the temporal dynamics of learning — when knowledge is gained, when it decays, and how concepts relate to each other. ### Approaches represented in the wiki - **Bayesian approaches:** [[stanbkt-bayesian-knowledge-tracing]] standardizes BKT implementations, while [[mbp-kt-meta-behavioral-knowledge-tracing]] incorporates meta-behavioral signals - **Neural and hybrid models:** [[neural-symbolic-knowledge-tracing]] combines symbolic reasoning with neural networks; [[explainable-probabilistic-kt]] advances interpretable probabilistic models - **Dialogue-based KT:** [[difficulty-aware-dialogue-kt]] adapts knowledge tracing for conversational tutoring - **LLM-enhanced:** [[xie-hillm-cd-2026|HiLLM-CD]] uses LLMs for automated concept tree construction and hierarchical proficiency inference - **Cold-start problem:** [[cold-start-knowledge-tracing-safeinsights]] examines model accuracy when students begin learning new skills ### Relationship to other concepts Knowledge tracing is closely related to [[student-modeling]] — while knowledge tracing specifically models cognitive knowledge over time, student modeling is the broader practice of representing all aspects of a learner (affective state, engagement, preferences). Knowledge tracing feeds into [[adaptive-learning]] and [[personalized-learning]] systems that need to know what to teach next, and into [[intelligent-tutoring]] platforms that use mastery estimates to select appropriate problems. It connects to [[learning-analytics]] for dashboard and intervention design, and to [[cognitive-diagnosis]] for fine-grained skill assessment. ## Connected Concepts - [[student-modeling]] - [[adaptive-learning]] - [[personalized-learning]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[formative-assessment]] - [[ai-education]] - [[ai-ed-evaluation]] - [[multimodal]] - [[teacher-role]] - [[over-reliance]] - [[llm]] ## Connected Articles - [[multimodal-item-parameter-estimation-2026]] - [[educlaw-bench-pedagogical-llm-agents-2026]] - [[huang-interpretable-knowledge-tracing-2026]] - [[learning-engagement-assistant-lea]] - [[llm-cognitive-diagnosis-handwritten-math]] - [[multimodal-knowledge-graph-educational-reasoning]] - [[pattern-kc-programming-recommendation]] - [[proprl-prerequisite-relation-learning]] - [[reinforcement-learning-measurement-model-assessment]] - [[skill-acquisition-without-temporal-info]] - [[xie-hillm-cd-2026]] - [[zerkouk-comprehensive-review-its-2025]] --- ## [Language Learning](https://edtechdev.github.io/aied/concepts/language-learning/) > **Language Learning** — the study of how AI supports second language (L2) acquisition, writing development, and linguistic diversity in educational settings. AI in education research in this wiki spans AI interlocutors for spoken dialogue, automated writing evaluation for L2 learners, reading support, and concerns about language bias in AI scoring systems. Language learning has emerged as a significant AI in education domain because language is inherently interactive — making it well-suited to conversational AI — and because AI's linguistic capabilities raise both opportunities (personalized language practice at scale) and risks (systematic bias against non-native language patterns). The articles in this wiki explore both sides of this equation. **AI as language tutor and interlocutor** is the most developed theme. **[[ai-interlocutor-l2-spoken-dialogue|What Changes When the Interlocutor Is an AI?]]** examines interactional fluency and linguistic uptake when L2 learners converse with AI versus humans. **[[tact-pedagogically-adaptive-esl-tutoring|TACT]]** provides pedagogically adaptive ESL tutoring. **[[llm-children-reading-story-generation]]** explores AI-generated stories for children's reading development. These connect to [[intelligent-tutoring]] and [[generative-ai]]. **Automated writing evaluation for L2 learners** evaluates AI's ability to assess non-native writing. **[[self-referential-l2-writing-llm-assessment|Bannò et al.]]** proposed a self-referential approach comparing student writing to their own prior work rather than native-speaker norms. **[[ai-scoring-language-bias-physics|Feser & Tschisgale]]** found AI scoring systematically underestimates linguistically weak students — a finding that connects to [[assessment-validity]] and [[bias-mitigation]] concerns. **[[genai-linguistic-diversity-academic-writing]]** explores how AI affects linguistic diversity in academic contexts. **Accessibility for language learners** connects to [[accessible-learning]]: **[[dyslexlens-dyslexic-learners-ai|DysLexLens]]** analyzed how dyslexic learners use AI for literacy support, and **[[ai-tools-arab-english-classrooms]]** explored AI tools in Arabic-English classroom contexts. These studies connect language learning to [[equity]] and [[special-education]]. ## Connected Concepts - [[writing-education]] - [[ai-literacy]] - [[equity]] - [[assessment-validity]] - [[bias-mitigation]] - [[accessible-learning]] - [[special-education]] - [[intelligent-tutoring]] - [[generative-ai]] - [[student-experience]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] - [[ai-interlocutor-l2-spoken-dialogue]] - [[self-referential-l2-writing-llm-assessment]] - [[ai-scoring-language-bias-physics]] - [[genai-linguistic-diversity-academic-writing]] - [[dyslexlens-dyslexic-learners-ai]] - [[tact-pedagogically-adaptive-esl-tutoring]] - [[ai-tools-arab-english-classrooms]] - [[structural-silence-underrepresented-language-ai-2026]] --- ## [Learning Analytics](https://edtechdev.github.io/aied/concepts/learning-analytics/) > **Learning analytics** — the measurement, collection, analysis, and reporting of data about learners and their contexts for the purpose of understanding and optimizing learning. AI has transformed learning analytics from descriptive dashboards to predictive and prescriptive systems. ### AI-enhanced analytics - **Predictive analytics:** Machine learning on learner interaction data predicts outcomes — from [[at-risk-students-ml-prediction|at-risk identification]] to [[knowledge-tracing|knowledge state estimation]]. - **Engagement analytics:** [[engagement-metrics|Engagement measurement]] and [[engagement-intensity-learner-modeling|intensity modeling]] quantify how students interact with AI systems. - **Feedback analytics:** [[teaching-feedback-classification-benchmark|Feedback classification]] and [[ai-feedback-quality|quality assessment]] analyze the feedback students receive. - **Network analysis:** [[misiejuk-cognitive-offloading-prompting-2026|Co-Occurrence Network Analysis]] and [[epistemic-emotions-collaborative-problem-solving|epistemic network analysis]] reveal interaction patterns. - **Privacy tensions:** [[privacy]] and [[privacy]] concerns grow as analytics become more granular and AI-driven. ### From description to intervention Learning analytics has evolved through three generations in the wiki: descriptive (what happened?), predictive (what will happen?), and prescriptive (what should we do?). AI enables the prescriptive layer — analytics that directly trigger [[feedback-loop|instructional interventions]]. ### Connections Learning analytics connects to [[knowledge-tracing]] (the core analytic), [[formative-assessment]] (analytics-driven assessment), [[student-modeling]] (the learner representation analytics populate), [[privacy]] (the ethical constraint), and [[edtech-platform]] (where analytics are deployed). ## Connected Concepts - [[knowledge-tracing]] - [[student-modeling]] - [[formative-assessment]] - [[privacy]] - [[edtech-platform]] - [[engagement-metrics]] - [[ai-ed-evaluation]] - [[feedback-loop]] - [[higher-ed]] - [[k-12]] - [[llm]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] - [[at-risk-students-ml-prediction]] - [[engagement-intensity-learner-modeling]] - [[misiejuk-cognitive-offloading-prompting-2026]] - [[teaching-feedback-classification-benchmark]] - [[wordstream-glass-learning-analytics]] - [[llm-difficulty-calibration-programming-exams-2026]] --- ## [Learning Gains](https://edtechdev.github.io/aied/concepts/learning-gains/) > **Learning gains** — measurable improvements in student knowledge, skills, or competencies resulting from educational interventions, including AI-assisted instruction. In AI in education research, learning gains serve as the primary outcome measure for evaluating whether AI tools actually improve learning — not just engagement or satisfaction. Learning gains are the ultimate test of any educational technology. In the wiki's research, they appear as dependent variables in [[rct|randomized controlled trials]], pre-post comparisons in quasi-experimental studies, and correlational analyses linking AI tool usage to academic outcomes. Key findings from the wiki: - **[[adaptive-pretesting-retention|Adaptive pretesting]]** research examines whether GenAI-enabled pretesting produces durable learning gains that persist beyond immediate testing. - **[[genai-meta-analysis-programming-learning|Meta-analyses of GenAI in programming]]** find positive learning gains from structured AI use but negative effects from unguided reliance. - **[[lak2026-hint-button-unproductive-use|Hint button research]]** shows negative associations between hint abuse and learning gains — more hints correlate with less learning. - **[[instructional-guidance-genai-learning|Instructional guidance]]** studies demonstrate that learning gains depend on HOW AI is used, not just WHETHER it's available. ### Measuring what matters Learning gains connect to [[assessment-validity]] — if assessments fail to capture deeper understanding, learning gain measures are misleading. They also intersect with [[over-reliance]] and [[cognitive-offloading]], where apparent performance improvements may mask learning losses. ## Connected Concepts - [[rct]] - [[learning-gains]] - [[formative-assessment]] - [[over-reliance]] - [[cognitive-offloading]] - [[math-education]] - [[human-in-the-loop-ai]] - [[affective-tutoring]] ## Connected Articles - [[ai-literacy-assessment-misalignment]] — AI Literacy Assessment: Self-Reported vs Performance Misalignment - [[generative-ai-reduced-study-time-math]] — Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build - [[genai-meta-analysis-programming-learning]] — A meta-analysis of the effect of generative AI on productivity and learning in programming - [[absent-cognitive-baseline-2026]] — The Absent Cognitive Baseline: Theorizing a Structural Gap in AI-Native College Students' Academic Self-Assessment - [[oecd-digital-education-outlook-2026]] — OECD Digital Education Outlook 2026 - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows --- ## [Lifelong Learning and AI](https://edtechdev.github.io/aied/concepts/lifelong-learning/) > Stub — pending source ingestion. Lifelong learning and AI support for continuous education beyond formal schooling. ## Connected Concepts - [[faculty-development]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[equity]] - [[regulation]] - [[over-reliance]] - [[teacher-role]] - [[personalized-learning]] - [[metacognition]] - [[scaffolding]] - [[professional-training]] - [[self-regulated-learning]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] - [[ai-adult-learning-design]] - [[ai-lifelong-learning-policy]] - [[community-centered-ai-education-adults]] - [[metacognitive-learning-scenarios-taxonomy]] - [[learnity-graphs-lifelong-learning-framework-2026]] --- ## [Large Language Models (LLMs)](https://edtechdev.github.io/aied/concepts/llm/) > **Large Language Models (LLMs)** — neural network models trained on vast text corpora that generate human-like text, powering most modern AI in education applications. LLMs are the computational backbone of generative AI tutoring, assessment, and content generation in education. ### LLMs as the engine of AIED LLMs are the most-referenced concept in the wiki (60+ articles) because they underpin nearly every AI education application: - **Tutoring:** [[ai-tutoring|AI tutors]] use LLMs for dialogue, explanation, and problem-solving guidance. [[pedagogical-llm-training|Pedagogical training]] adapts general LLMs for educational use. - **Assessment:** [[automated-grading|Grading systems]], [[automated-essay-scoring|essay scoring]], and [[llm-item-difficulty-prediction|item difficulty prediction]] leverage LLM capabilities. - **Content:** [[generative-ai|Generative AI]] content creation relies on LLMs. [[automated-question-generation|Question generation]] and [[ai-generated-instructional-videos-computing-ed|video generation]] are LLM-driven. - **Safety:** [[pedagogical-safety]], [[hallucination-risk]], and [[ai-tutor-safety-harms]] research examine LLM-specific risks. - **Diagnosis:** [[knowledge-tracing]] and [[cognitive-diagnosis]] increasingly incorporate LLMs for richer student modeling. ### Model-specific research The wiki covers both general-purpose LLMs (GPT-4, Claude) and education-specific adaptations. [[cstutorbench-slm-tutors|Small language model benchmarks]] compare SLM performance for tutoring. [[educational-llm-alignment|Educational alignment]] research addresses how to make LLMs pedagogically appropriate. ### Connections LLMs connect to [[generative-ai]] (the broader category), [[prompt-engineering]] (how outputs are controlled), [[rag]] (knowledge grounding), [[hallucination-risk]] (the primary limitation), and [[pedagogical-safety]] (educational guardrailing). ## Connected Concepts - [[generative-ai]] - [[prompt-engineering]] - [[rag]] - [[hallucination-risk]] - [[pedagogical-safety]] - [[ai-tutoring]] - [[automated-grading]] - [[ai-literacy]] - [[knowledge-tracing]] - [[higher-ed]] - [[scaffolding]] ## Connected Articles - [[multimodal-item-parameter-estimation-2026]] - [[pedagogical-llm-training]] - [[educational-llm-alignment]] - [[cstutorbench-slm-tutors]] - [[ai-tutor-safety-harms]] - [[llm-item-difficulty-prediction]] - [[eduguard-safe-rag-llm-tutor]] - [[llm-intervention-design-cs-review]] - [[llm-difficulty-calibration-programming-exams-2026]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] - [[elbench-education-llm-benchmark-2026]] - [[ai-feedback-enactment-workflow-2026]] --- ## [Math Education](https://edtechdev.github.io/aied/concepts/math-education/) > **Math Education** — the study of how students learn mathematics and how AI can support mathematics teaching, spanning affective tutoring, cognitive diagnosis from handwritten work, productive struggle evaluation, help-seeking behavior, teacher-AI collaboration for visual generation, and student-AI interaction trajectories. Math education is the most active domain-specific research area in this wiki, with 10 articles that collectively explore how AI can support — and sometimes undermine — mathematical learning from elementary fractions through higher education. Mathematics education has become a primary domain for AI in education research because math problems have clear right answers yet require rich reasoning — making them ideal for studying tutoring effectiveness, assessment validity, and how AI tools interact with student cognition and affect. The articles in this wiki reveal both the promise of AI math tutors and persistent challenges: over-scaffolding that undermines productive struggle, hallucination in cognitive diagnosis, and the difficulty of balancing AI assistance with genuine learning. ### Key research themes **AI math tutoring and scaffolding** is the largest cluster, with four articles examining how AI tutors support or undermine math learning. **[[kar-mathbuddy-affective-math-tutoring-2025|MathBuddy]]** demonstrates that adding affective awareness — detecting student emotions from text and facial expressions — produces a +23-point win rate advantage in math tutoring, connecting to [[affective-computing]] and [[affective-tutoring]]. **[[zhang-tutormoments-2026|TutorMoments]]** evaluates 462 teacher-annotated transcripts from grades 2-7 math tutoring and finds frontier models default toward over-helpfulness, rarely pushing for rigor even when students are ready — directly challenging the alignment between AI helpfulness and [[scaffolding]] principles. **[[lak2026-hint-button-unproductive-use|An et al.]]** analyzed 999 students across three semesters in the *Decimal Point* ITS, finding that premature hint requests and superficial hint reading consistently predict reduced learning gains, even after controlling for prior knowledge — a finding that connects to [[help-seeking]] and [[learning-analytics]]. **Cognitive diagnosis and assessment** explores AI's ability to evaluate math thinking. **[[llm-cognitive-diagnosis-handwritten-math|MathCog]]** benchmarked 18 LLMs on 3,036 teacher-annotated diagnostic verdicts from handwritten math work, finding all models severely underperform (F1 < 0.5) with systematic over-attribution and hallucination of evidence — connecting to [[knowledge-tracing]], [[hallucination-risk]], and [[multimodal]] assessment challenges. **[[representation-robustness-llm-math-problem-solving|Nath et al.]]** showed that LLM math problem-solving is highly sensitive to surface representation — models flip correctness across equivalent problem formulations — raising [[assessment-validity]] concerns for AI-based math scoring. **Student engagement and AI literacy** examines how students interact with AI math tools. **[[epistemic-proactivity-math|Abdelghani et al.]]** traced temporal trajectories of student-AI interaction in math learning, identifying a developmental path from superficial prompting to "epistemic proactivity" — active, self-directed pursuit of conceptual understanding. This connects to [[ai-literacy]], [[metacognition]], and [[self-regulated-learning]]. **[[ai-powered-personalized-learning-elementary-fractions-2026|Holman]]** found that AI-adaptive platforms significantly improved fraction comprehension for students with math learning difficulties, connecting to [[personalized-learning]] and [[adaptive-learning]]. **Teacher support** explores AI tools for math educators. **[[teacher-control-ai-generation-math-visuals|Li et al.]]** investigated when teachers should control AI generation of math visuals, proposing a framework balancing AI efficiency with pedagogical correctness — connecting to [[teacher-role]] and [[curriculum-design]]. **[[ai-tpack-preservice-math-teachers|Egara et al.]]** examined AI-TPACK readiness among preservice math teachers, connecting to [[faculty-development]]. **Higher education math** explores AI's impact on advanced math practice. **[[genai-runaway-object-math-higher-ed|Bui et al.]]** applied socio-cultural theory to GenAI in university mathematics, analyzing AI as a "runaway object" that transforms academic practice in ways that outpace institutional and pedagogical norms. ### Connections to related concepts Math education sits within the broader [[stem-education]] domain with distinctive connections to [[intelligent-tutoring]] and [[ai-tutoring]] through the strong tradition of cognitive tutors and ITS research in mathematics, to [[scaffolding]] through the productive struggle and hint-use literature, to [[affective-computing]] through math anxiety and emotion-aware tutoring, to [[knowledge-tracing]] and [[assessment-validity]] through cognitive diagnosis and assessment research, and to [[teacher-role]] through teacher-AI collaboration in math instruction. The [[k-12]] connection is particularly strong — 8 of 10 math articles involve K-12 contexts — while [[higher-ed]] connections emerge in teacher preparation and advanced math practice. ## Connected Concepts - [[stem-education]] - [[intelligent-tutoring]] - [[ai-tutoring]] - [[scaffolding]] - [[affective-computing]] - [[affective-tutoring]] - [[k-12]] - [[higher-ed]] - [[student-experience]] - [[ai-literacy]] - [[metacognition]] - [[self-regulated-learning]] - [[personalized-learning]] - [[adaptive-learning]] - [[help-seeking]] - [[learning-analytics]] - [[knowledge-tracing]] - [[assessment-validity]] - [[multimodal]] - [[hallucination-risk]] - [[over-reliance]] - [[teacher-role]] - [[faculty-development]] - [[generative-ai]] - [[open-source]] ## Connected Articles - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[zhang-tutormoments-2026]] - [[lak2026-hint-button-unproductive-use]] - [[llm-cognitive-diagnosis-handwritten-math]] - [[representation-robustness-llm-math-problem-solving]] - [[epistemic-proactivity-math]] - [[ai-powered-personalized-learning-elementary-fractions-2026]] - [[teacher-control-ai-generation-math-visuals]] - [[ai-tpack-preservice-math-teachers]] - [[genai-runaway-object-math-higher-ed]] --- ## [Metacognition](https://edtechdev.github.io/aied/concepts/metacognition/) > Metacognition — thinking about one's own thinking — is both a target of AI education research (can AI tools develop students' metacognitive skills?) and a risk factor (AI completing tasks may suppress metacognitive practice).^[[stanford-evidence-base-ai-k12-2026]]^[[scheu-mobile-chatbot-journaling-motivation-2026]] Metacognition in education refers to learners' awareness, monitoring, and regulation of their own cognitive processes: - **Metacognitive knowledge:** Understanding what one knows, what strategies are available, and when to deploy them - **Metacognitive regulation:** Planning, monitoring, and evaluating one's own learning in real time Within [[self-regulated-learning]] frameworks, metacognition is the central mechanism that enables learners to adapt strategies, recognize confusion, and seek help appropriately.^[[scheu-mobile-chatbot-journaling-motivation-2026]] ## How AI Tools Affect Metacognition ### The Suppression Risk (Stanford SCALE, 2026) When AI completes reasoning tasks for students — solving math problems, writing essays, generating code — the student loses practice in monitoring their own understanding and selecting strategies.^[[stanford-evidence-base-ai-k12-2026]] Key findings: - **Kosmyna et al. (2025):** Students who used AI essay assistance were **83% unable to recall quotes** from their own essays, vs. 11% for non-AI users — indicating they did not engage with the content during production. - **Stadler et al. (2024):** General-purpose AI reduced cognitive load but produced **lower-quality reasoning** vs. traditional search, suggesting metacognitive engagement was displaced. - **Lehmann et al. (2025):** General AI for programming harmed understanding for low-prior-knowledge students — the students most in need of metacognitive scaffolding received answers instead. ### The Augmentation Opportunity (Scheu et al., 2026) When AI is designed to support reflection rather than replace it, metacognition can be strengthened: - **Learning journals** are a classic metacognitive practice: by reflecting on learning processes, students increase awareness of their cognition - **Structured prompts** that ask students to self-explain, evaluate strategies, or identify knowledge gaps preserve metacognitive demand - The **example-based course** in Scheu et al.'s chatbot increased **perceived competence** (a metacognitive self-evaluation) even when the LLM assistant alone did not ## The Engagement–Motivation Distinction Scheu et al. (2026) found a critical split: | Dimension | LLM Assistant Effect | Course Effect | |---|---|---| | **Intrinsic motivation** (willingness to engage) | **No effect** | **Positive** | | **Behavioral engagement** (amount written) | **Increasing over time** (feedback loop) | **Constant positive** | This suggests that **metacognitive support and motivation are not identical**. The LLM assistant's scaffolding of journal entries increased how much students wrote (behavioral engagement) but did not make them *want* to write more (intrinsic motivation).^[[scheu-mobile-chatbot-journaling-motivation-2026]] ## Implications for Tool Design 1. **Preserve the "friction" of thinking:** If AI writes the reflection, the student does not build metacognitive skill. Journaling assistants should scaffold, not author. 2. **Model metacognitive language:** The example-based course worked partly because it exposed students to proficient models' metacognitive self-talk. 3. **Separate support for motivation vs. skill:** Metacognitive skill development (course-structured) and productivity enhancement (AI-assisted) may require different design strategies. ## Connected Concepts - [[self-regulated-learning]] - [[ai-literacy]] - [[scaffolding]] - [[over-reliance]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[student-experience]] - [[formative-assessment]] - [[cognitive-offloading]] - [[prompt-engineering]] - [[agentic-ai]] - [[human-in-the-loop-ai]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[ai-adoption-training-public-sector]] - [[stanford-evidence-base-ai-k12-2026]] - [[ai-learning-companions-framework]] - [[ai-making-us-stupid]] - [[ai-metacognition-stem-review]] - [[ai-powered-materials-discovery-ai-literacy]] - [[ai-tutor-safety-harms]] - [[aied-unfinished-mission-bypass]] - [[ase-26-agentic-software-engineering-curriculum]] - [[authentic-assessment]] - [[buggy-genai-code-student-responses]] - [[chatgpt-critical-creative-thinking-review]] - [[code-review-genai-cs1]] - [[codify-socratic-programming-tutor]] - [[cognitive-offloading-speedup-illusion]] --- ## [Motivation](https://edtechdev.github.io/aied/concepts/motivation/) > **Motivation** — the psychological processes that initiate, direct, and sustain goal-directed behavior. In AI in education, motivation research examines how AI tools affect learners' and teachers' motivation — whether AI scaffolds or undermines persistence, curiosity, and intrinsic engagement — and how motivational states shape the effectiveness of AI-mediated learning. Motivation is a foundational construct in education research, and the rise of AI in education has made it more consequential: AI tools can remove friction and make learning more accessible, but they can also reduce the cognitive effort and struggle that support intrinsic motivation and deep learning. The articles in this wiki explore motivation across learner-facing AI tools, teacher-facing AI systems, and the psychological mechanisms — [[self-determination-theory|self-determination]], [[self-efficacy-tutoring-learning|self-efficacy]], emotions — through which AI shapes motivated behavior. ### Key research themes **AI effects on student motivation** is the most direct line of research. **[[ai-availability-student-motivation]]** examines how the availability of AI assistance affects student motivation and persistence, connecting to [[over-reliance]] research on motivation erosion when AI does the work. **[[scheu-mobile-chatbot-journaling-motivation-2026]]** explores mobile chatbot journaling as a motivational intervention. **[[ai-learning-tools-engineering-education-needs]]** examines what motivates students to adopt AI learning tools in engineering education. **Motivation in AI-mediated engagement** examines how motivational quality (not just quantity) changes with AI. **[[students-engagement-with-generative-ai-in-academic-learning-a-self-determination|Isaeva et al.]]** combined self-determination theory with epistemic network analysis to study engagement with generative AI. **[[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2|Wang & Pang]]** found the motivational effects of emotional AI on L2 pre-service teachers are not universal — they vary by individual and context. **Teacher motivation and persistence** examines motivation among educators. **[[framing-5-percent-problem-teachers-persistence|Framing the 5 Percent Problem]]** studies teacher persistence with AI tools, and **[[teacher-education-ai-literacy-sdt-2026|Chiu et al.]]** found need-supportive professional development fosters sustained behavioral engagement in professional learning communities. ### Connections to related concepts Motivation is the parent construct of [[self-determination-theory]], which specifies the psychological needs (autonomy, competence, relatedness) that sustain intrinsic motivation. It connects to [[student-experience]] as the experiential layer of motivated engagement, to [[engagement-metrics]] as its measurable dimension, and to [[affective-computing]] for the emotional mechanisms that shape motivation. Motivation also connects to [[over-reliance]] (AI reducing productive struggle), [[self-regulated-learning]] (motivated learners self-regulate), and [[teacher-role]] (motivation applies to educators as well as students). ## Connected Concepts - [[self-determination-theory]] - [[student-experience]] - [[engagement-metrics]] - [[affective-computing]] - [[affective-tutoring]] - [[over-reliance]] - [[self-regulated-learning]] - [[self-efficacy-tutoring-learning]] - [[teacher-role]] - [[ai-education]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[ai-availability-student-motivation]] - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] - [[teacher-education-ai-literacy-sdt-2026]] - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] - [[scheu-mobile-chatbot-journaling-motivation-2026]] - [[framing-5-percent-problem-teachers-persistence]] - [[from-emotion-regulation-to-academic-success-a-self-determination-theory-based-em]] --- ## [Multimodal](https://edtechdev.github.io/aied/concepts/multimodal/) > **Multimodal** — a key concept in AI in education research. Explored across 3 articles in this wiki. Research themes include: - Systematic study of domain-adapted text-to-image models for nuclear engineering education. Fine-tunes Stable Diffusion on nuclear domain images; fine-tuned model achieves 78% domain accuracy vs 12% for base model. Proposes NuclearDiffusion as an educ - Pilot study on privacy-aware computer vision for classroom incident detection. Introduces a hybrid benchmark combining generative CCTV-style videos with real classroom pose data. Proposes a lightweight motion reasoning model that achieves strong inci ## Connected Concepts - [[generative-ai]] - [[edtech-platform]] - [[stem-education]] - [[privacy]] - [[teacher-role]] - [[over-reliance]] - [[scaffolding]] - [[personalized-learning]] - [[intelligent-tutoring]] - [[knowledge-tracing]] - [[adaptive-learning]] - [[hallucination-risk]] ## Connected Articles - [[multimodal-item-parameter-estimation-2026]] - [[ai-guided-learning-audiovideo-2026]] --- ## [Neurodiversity](https://edtechdev.github.io/aied/concepts/neurodiversity/) > **Neurodiversity** — the framing that neurological differences such as autism, ADHD, dyslexia, and dyspraxia are natural variations in human cognition rather than deficits to be corrected. In education, a neurodiversity-affirming approach designs learning environments that accommodate and leverage these differences rather than forcing conformity to a single cognitive norm. The neurodiversity paradigm shifts the goal of special-education and accessibility work from "fix the learner" to "adapt the environment." It overlaps with [[universal-design-for-learning]] and [[accessible-learning]] but emphasizes affirming identity and strength-based design over accommodation-as-compensation. ### Neurodiversity in the AI era Generative AI offers both promise and risk for neurodivergent learners. On the promise side, AI can provide alternative means of engagement, representation, and expression — supporting learners who struggle with conventional text, executive function, or social expectations — and can reduce [[cognitive-load-theory|cognitive load]] through [[personalized-learning]]. On the risk side, AI tools that assume a dominant communication style, or that encourage dependency, can disadvantage or undermine neurodivergent learners. Research on [[student-experience]] and [[ai-misuse-learning-harm]] suggests AI must be designed inclusively or it recapitulates [[equity]] gaps. Understanding a learner's neurotype is also essential for interpreting behavior signals in [[learning-analytics]] and [[student-modeling]]. ### Connections Neurodiversity connects to [[special-education]], [[accessible-learning]], [[universal-design-for-learning]], and [[equity]]. It informs both how AI is deployed for [[student-experience]] and how assessments and literacy programs are designed to be fair across cognitive variability. ## Connected Concepts - [[special-education]] - [[accessible-learning]] - [[universal-design-for-learning]] - [[equity]] - [[student-experience]] - [[personalized-learning]] - [[cognitive-load-theory]] - [[learning-analytics]] - [[generative-ai]] ## Connected Articles - [[neurodivergent-computing-students]] — Neurodivergent Computing Students - [[adhd-video-segmentation-computing-education]] — Temporal Video Segmentation for Learners with ADHD - [[tactile-statistical-graphs-accessibility]] — Tactile Statistical Graphs for Accessibility - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools --- ## [Open Source](https://edtechdev.github.io/aied/concepts/open-source/) > **Open-source** AI in education is studied in [[lata-ferpa-compliant-local-llm-autograder]], [[vismatic-secure-sandbox-cs-education]], and [[open-source]] (tag) pages: local open models address [[privacy]], cost, and customization but bring deployment and quality burdens ([[regulation]], [[ai-education]]). ## Connected Concepts - [[intelligent-tutoring]] - [[pedagogical-llm-training]] - [[edtech-platform]] - [[writing-education]] - [[student-experience]] - [[pedagogical-safety]] - [[reinforcement-learning]] - [[student-modeling]] - [[agentic-ai]] - [[adaptive-learning]] - [[academic-integrity]] - [[automated-grading]] ## Connected Articles - [[agentic-ai-education-scoping-review]] - [[aiawe-automated-writing-evaluation]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[singh-eduqwen-pedagogical-rl-2026]] - [[stanbkt-bayesian-knowledge-tracing]] --- ## [Over-Reliance](https://edtechdev.github.io/aied/concepts/over-reliance/) > **Over-reliance** — excessive or uncalibrated dependence on AI tools where students delegate cognitive work they should perform themselves, resulting in reduced learning, diminished agency, and the displacement of skill development. Over-reliance is the behavioral manifestation of excessive [[cognitive-offloading]]: when offloading becomes the default rather than a strategic choice. Over-reliance sits at the intersection of AI tool design and human cognitive behavior. It is not simply about using AI too much — it is about using AI in ways that substitute for rather than complement learning processes. The wiki's research documents over-reliance as one of the most consequential risks of AI in education. ### How over-reliance appears in the research - **Learning displacement:** [[ai-making-us-stupid|Research on AI's cognitive effects]] documents how AI availability reduces effortful processing. Students who can access AI answers invest less in understanding, producing what researchers call the "Google effect extended to reasoning." - **The agency problem:** [[aied-unfinished-mission-bypass|AIED's unfinished mission]] frames over-reliance as an agency and motivation crisis — students bypass learning not because AI is compelling, but because learning tasks feel pointless when AI can complete them effortlessly. - **Motivation erosion:** [[ai-availability-student-motivation|Student motivation research]] finds that knowing AI is available reduces the perceived value of learning the skill yourself, a motivational calculus that particularly affects novice learners. - **Literacy debt:** [[agentic-literacy-debt|Agentic literacy debt]] describes the cumulative skill deficit that develops when students habitually rely on AI rather than developing their own competencies, analogous to technical debt in software. - **Fatigue cycles:** [[ai-fatigue-academic-contexts|AI fatigue]] research identifies a paradox where over-reliance leads to cognitive fatigue from constant AI interaction management, which in turn drives MORE reliance — a vicious cycle. - **Vibe Compiler and metacognitive preservation:** [[vibe-compiler-metacognition-genai-agency-2026|The Synthesis-Analysis Reciprocity Model]] proposes tools that preserve human epistemic agency by structuring AI interaction around human analysis cycles rather than AI generation cycles. ### Relationship to cognitive offloading Over-reliance is distinguished from [[cognitive-offloading]] by intent and outcome. Cognitive offloading can be productive (using a calculator to free mental resources for conceptual understanding) or unproductive (asking AI to solve a problem you should learn to solve). Over-reliance is specifically the unproductive pattern — where offloading has crossed the threshold from strategic support to learning displacement. ### Connections to related concepts Over-reliance connects to [[trust-calibration]] — knowing when to trust and when to question AI — and [[ai-literacy]], which includes the metacognitive skill of recognizing one's own reliance patterns. It intersects with [[self-regulated-learning]], as self-regulated learners monitor and adjust their AI use. The connection to [[scaffolding]] is bidirectional: well-designed scaffolds reduce over-reliance risk by providing support that fades as competence grows. ## Connected Concepts - [[cognitive-offloading]] - [[ai-literacy]] - [[trust-calibration]] - [[self-regulated-learning]] - [[scaffolding]] - [[metacognition]] - [[student-experience]] - [[generative-ai]] - [[academic-integrity]] - [[higher-ed]] - [[teacher-role]] - [[prompt-engineering]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[ai-making-us-stupid]] — Is AI Making Us Stupid? - [[aied-unfinished-mission-bypass]] — AIED's Unfinished Mission: Agency and Motivation - [[ai-availability-student-motivation]] — AI Availability and Student Motivation - [[agentic-literacy-debt]] — Agentic Literacy Debt - [[ai-fatigue-academic-contexts]] — AI Fatigue in Academic Contexts - [[vibe-compiler-metacognition-genai-agency-2026]] — Vibe Compiler: Synthesis-Analysis Reciprocity - [[ai-assisted-learning-modes-eeg]] — AI-Assisted Learning Modes (EEG Study) - [[generative-ai-guardrails-harm-learning]] — GenAI Without Guardrails Can Harm Learning - [[genai-performance-vs-learning]] — Distinguishing Performance from Learning - [[misiejuk-cognitive-offloading-prompting-2026]] — Cognitive Offloading in Student–AI Collaboration - [[veriforge-narrative-drafting-scaffolding-2026]] --- ## [Pedagogical Agent](https://edtechdev.github.io/aied/concepts/pedagogical-agent/) > **Synthesis**: Pedagogical agents are AI-driven conversational interfaces embedded in learning environments that use pedagogical strategies (eliciting, telling, scaffolding) to support learner engagement, reflection, and metacognition. Designs vary from simple information providers to interactive dialogue partners that adapt to learner states. A pedagogical agent is an interactive AI component within a learning system that engages learners through dialogue, questions, or prompts to support cognitive and metacognitive processes. Unlike passive dashboards or static feedback, pedagogical agents employ evidence-based tutoring strategies — such as eliciting learner self-assessments before providing feedback, or scaffolding problem-solving through Socratic dialogue. ## Connected Concepts - [[pedagogical-safety]] - [[agentic-ai]] - [[ai-education]] - [[ai-tutoring]] ## Connected Articles - [[ai-generated-interactive-fiction-education-2026]] - [[adversarial-stress-testing-role-playing-agents]]- [[embodied-inquiry-ai-facilitator-physics-2026]] --- ## [Training Pedagogical LLMs for Tutoring](https://edtechdev.github.io/aied/concepts/pedagogical-llm-training/) > Domain-specialized optimization can transform a mid-sized open-source model (Qwen3-32B) into a pedagogical domain expert that outperforms far larger proprietary systems — but only when training rewards *guiding* rather than *answering*.^[[singh-eduqwen-pedagogical-rl-2026]] Classical instructional design theory (ADDIE, Dick & Carey) combined with modern ReAct reasoning achieves the highest performance in automated instructional design.^[[jeon-isd-agent-bench-2026]] General-purpose LLMs are optimized for helpfulness: users want quick, correct answers. Tutoring requires the opposite: the goal is **not to provide the answer, but to help the student get to the answer themselves**. This creates a fundamental incentive mismatch. ## Approach 1: RL-SFT-RL Pipeline for Pedagogical Reasoning (EduQwen) Singh et al. (2026) developed a three-stage pipeline transforming Qwen3-32B into EduQwen, achieving **96.52%** on the CDPK Benchmark and surpassing Gemini-3 Pro (90.55%). ### Stage 1: Initial RL (EduQwen 32B-RL1) - **Algorithm:** DAPO (Decoupled Advantage Policy Optimization) with asymmetric clipping - **Reward model:** Prioritizes *guiding* responses over direct answers - **Curriculum learning:** Progressive difficulty; hard-negative mining excludes questions the base model already solves perfectly - **Extended rollouts:** 5→8 steps to capture multi-step pedagogical decisions - **Result:** 94.13% (already SOTA) ### Stage 2: Synthetic SFT (EduQwen 32B-SFT) - RL1 model generates 40,000 synthetic responses - Gradient-based selection retains only hard examples - Difficulty-weighted sampling: easy questions → one example; hard questions → all, weighted up - **Result:** 96.20% ### Stage 3: Final RL (EduQwen 32B-SFT-RL2) - Second DAPO round, reusing the original hard-negative set - Model now solves problems it originally found challenging - **Result:** 96.52% (definitive SOTA) ## The Pedagogy Benchmark: Evaluating Pedagogical Knowledge Lelièvre et al. (2025) introduced **The Pedagogy Benchmark**, measuring Cross-Domain Pedagogical Knowledge (CDPK) and Special Education Needs and Disability (SEND) knowledge from real teacher professional development exams. Across **97 models**, accuracy ranged from **28% to 89%**—revealing that pedagogical knowledge is not automatically acquired in general pretraining. **EduQwen connection:** Singh et al.’s EduQwen achieved **96.52% on CDPK**, demonstrating that targeted RL+SFT optimization can close the pedagogical knowledge gap that Lelièvre et al. document. The benchmark serves as both a diagnostic (showing most models fail at pedagogy) and a training target (showing optimization works). Live leaderboards track cost-accuracy Pareto frontiers: [rebrand.ly/pedagogy](https://rebrand.ly/pedagogy) ## Approach 2: Theory-Grounded Instructional Design Agents (ISD-Agent-Bench) Jeon et al. (2026) created a benchmark for LLM agents automating Instructional Systems Design (ISD), testing whether classical pedagogy theory improves agent performance. | Architecture | Performance | Why | |-------------|-------------|-----| | **Hybrid: theory + ReAct** | **Best** | Classical ADDIE/Dick & Carey frameworks provide structure; ReAct enables flexible multi-step reasoning | | Pure theory-based | Moderate | Structured but inflexible | | Technique-only (pure ReAct) | Worst | Flexible but lacks pedagogical grounding | **Key insight:** Theoretical quality strongly correlates with benchmark performance. Theory-based agents excel in **problem-centered design** and **objective-assessment alignment**. ### Benchmark Design - **25,795 scenarios** from Context Matrix (51 variables × 5 categories × 33 ISD sub-steps) - **Multi-judge protocol** across diverse LLM providers to mitigate LLM-as-judge bias - High inter-judge reliability achieved ## Synthesis: What Makes Pedagogical Training Work | Principle | EduQwen | ISD-Agent-Bench | |-----------|---------|-----------------| | **Reward/guide, don't answer** | DAPO reward model penalizes direct solutions | Theory-enforced ISD steps require alignment between objectives and assessment | | **Curriculum by difficulty** | Hard-negative mining + progressive rollouts | Context Matrix systematically varies complexity | | **Multi-step reasoning** | Extended rollouts (5→8 steps) | ReAct-style reasoning chains | | **Validate with theory** | CDPK benchmark measures pedagogical knowledge | ADDIE/Dick & Carey frameworks ground design decisions | | **Iterative refinement** | RL → SFT → RL pipeline | Multi-judge evaluation reduces bias | ## Relationship to Safety and Design Training for pedagogy is not just about accuracy — it is a **safety intervention**: - A model that rewards "guiding" over "answering" is less likely to commit [[ai-tutor-safety-harms|answer over-disclosure harms]] - Theory-grounded agents (ISD-Agent-Bench) align with pedagogical principles that prevent [[metacognition|metacognitive suppression]] - However, training on pedagogical benchmarks does not guarantee multi-turn safety; SafeTutors shows even specialized models degrade over sustained dialogue ## Open Questions 1. Does pedagogical RL training generalize across subjects, or is subject-specific tuning (as SafeTutors suggests) always needed? 2. Can the RL-SFT-RL pipeline be combined with longitudinal memory (see [[llm-student-modeling-memory]]) for personalized tutoring? 3. Would ISD-agent theory improve general tutoring conversation, or is it limited to macro-level curriculum design? ## Connected Concepts - [[intelligent-tutoring]] - [[scaffolding]] - [[adaptive-learning]] - [[metacognition]] - [[affective-tutoring]] - [[human-in-the-loop-ai]] - [[personalized-learning]] - [[student-modeling]] - [[self-regulated-learning]] - [[pedagogical-safety]] - [[formative-assessment]] - [[llm]] ## Connected Articles - [[ai-tutor-effectiveness-review]] - [[authentic-assessment]] - [[civic-education-ai-lesson-plans]] - [[codify-socratic-tutoring-programming]] - [[cognitive-agent-compilation]] - [[contextual-sycophancy-ai-literacy]] - [[educational-llm-alignment]] - [[eduguard-safe-rag-llm-tutor]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[llm-tts-dialogue-lesson-generation]] - [[multimodal-learning-genai]] - [[neural-symbolic-knowledge-tracing]] - [[nsmq-riddles-science-math-benchmark]] - [[singh-eduqwen-pedagogical-rl-2026]] - [[tact-pedagogically-adaptive-esl-tutoring]] --- ## [Pedagogical Safety](https://edtechdev.github.io/aied/concepts/pedagogical-safety/) > **Pedagogical safety** — the design principle that AI education systems must protect learners from harm, including inappropriate content, unsafe advice, biased treatment, and manipulative interaction patterns. Safety is particularly critical for K-12 contexts. ### Safety dimensions - **Content safety:** [[eduzone-llm-safety-k12|EduZone]] and [[eduguard-safe-rag-llm-tutor|EduGuard]] implement guardrails that prevent AI tutors from generating harmful, inappropriate, or inaccurate content. - **Interaction safety:** [[ai-tutor-safety-harms|AI tutor harm taxonomies]] categorize potential harms — from [[over-reliance]] to [[cognitive-offloading]] to emotional manipulation. - **Age-appropriate design:** [[child-safety-genai|Child safety frameworks]] and [[k-12]] research establish age-specific safety requirements. - **Hallucination as safety risk:** [[hallucination-risk]] is a safety concern when students cannot distinguish fabricated from factual information. ### Safety by design The wiki advocates safety as a design requirement, not an afterthought. [[pedagogical-llm-training|Pedagogical LLM training]] and [[educational-llm-alignment|educational alignment]] build safety into model behavior. [[rag|Retrieval-augmented generation]] grounds responses in verified content. ### Connections Pedagogical safety connects to [[hallucination-risk]], [[rag]], [[k-12]], [[ethics]], [[regulation]], [[ai-governance-education]], and [[llm]]. It is the protective layer that enables safe AI education deployment. ## Connected Concepts - [[hallucination-risk]] - [[rag]] - [[k-12]] - [[ethics]] - [[regulation]] - [[ai-governance-education]] - [[llm]] - [[over-reliance]] - [[ai-tutoring]] - [[child-safety-genai]] ## Connected Articles - [[eduzone-llm-safety-k12]] - [[eduguard-safe-rag-llm-tutor]] - [[ai-tutor-safety-harms]] - [[child-safety-genai]] - [[educational-llm-alignment]] - [[eduframetrap-llm-sycophancy-educational-safety]] --- ## [Peer Review](https://edtechdev.github.io/aied/concepts/peer-review/) > **Peer review** — the practice in which students read, evaluate, and provide feedback on one another's work, most often writing. In writing pedagogy, peer review is a long-standing best practice: students learn both from receiving feedback and from providing criteria-based feedback to others, and interactions with peers about their writing correlate with deeper learning, audience awareness, and personal and social development. In the AI era, peer review is being re-examined as a human complement to [[ai-feedback-quality|AI-generated feedback]], with models such as PAIRR pairing the two. Peer review is valued because it gives students an authentic audience, develops their evaluative judgment through criteria-based responding, and builds the social and relational context that supports engagement and motivation. However, its quality depends heavily on scaffolding — how well it is structured, and whether students are given clear criteria and training. This is precisely where AI feedback is positioned as a complement: AI can provide consistent, rubric-driven, actionable feedback on organization, focus, and structure, while peers offer specific, context-aware feedback rooted in their shared understanding of the assignment, an authentic audience, and emotional support. ### How peer review appears in the research - **Peer + AI feedback (PAIRR):** [[pairr-ai-peer-review-2025|The PAIRR model]] combines peer review with AI review in a human-centered process, finding students value the similarity of the two (as reassurance) and their complementarity (AI's broad, rubric-driven feedback vs. peers' specific, contextual feedback), while critically assessing AI outputs builds [[ai-literacy]] and writerly agency. - **AI peer feedback systems:** [[ai-peer-feedback-systems|AI peer feedback research]] examines how AI tools support or mediate peer-feedback workflows, and how their design affects the quality of feedback students give and receive. - **Authentic assessment and collaboration:** Peer review features in [[beyond-detection-authentic-assessment-ai-2025|authentic assessment redesign]] and collaborative learning contexts, where it supports [[self-regulated-learning]] and the metacognitive development of students' evaluative judgments. - **Learning through giving feedback:** Research consistently shows students learn from providing criteria-based feedback ([[ai-internal-feedback-evaluative-judgments|evaluative-judgment development]]), an insight that informs how AI-supported peer review is designed to preserve this learning. ### Peer review vs. AI feedback A key finding in the wiki's research is that AI and peer feedback are best understood as complementary rather than substitutable. [[pairr-ai-peer-review-2025|PAIRR research]] found only 6% of students preferred AI feedback alone, while 58% preferred combined feedback — AI utility is experienced in the context of human feedback. Peers bring contextual knowledge, authentic audience awareness, and human connection that AI lacks; AI brings consistency, immediacy, and actionable rubric-driven revision strategies. The quality of both depends on scaffolding, and the emphasis on reflection — students articulating why they accept or reject feedback — is what builds transferable writing knowledge and agency. ### Connections to related concepts Peer review connects to [[writing-education]] and [[formative-assessment]] as a core instructional practice, and to [[ai-feedback-quality]] and [[ai-literacy]] as it evolves in the AI era. It supports [[self-regulated-learning]] and [[metacognition]] through students' reflection on feedback, and relates to [[student-experience]] and [[collaborative-learning]] through the social and relational context it creates. Its role in giving and evaluating feedback ties to [[ai-internal-feedback-evaluative-judgments|evaluative judgment]] and to [[academic-integrity]] as institutions re-examine assessment in the presence of generative AI. ## Connected Concepts - [[writing-education]] - [[formative-assessment]] - [[ai-feedback-quality]] - [[ai-literacy]] - [[self-regulated-learning]] - [[metacognition]] - [[student-experience]] - [[collaborative-learning]] - [[academic-integrity]] ## Connected Articles - [[pairr-ai-peer-review-2025]] — Peer and AI Review + Reflection (PAIRR) - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[beyond-detection-authentic-assessment-ai-2025]] — Beyond Detection: Redesigning Authentic Assessment - [[ai-internal-feedback-evaluative-judgments]] — Unravelling Undergraduates' Development of Evaluative Judgments - [[learner-centered-feedback-ai]] — Enhancing Learner-Centered Feedback With AI - [[genai-linguistic-diversity-academic-writing]] — Generative AI and Linguistic Diversity in Academic Writing --- ## [Personalized Learning](https://edtechdev.github.io/aied/concepts/personalized-learning/) Tailoring educational experiences to individual learner profiles, including prior knowledge, learning pace, preferences, and affective states. AI enables personalization at scale, though the gap between *system personalization* and *learner-perceived personalization* remains an open measurement challenge. ## Architectures for AI-Driven Personalization ### Longitudinal Memory (PersonaVLM → Education) Nie et al. (2026) developed a multimodal long-term memory architecture (PersonaVLM) that maintains persona consistency across interactions. Mapped to education, this enables tutoring systems that remember a learner's misconceptions, preferred explanations, and progress history across sessions—addressing a critical deficit in stateless chatbot tutors. ### Agent-Native Personalization Substrate (DeepTutor) Ma et al. (2026) design every DeepTutor feature to share a common personalization substrate, rather than bolting personalization onto reactive tools. This architecture ensures cross-modality coherence: the same learner profile drives problem solving, question generation, and collaborative writing. ### Multi-Agent Social Personalization (MAIC) Yu et al. (2024) personalize not only content but *social context*. Classmate archetypes (Class Clown, Deep Thinker, Note Taker, Inquisitive Mind) create varied peer-learning dynamics matched to individual learner needs. ## Measurement Challenges - **System vs. perceived personalization** — A system can adapt without the learner feeling recognized - **Longitudinal validity** — Personalization benefits may decay if profiles become stale or overfit - **Equity risks** — Over-personalization can strand learners in low-expectation tracks ## Connected Concepts - [[adaptive-learning]] - [[intelligent-tutoring]] - [[student-experience]] - [[formative-assessment]] - [[higher-ed]] - [[scaffolding]] - [[stem-education]] - [[generative-ai]] - [[edtech-platform]] - [[ai-literacy]] - [[learning-analytics]] - [[teacher-role]] ## Connected Articles - [[ai-guided-learning-audiovideo-2026]] - [[a4l-analytics-pipeline]] - [[adaptive-pretesting-retention]] - [[ai-adult-learning-design]] - [[ai-coaching-rl-skill-development]] - [[ai-generated-traces-novice-programmers]] - [[ai-interlocutor-l2-spoken-dialogue]] - [[ai-learning-tools-engineering-education-needs]] - [[ai-lms-middle-school-longitudinal]] - [[ai-powered-personalized-learning-elementary-fractions-2026]] - [[antiskillbench-persona-skills-privacy-2026]] - [[automated-presentation-coaching]] - [[cognitive-agent-compilation]] - [[courseblueprint-adaptive-video-generation]] - [[ecnuclaw-k12-personalized-companion]] - [[empathy-coaching-chatbot]] - [[learnity-graphs-lifelong-learning-framework-2026]] - [[genai-higher-education-systematic-review-2026]] --- ## [Physics Education](https://edtechdev.github.io/aied/concepts/physics-education/) > **Physics Education** — the study of how students learn physics and how to teach it more effectively, spanning Socratic AI tutoring, computational thinking assessment, student trust and AI adoption patterns, automated scoring validity, and teacher preparation. The physics education articles in this wiki are notable for their domain-specificity: they explore how AI tools interact with the unique cognitive demands of physics reasoning — visual-spatial thinking, mathematical modeling, abstract systems thinking, and multi-step problem-solving. Physics education research has become a proving ground for AI in education because physics problems are well-structured yet cognitively demanding, making them ideal for studying how AI tools affect learning, reasoning, and assessment. The seven articles in this wiki collectively paint a picture of a field grappling with both the promise and the limits of AI — from Socratic chatbots that improve student question quality to systematic scoring biases that penalize linguistically diverse learners. ### Key research themes **Socratic AI tutoring in physics** is the most developed theme, with three articles deploying LLM-powered Socratic dialogue in real physics courses. **[[hashmi-socratic-physics-chatbot-2025|Hashmi et al.]]** demonstrated that sustained Socratic interaction with an AI chatbot dramatically improves question specificity in introductory mechanics, with 150 STEM majors in a live course. **[[socratic-ai-physics-tutor-taxonomy-2026|Hashmi & Rebello]]** built a bottom-up taxonomy of 357 student discourse categories from the same deployment, revealing that meta-procedural turns — where students cede strategic control to the tutor — dominate student interactions. Both contribute to broader [[socratic-method]] research and connect to [[ai-tutoring]] and [[intelligent-tutoring]] frameworks. **Student AI adoption and trust** explores how physics students actually use AI tools. **[[fouad-bentley-trust-utility-gap-physics-2026|Fouad & Bentley]]** found a 50-point trust-utility gap: 91% use AI for coursework but only 41% trust it, with students spontaneously identifying AI failure modes in visual-spatial reasoning and circuits. **[[becker-chatgpt-typology-physics-2026|Becker et al.]]** developed a two-profile typology — 70% "Pragmatic Users" and 30% "Skeptical Non-Users" — from 1,189 survey responses, showing both groups make calculated risk-utility trade-offs. These studies advance [[ai-literacy]] and [[trust-calibration]] research, and challenge one-size-fits-all AI policies. **Assessment and computational thinking** examines how AI can evaluate physics learning. **[[llm-computational-thinking-physics-2026|Savage et al.]]** used LLMs to assess computational thinking growth in introductory physics, finding LLMs can scale CT assessment but struggle with complex constructs like Systems Thinking. **[[ai-scoring-language-bias-physics|Feser & Tschisgale]]** demonstrated that AI scoring systematically underestimates linguistically weak students' physics explanations — a finding that connects to [[assessment-validity]], [[bias-mitigation]], and [[equity-in-ai-education]]. **Teacher preparation and simulation** uses AI to train physics teachers. **[[multiagent-classroom-dual-process-physics-teachers-2026|Tufino]]** created a simulated multi-agent classroom where five AI students enact dual-process theory reasoning hazards, giving prospective physics teachers rare practice in responding to authentic student reasoning. This connects to [[professional-training]], [[dual-process-theory]], and [[simulation]]-based learning. ### Connections to related concepts Physics education sits within the broader [[stem-education]] domain but has distinctive connections: to [[socratic-method]] through the strong tradition of Socratic dialogue in physics problem-solving; to [[computational-thinking]] through the increasing role of computation in physics; to [[assessment-validity]] through the challenges of scoring physics explanations; and to [[professional-training]] through simulation-based preparation. The [[student-experience]] and [[ai-literacy]] concepts are essential for understanding how physics students navigate AI tools, while [[educational-measurement]] and [[automated-grading]] connect to the assessment dimension. ## Connected Concepts - [[stem-education]] - [[socratic-method]] - [[ai-tutoring]] - [[intelligent-tutoring]] - [[computational-thinking]] - [[ai-literacy]] - [[trust-calibration]] - [[student-experience]] - [[assessment-validity]] - [[bias-mitigation]] - [[equity-in-ai-education]] - [[automated-grading]] - [[educational-measurement]] - [[learning-analytics]] - [[professional-training]] - [[dual-process-theory]] - [[simulation]] - [[generative-ai]] - [[higher-ed]] ## Connected Articles - [[hashmi-socratic-physics-chatbot-2025]] - [[socratic-ai-physics-tutor-taxonomy-2026]] - [[fouad-bentley-trust-utility-gap-physics-2026]] - [[becker-chatgpt-typology-physics-2026]] - [[llm-computational-thinking-physics-2026]] - [[ai-scoring-language-bias-physics]] - [[multiagent-classroom-dual-process-physics-teachers-2026]] - [[physics-chatbot-epistemological-beliefs-2026]] - [[ai-generated-smartphone-circular-motion-lab-2026]] - [[genai-ar-physics-simulation-prompt-2026]] - [[embodied-inquiry-ai-facilitator-physics-2026]] --- ## [AI Plagiarism Detection](https://edtechdev.github.io/aied/concepts/plagiarism-detection/) Technologies and methods for detecting AI-generated content in academic submissions, including classifier-based approaches, watermarking, and stylistic analysis. The effectiveness and reliability of these tools remain contested, particularly as LLM outputs become more sophisticated. ## Connected Concepts - [[academic-integrity]] - [[llm]] - [[higher-ed]] - [[ai-education]] ## Connected Articles - [[detecting-llm-generated-text-latent-prompt]] - [[llm-detecting-llm-generated-content-education]] --- ## [Privacy in AI Education](https://edtechdev.github.io/aied/concepts/privacy/) > **Privacy** — the protection of student data, identity, and autonomy in AI-augmented learning environments. Privacy concerns intensify as AI systems collect increasingly granular behavioral data for personalization and analytics. ### Privacy challenges - **Data collection at scale:** [[learning-analytics]] and [[edtech-platform|educational platforms]] collect clickstream, writing, and interaction data. [[privacy]] research examines whether this collection is proportionate to educational benefit. - **Student surveillance:** [[ai-fatigue-academic-contexts|AI fatigue]] and [[over-reliance]] research connect to privacy concerns — constant AI monitoring can feel invasive even when well-intentioned. - **K-12 protections:** [[k-12]] settings demand stronger privacy safeguards due to minor status. [[child-safety-genai|Child safety]] research extends privacy to safety considerations. - **Federated and edge AI:** [[privacy|Edge AI approaches]] keep student data local, reducing central collection. [[ai-lms-middle-school-longitudinal|LMS privacy architectures]] demonstrate privacy-first design. ### The personalization-privacy tradeoff [[personalized-learning]] requires detailed learner data to function, creating a tension with privacy. The wiki explores [[privacy]] approaches that balance personalization with data minimization. ### Connections Privacy connects to [[learning-analytics]] (the data collector), [[personalized-learning]] (the data consumer), [[k-12]] (heightened protections), [[ethics]] (normative framework), and [[regulation]] (legal requirements). ## Connected Concepts - [[learning-analytics]] - [[personalized-learning]] - [[k-12]] - [[ethics]] - [[regulation]] - [[equity]] - [[ai-governance-education]] - [[educational-policy-ai]] - [[pedagogical-safety]] - [[student-experience]] ## Connected Articles - [[ai-fatigue-academic-contexts]] - [[ai-lms-middle-school-longitudinal]] - [[child-safety-genai]] - [[eduzone-llm-safety-k12]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] --- ## [Professional Training and AI](https://edtechdev.github.io/aied/concepts/professional-training/) > **Professional training** — the use of AI for workforce development, corporate learning, and professional skill acquisition. Professional training extends AI in education beyond formal schooling into workplace and lifelong learning contexts. ### AI in professional training - **Skill development:** [[crewscaler-ai-upskilling-framework|AI upskilling frameworks]] and [[ai-coaching-rl-skill-development|RL-based skill coaching]] demonstrate AI-driven professional development. - **Simulation and practice:** [[adaptive-virtual-patient-psychotherapy-training|Virtual patient training]] and [[astra-atco-training-simulator|ATCO training simulators]] create AI-powered professional practice environments. - **Lifelong learning integration:** [[lifelong-learning]] and [[adult-learning]] research connect professional training to continuous education. - **Public sector:** [[ai-adoption-training-public-sector|Public sector AI adoption]] examines training in government contexts. ### Distinct from academic education Professional training differs from academic education in its focus on applied skills, immediate workplace relevance, and adult learner characteristics. [[adult-learning]] theory and [[adult-learning]] principles inform professional AI training design. ### Connections Professional training connects to [[lifelong-learning]], [[adult-learning]], [[faculty-development]] (the academic parallel), [[simulation]], and [[ai-literacy]] (workplace AI competency). ## Connected Concepts - [[lifelong-learning]] - [[adult-learning]] - [[faculty-development]] - [[ai-literacy]] - [[simulation]] - [[higher-ed]] - [[generative-ai]] - [[llm]] - [[adaptive-learning]] - [[personalized-learning]] ## Connected Articles - [[ai-interior-design-malaysia-2026]] - [[crewscaler-ai-upskilling-framework]] - [[ai-coaching-rl-skill-development]] - [[adaptive-virtual-patient-psychotherapy-training]] - [[astra-atco-training-simulator]] - [[ai-adoption-training-public-sector]] - [[genai-pd-ai-pck-learning-gain-2026]] - [[cyberagents-gamified-cybersecurity-learning-2026]] - [[hdr-brachytherapy-agentic-ai-simulation-2026]] - [[residencyrl-clinical-rl-training-2026]] --- ## [Programming Education](https://edtechdev.github.io/aied/concepts/programming-education/) > **Programming education** — the teaching and learning of computer programming, from introductory block-based programming to advanced software development. In the AI era, programming education increasingly incorporates computational thinking, AI-assisted tools, and embodied approaches (such as robots) to help learners connect abstract code to meaningful, observable outcomes. It spans [[k-12]] and [[higher-ed]], formal and informal settings, and ranges from text-based languages (Python, Scratch) to block-based visual programming. Programming education is closely related to [[computational-thinking]] (the underlying problem-solving concepts) and [[cs-education]] (the broader computer science discipline). A key challenge in the AI era is that learners and teachers struggle to connect abstract program logic to meaningful outcomes, and that AI tools can both assist and potentially undermine genuine learning. Embodied and game-based approaches — using robots, block programming, and simulation — are increasingly used to ground programming in concrete, motivating tasks. ### How programming education appears in the wiki's research - **Embodied block programming:** [[roboblockly-conversational-block-robotics-ct-2026|RoboBlockly Studio]] combines block-based programming with a conversational AI teaching agent and embodied robot execution, creating an iterative authoring–running–observing–revising loop that preserves learner agency. - **Natural-language robot control:** [[edusim-llm-robotic-simulation-education-2026|EduSim-LLM]] lets beginners control simulated robots through natural-language instructions, lowering the barrier to robot programming. - **Robotics and computational thinking:** [[computational-thinking-educational-robotics-secondary-2026|Valls i Pou]] links computational thinking to educational robotics in secondary STEAM curricula. - **Game-based and gamified learning:** [[game-based-gamified-robotics-education-review-2026|A systematic review]] compares game-based learning and gamification in robotics education, which emphasizes introductory programming and modular kits. - **Project-based robotics:** [[bots-blocks-project-based-robotics-education-2026|Bots and Blocks]] teaches robotics programming through an agile, semester-spanning project. Programming education connects to [[educational-robotics]], [[computational-thinking]], [[cs-education]], [[stem-education]], [[llm|AI-assisted programming]], [[game-based-learning]], [[gamification]], [[project-based-learning]], and [[embodied-learning]]. ## Connected Concepts - [[computational-thinking]] - [[cs-education]] - [[educational-robotics]] - [[stem-education]] - [[block-programming]] - [[llm]] - [[game-based-learning]] - [[project-based-learning]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[edusim-llm-robotic-simulation-education-2026]] — EduSim-LLM - [[computational-thinking-educational-robotics-secondary-2026]] — Computational Thinking and Educational Robotics - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks - [[jeon-isd-agent-bench-2026]] — ISD Agent Bench --- ## [Project-Based Learning](https://edtechdev.github.io/aied/concepts/project-based-learning/) > **Project-based learning (PBL)** — an active, learner-centred pedagogy in which students learn by engaging in extended, real-world projects that require inquiry, problem solving, and the application of knowledge to produce tangible outcomes. PBL emphasizes student autonomy, collaboration, and authentic tasks, and is widely used with technology — including [[educational-robotics|educational robotics]] and AI — to give learners hands-on, meaningful projects. It contrasts with purely theoretical or lecture-based instruction. PBL is closely related to [[active-learning]], [[experiential-learning]], [[collaborative-learning]], and constructivist pedagogy. It is especially valuable for AI and robotics education because these fields are inherently applied: learners best understand robots, algorithms, and systems by building and testing them in project contexts. PBL also fosters [[computational-thinking]], problem solving, and [[self-regulated-learning|self-direction]]. ### How PBL appears in the wiki's research - **Robotics projects:** [[bots-blocks-project-based-robotics-education-2026|Bots and Blocks]] presents an agile, semester-spanning project-based approach to teach robotics in an applied computer science program, addressing the theory-practice gap. - **Gamification coupling:** [[game-based-gamified-robotics-education-review-2026|A systematic review]] found gamification in robotics education strongly favored project-based learning (p = .009). - **AI literacy and co-design:** PBL underlies many AI literacy and teacher-education interventions, where learners co-create AI tools or resources. PBL connects to [[active-learning]], [[experiential-learning]], [[collaborative-learning]], [[educational-robotics]], [[game-based-learning]], [[computational-thinking]], and [[higher-ed]]/[[k-12]] pedagogy. ## Connected Concepts - [[active-learning]] - [[experiential-learning]] - [[collaborative-learning]] - [[educational-robotics]] - [[game-based-learning]] - [[computational-thinking]] - [[higher-ed]] ## Connected Articles - [[bots-blocks-project-based-robotics-education-2026]] — Bots and Blocks - [[game-based-gamified-robotics-education-review-2026]] — Game-Based and Gamified Robotics Education - [[genai-literacy-training-teacher-education-dbr-2026]] — AI Literacy Training for Teachers - [[roboblockly-conversational-block-robotics-ct-2026]] — RoboBlockly Studio - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education --- ## [Prompt Engineering](https://edtechdev.github.io/aied/concepts/prompt-engineering/) > **Prompt engineering** — the practice of designing and refining inputs to large language models to achieve desired outputs. In education, prompt engineering serves dual roles: as a learner skill (students must learn to prompt effectively) and as a system design lever (developers craft prompts that shape AI tutoring behavior). Prompt engineering is central to effective [[generative-ai]] use in education. Unlike traditional programming interfaces, LLMs respond to natural language — but the quality, accuracy, and pedagogical value of those responses depend heavily on prompt design. Research in this wiki reveals that prompting is not a neutral act: it reflects how students think, plan, and allocate cognitive effort. ### How prompt engineering appears in the research - **Prompting as cognitive trace:** [[misiejuk-cognitive-offloading-prompting-2026|Misiejuk et al.]] show that prompt patterns reveal cognitive offloading — high-quality work uses context-rich, polite, and instructional prompts; low-quality work shows reactive disagreement without domain grounding - **Prompting as literacy:** [[tracing-genai-literacy-interaction-patterns|Tracing GenAI literacy]] and [[aaai2026-prompting-literacy-k12|K-12 prompting literacy]] research frame prompting as a core [[ai-literacy]] component - **Prompting as system design:** [[cotal-formative-assessment-scoring-2026|CoTAL]] uses human-in-the-loop prompt engineering for formative assessment scoring; [[choi-anchor-aes-prompting-2025|anchor-based prompting]] improves automated essay scoring - **Prompt modalities:** [[voice-text-prompt-problems-computing-education|Voice vs. text input research]] examines whether prompting modality affects learning outcomes - **Scaffolded prompting:** [[guided-llm-scaffolding-independent-learning|Guided LLM scaffolding]] and [[scaffolding-critical-engagement-genai-minority-students|critical engagement scaffolding]] teach structured prompting as a learning intervention ### Connections to broader concepts Prompt engineering connects to [[scaffolding]] — well-designed prompts can scaffold student thinking rather than bypass it. It intersects with [[metacognition]] and [[ai-literacy]], as effective prompting requires understanding both the AI's capabilities and one's own learning goals. The [[cognitive-offloading]] research directly links prompt quality to whether AI use supports or undermines learning. ## Connected Concepts - [[scaffolding]] - [[ai-literacy]] - [[agentic-ai]] - [[metacognition]] - [[curriculum-design]] - [[over-reliance]] - [[writing-education]] - [[k-12]] - [[generative-ai]] - [[instructional-design]] - [[cs-education]] - [[higher-ed]] ## Connected Articles - [[aaai2026-prompting-literacy-k12]] - [[ai-adoption-training-public-sector]] - [[ase-26-agentic-software-engineering-curriculum]] - [[choi-anchor-aes-prompting-2025]] - [[guided-llm-scaffolding-independent-learning]] - [[llm-intervention-design-cs-review]] - [[misiejuk-cognitive-offloading-prompting-2026]] - [[tracing-genai-literacy-interaction-patterns]] - [[pchl-he-framework-genai-content-creation-2026]] --- ## [Psychometrically Aware AI](https://edtechdev.github.io/aied/concepts/psychometrically-aware-ai/) > **Psychometrically aware AI** — models aligned with measurement theory — is the standard advanced in [[llm-psychometric-calibration-cdp]], [[llm-item-difficulty-prediction]], [[confidence-aware-ai-assessment]], and [[item-response-theory]]: calibrated, uncertainty-aware AI assessment preserves validity and trust ([[educational-measurement]], [[assessment-validity]]). ## Connected Concepts - [[confidence-aware-ai-assessment]] - [[item-response-theory]] - [[assessment-validity]] - [[educational-measurement]] ## Connected Articles - [[competency-based-education-genai-production-2026]] - [[multimodal-item-parameter-estimation-2026]] - [[llm-difficulty-calibration-programming-exams-2026]] --- ## [RAG (Retrieval-Augmented Generation)](https://edtechdev.github.io/aied/concepts/rag/) > **RAG (Retrieval-Augmented Generation)** — an AI architecture that combines information retrieval with text generation, allowing LLMs to ground responses in external knowledge sources rather than relying solely on training data. In education, RAG addresses hallucination, enables curriculum-grounded tutoring, and powers domain-specific AI tutors. ### How RAG is used in education - **Hallucination reduction:** [[eduguard-safe-rag-llm-tutor|EduGuard]] and [[eduzone-llm-safety-k12|EduZone]] use RAG to keep AI tutor responses grounded in verified educational content, reducing [[hallucination-risk]]. - **Curriculum-grounded tutoring:** [[retrieval-augmented-tutoring-algorithm-kite|KITE]] retrieves relevant curriculum materials to inform tutoring responses, ensuring alignment with course content. - **Textbook and materials indexing:** [[book-level-synthetic-textbook-organization|Synthetic textbook organization]] indexes educational content for retrieval. [[structrag-diagram-reasoning-ai-tutoring|StructRAG]] extends retrieval to structured diagrams. - **Training pipeline integration:** [[pedagogical-llm-training|Pedagogical LLM training]] uses RAG to ground tutor training in educational best practices. ### RAG vs fine-tuning RAG serves a complementary role to [[llm]] fine-tuning — retrieval provides up-to-date, domain-specific grounding without retraining, while fine-tuning embeds pedagogical behaviors. The wiki's research explores both approaches and their combination. ### Connections RAG connects to [[llm]] (the generation component), [[knowledge-graph]] (structured knowledge for retrieval), [[hallucination-risk]] (the primary problem RAG addresses), and [[edtech-platform]] (RAG powers production educational systems). ## Connected Concepts - [[llm]] - [[generative-ai]] - [[hallucination-risk]] - [[knowledge-graph]] - [[edtech-platform]] - [[ai-tutoring]] - [[pedagogical-llm-training]] - [[pedagogical-safety]] - [[k-12]] - [[higher-ed]] ## Connected Articles - [[eduguard-safe-rag-llm-tutor]] - [[eduzone-llm-safety-k12]] - [[retrieval-augmented-tutoring-algorithm-kite]] - [[structrag-diagram-reasoning-ai-tutoring]] - [[book-level-synthetic-textbook-organization]] - [[veriforge-narrative-drafting-scaffolding-2026]] - [[pchl-he-framework-genai-content-creation-2026]] --- ## [RCT](https://edtechdev.github.io/aied/concepts/rct/) > **RCT** — a key concept in AI in education research. Explored across 2 articles in this wiki. Research themes include: - In a with 1,174 participants, Cruces et al. find that substantially narrows education-based productivity gaps, closing approximately three-quarters of the initial performance differe - **Authors:** Zerkouk et al. · **Venue:** Journal of Computers in Education (2025) · **arXiv:** 2507.18882 ## Connected Concepts - [[scaffolding]] - [[adaptive-learning]] - [[teacher-role]] - [[metacognition]] - [[intelligent-tutoring]] - [[automated-grading]] - [[generative-ai]] - [[ai-literacy]] - [[feedback-loop]] - [[learning-analytics]] - [[ai-education]] - [[regulation]] ## Connected Articles - [[ai-changing-teaching-workflows]] — How AI Is Changing Teaching Workflows - [[generative-ai-education-productivity-gaps]] — Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment - [[multimodal-learning-genai]] — Multimodal Learning with Generative AI - [[llm-educational-simulation-adhd]] — LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles - [[bloom-aligned-educational-control-llms]] — From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs - [[ai-learning-assistants-higher-ed-large-scale]] — Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis --- ## [Reducing AI Misuse](https://edtechdev.github.io/aied/concepts/reducing-ai-misuse/) > **Reducing AI misuse** — the design, pedagogical, and policy levers that prevent students from substituting generative AI for their own cognitive work and instead steer them toward ethical, productive use. Effective approaches are sorted by impact rather than popularity, and the strongest evidence favors **structural levers** — tool guardrails and assessment redesign — that change the environment so misuse is harder, regardless of a student's motivation, over **educative levers** that rely on building durable capacity and student buy-in. The concept rests on the evidence that AI misuse actively harms durable learning — the performance–learning gap documented in [[ai-misuse-learning-harm]] — even while inflating immediate performance. Interventions therefore target the mechanisms of that harm: answer-copying, [[cognitive-offloading]], motivation erosion, and learning displacement. They are not mutually exclusive; a robust approach combines a structural floor with educative capacity-building. ### Why structural levers matter most Interventions can be sorted by **causal evidence × structural reach × scalability × sustainability**. On this basis, the two *structural* levers rank highest because they work whether or not students choose the right behavior — they constrain the environment rather than depending on internal motivation. The *educative* levers are essential but only effective when students buy in, so they are treated as the second tier despite their conceptual promise. ### Tier 1 — Directly proven to reduce learning harm **Guardrailed AI tool design ("hint-not-answer" scaffolding).** In the strongest causal finding in the wiki, a field RCT showed an unguarded ChatGPT-style tutor raised assisted practice performance **+48%** but reduced unassisted exam scores **−17%**, while a guardrailed tutor (hints instead of answers, plus teacher-authored problem information) eliminated the harm entirely. This mechanically prevents the answer-copying "crutch" behavior behind the damage. Activities include hint-not-answer tutoring, seeding prompts with correct solutions and common misconceptions, and requiring a student attempt before AI output is revealed. **Assessment redesign (AI-resistant + unassisted measures).** Because misuse harm is assessment-dependent — surfacing on proctored, closed-book, and unassisted measures while inflating ordinary graded coursework — changing what counts as achievement both deters misuse and surfaces it. Activities include unassisted in-class exams and oral defenses, requiring process artifacts (drafts, reflections, annotated reasoning), rewarding reasoning over surface fluency, and designating AI-free zones. ### Tier 2 — Strong framework support, high potential **Scaffolded use sequences: think first, AI second, reflect third.** Eight design principles for integrating LLMs without displacing [[critical-thinking]]: preserve cognitive friction, position AI as a provisional thinking partner rather than an authority, embed evaluation checkpoints, require metacognitive journaling and prompt logs, and balance AI-mediated with AI-free phases. Correlational evidence (independent work before AI produces stronger outputs) is strong; it is the pedagogically complete version of Tier 1. **AI literacy and prompting literacy with deliberate practice and immediate feedback.** A K-12 module using scenario-based prompt practice with an LLM auto-grader improved actual prompting skills and raised confidence in using AI for learning **+10.4%**, with 87% reporting they learned how to use AI responsibly. Demonstrated skill gains; the open question is whether these convert into downstream learning outcomes. It also addresses the equity gap in prior AI access. **Structured AI-use declaration frameworks.** Replacing generic "I used AI" checkboxes with domain-specific declarations that map use to cognitive stages (structural planning vs. content generation) forces reflection on the learning process and clarifies the boundary between acceptable assistance and misconduct, shifting the emphasis from policing to professional practice. ### Tier 3 — Promising, lower direct causal evidence **Metacognitive and self-assessment interventions.** Reflective journals, prompt logs, and calibration training rebuild the "absent cognitive baseline" of AI-native students who cannot locate their own cognitive boundary because AI-generated fluency masks it. Conceptually central but not yet causally tested. **Motivation redesign.** Because AI availability erodes autonomous motivation ("why put in the effort?"), restructuring tasks around goals AI cannot fulfill and around learner agency directly targets the persistence erosion that compounds the direct harm. **Critical AI literacy.** A power-knowledge framing that teaches learners to interrogate, challenge, and participate in AI governance rather than consume it. Long-term, equity-oriented, and structural in its ambitions, though its learning effects are largely untested. ## Connected Concepts - [[ai-misuse-learning-harm]] - [[over-reliance]] - [[cognitive-offloading]] - [[academic-integrity]] - [[assessment]] - [[ai-literacy]] - [[scaffolding]] - [[self-regulated-learning]] - [[metacognition]] - [[motivation]] - [[prompt-engineering]] ## Connected Articles - [[generative-ai-guardrails-harm-learning]] — GenAI Without Guardrails Can Harm Learning - [[genai-performance-vs-learning]] — Distinguishing Performance Gains from Learning - [[ai-assessment-scale-reform]] — The AI Assessment Scale and Assessment Reform - [[critical-thinking-genai-scaffolding]] — Scaffolding Critical Thinking with Generative AI - [[aaai2026-prompting-literacy-k12]] — Learning to Use AI for Learning (K-12 AI Literacy Module) - [[genai-declaration-frameworks-higher-education]] — Structuring Transparency: GenAI Declaration Frameworks - [[absent-cognitive-baseline-2026]] — The Absent Cognitive Baseline - [[ai-availability-student-motivation]] — AI Availability and Student Motivation - [[ai-literacy-power-knowledge]] — AI Literacy: An Exercise in Power-Knowledge - [[generative-ai-reduced-study-time-math]] — Generative AI Reduced Study Time on Math --- ## [AI Regulation in Education](https://edtechdev.github.io/aied/concepts/regulation/) > **AI regulation** — the laws, policies, and governance frameworks that control how AI is developed and deployed in educational settings. Regulation in the wiki spans government policy, institutional governance, and industry self-regulation. ### Regulatory landscape - **Government policy:** [[educational-policy-ai]] research examines national and regional AI education policies. [[state-policy-teacher-ai|State-level teacher AI policy]] and [[ai-lifelong-learning-policy|lifelong learning policy]] address regulatory gaps. - **Institutional governance:** [[ai-governance-education|AI governance frameworks]] and [[genai-policies-higher-ed-computing|institutional policy analysis]] document how universities develop internal AI rules. - **Safety regulation:** [[pedagogical-safety]], [[child-safety-genai|child safety]], and [[eduzone-llm-safety-k12|K-12 safety frameworks]] represent de facto regulation through safety requirements. - **Ethics as regulation:** [[ethics]] frameworks increasingly serve regulatory functions — [[ai-ethics-education-public-discourse|public discourse on AI ethics]] shapes policy expectations. ### The governance gap The wiki documents a persistent gap between AI deployment speed and regulatory maturity. [[institutional-change-framework-ai|Institutional change frameworks]] and [[regulation]] research argue for proactive governance rather than reactive policy. ### Connections Regulation connects to [[educational-policy-ai]], [[ai-governance-education]], [[ethics]], [[privacy]], [[pedagogical-safety]], and [[academic-integrity]]. It is the institutional layer that shapes how all other AI education practices operate. ## Connected Concepts - [[educational-policy-ai]] - [[ai-governance-education]] - [[ethics]] - [[privacy]] - [[pedagogical-safety]] - [[academic-integrity]] - [[equity]] - [[higher-ed]] - [[k-12]] ## Connected Articles - [[institutional-change-framework-ai]] - [[genai-policies-higher-ed-computing]] - [[state-policy-teacher-ai]] - [[ai-lifelong-learning-policy]] - [[ai-ethics-education-public-discourse]] --- ## [Reinforcement Learning](https://edtechdev.github.io/aied/concepts/reinforcement-learning/) > **Reinforcement learning** trains AI tutors and agents through reward signals: [[special-r1-rl-special-education]], [[singh-eduqwen-pedagogical-rl-2026]], [[pedagogical-safety-rl]], and [[ai-coaching-rl-skill-development]] align RL with pedagogical objectives, including safety and skill transfer ([[intelligent-tutoring]], [[agentic-ai]]). ## Connected Concepts - [[intelligent-tutoring]] - [[student-experience]] - [[stem-education]] - [[self-regulated-learning]] - [[scaffolding]] - [[active-learning]] - [[edtech-platform]] - [[higher-ed]] - [[learning-analytics]] - [[open-source]] - [[pedagogical-safety]] - [[pedagogical-llm-training]] ## Connected Articles - [[history-aware-student-simulation]] - [[q-learning-lab-rl-teaching]] - [[singh-eduqwen-pedagogical-rl-2026]]- [[residencyrl-clinical-rl-training-2026]] --- ## [Research Methods in AIED](https://edtechdev.github.io/aied/concepts/research-methods-aied/) > **Research methods in AIED** — the set of empirical designs, data-collection strategies, and analytic techniques researchers use to study AI in education: whether and how AI tools support (or harm) learning, and under what conditions. The wiki's corpus spans experimental, survey, qualitative, design-based, computational-benchmark, and review methods. Each has distinct strengths and limitations, and choosing among them involves trade-offs among internal validity (confidence in causal claims), external validity (generalizability), ecological validity (real-world authenticity), and the feasibility of studying fast-moving AI tools. The central tension in AIED research is that the strongest designs for causal inference — randomized experiments — are often the hardest to run with authentic AI tools in real classrooms, while the most authentic settings (field deployments, case studies, log-data analyses) offer weaker causal control. No single method resolves this; the field advances by triangulating across methods, and by being explicit about what kind of claim each design can support. ### Experimental and quasi-experimental designs Experiments randomly assign learners to conditions (e.g., AI tutor vs. human tutor, or AI-scaffolded vs. unassisted) to estimate causal effects on outcomes like learning gains, engagement, or motivation. **Randomized controlled trials** are the gold standard for internal validity. [[access-not-enough-ai-tutoring-2026|A randomized field study of human support plus AI tutoring]] and [[genai-can-harm-teaching-rct-2026|an RCT on generative AI in teaching]] use assignment to isolate causal effects. **Quasi-experimental** designs (pre/post, between-subjects, or matched groups without randomization) are more feasible in intact classrooms but weaker on causal claims. - **Strengths:** strongest causal inference; clean outcome measurement; supports effect-size estimation and [[efficacy-study|efficacy claims]]. - **Limitations:** costly and slow; artificial conditions can reduce ecological validity; fast-changing AI tools make long experiments date quickly; small samples often underpower detection of meaningful effects; ethical constraints on withholding potentially helpful tools. - **Exemplars:** [[access-not-enough-ai-tutoring-2026]], [[genai-can-harm-teaching-rct-2026]], [[adaptive-pretesting-retention]], [[agent-voice-accents-k12-group-learning]], [[ai-use-critical-thinking-medical-students-2026]]. ### Survey and structural-equation-modeling studies Cross-sectional surveys measure self-reported attitudes, perceptions, motivation, self-efficacy, and technology acceptance, often modeled with regression or structural equation modeling (SEM/PLS-SEM) to test hypothesized relationships and mediators. These dominate the wiki's corpus, particularly for acceptance, motivation, and psychological-mechanism questions. - **Strengths:** large samples; broad, low-cost coverage; can test complex mediational models of psychological mechanisms; feasible for studying attitudes that are hard to observe. - **Limitations:** cross-sectional data cannot establish causation; common-method/self-report bias; convenience sampling limits generalizability; mediators inferred from covariance, not manipulation. - **Exemplars:** [[acceptance-ai-english-tools-2026]], [[genai-motivation-engagement-2026]], [[ai-autonomous-learning-accomplishment-2026]], [[genai-over-reliance-learning-2026]], [[ai-use-critical-thinking-medical-students-2026]]. ### Qualitative methods Interviews, focus groups, and thematic analysis produce rich, contextual accounts of how students and teachers experience AI tools, the meanings they attach to them, and the tensions and harms that standardized measures miss. [[ai-tutor-safety-harms|Research on AI tutor safety]] and [[ai-changing-teaching-workflows|how AI changes teaching workflows]] rely heavily on qualitative evidence. - **Strengths:** deep ecological and conceptual insight; surfaces unexpected phenomena, risks, and mechanisms; essential for theory-building and for studying contested constructs like trust, autonomy, and authorship. - **Limitations:** limited generalizability; interpretive and researcher-dependent; small samples; weaker support for causal claims; findings can be hard to synthesize across studies. - **Exemplars:** [[ai-tutor-safety-harms]], [[ai-changing-teaching-workflows]], [[ai-education-global-capacity]], [[scaffolding-critical-engagement-genai-minority-students]]. ### Mixed-methods designs Mixed-methods studies combine quantitative and qualitative strands — often sequentially (e.g., QUAL→QUAN→qual) — so that qualitative data explains or contextualizes quantitative findings. [[genai-over-reliance-learning-2026|A mixed-method study of GenAI and sustainable learning]] pairs three-wave surveys with educator interviews; [[t2i-competence-paradox-2026|the competence-paradox study]] uses instructor focus groups, a student survey, and follow-up interviews. - **Strengths:** triangulation increases confidence; quantitative breadth plus qualitative depth; can explain unexpected results and bridge mechanism and magnitude. - **Limitations:** complex, resource-intensive, and methodologically demanding; integration can be shallow if not carefully designed; still inherits the weaknesses of each strand (e.g., self-report). - **Exemplars:** [[genai-over-reliance-learning-2026]], [[t2i-competence-paradox-2026]], [[same-ai-different-pathways]], [[fouad-bentley-trust-utility-gap-physics-2026]]. ### Design-based research (DBR) DBR iteratively designs, implements, and refines an educational intervention in authentic contexts, cycling between theory, design, and real-world practice. It is prominent in the wiki for developing AI learning environments and pedagogical models. - **Strengths:** high ecological validity and practical relevance; produces both usable artifacts and theory; responsive to the complexity of real classrooms and evolving AI tools. - **Limitations:** weak internal validity; findings are context-bound and hard to generalize; long timelines; difficult to isolate which design element caused an outcome. - **Exemplars:** [[ai-assisted-collaborative-learning-model-dbr]], [[genai-literacy-training-teacher-education-dbr-2026]], [[critical-thinking-genai-scaffolding]]. ### Systematic reviews and meta-analyses Reviews synthesize the evidence base. Systematic and scoping reviews map and appraise the literature; meta-analyses pool effect sizes across studies. [[zerkouk-comprehensive-review-its-2025|A comprehensive ITS review]] and [[genai-higher-education-systematic-review-2026|a systematic review of GenAI in higher education]] exemplify the approach. - **Strengths:** efficient synthesis of a large, fragmented literature; meta-analysis yields pooled effect estimates and detects moderators; essential for evidence-based practice and identifying gaps. - **Limitations:** depend on the quality of included studies (garbage-in/garbage-out); publication bias; heterogeneous methods and outcome measures make synthesis hard; rapidly aging given the speed of AI change. - **Exemplars:** [[zerkouk-comprehensive-review-its-2025]], [[genai-higher-education-systematic-review-2026]], [[chatgpt-critical-creative-thinking-review]], [[ai-tutor-effectiveness-review]], [[agentic-ai-education-scoping-review]]. ### Computational and benchmark evaluation Computational evaluation assesses AI systems directly — against benchmarks, ground-truth labels, or human judgments — rather than studying human learners. This includes [[benchmark|benchmarks]], [[automatic-short-answer-grading|grading accuracy]], [[teachbench-llm-teaching-evaluation|teaching-ability evaluation]], and LLM-as-judge approaches. This is the closest method to [[ai-ed-evaluation]] (see the distinction below). - **Strengths:** fast, scalable, reproducible; enables head-to-head comparison of models and system versions; essential for system development and quality assurance. - **Limitations:** measures system output, not learning — high benchmark accuracy does not entail educational effectiveness; ground-truth and rubric quality are themselves contested; can miss pedagogical quality that humans perceive. - **Exemplars:** [[teachbench-llm-teaching-evaluation]], [[jeon-isd-agent-bench-2026]], [[ground-truth-reliability-aied]], [[automatic-short-answer-grading]], [[educational-vlm-evaluation]]. ### Other designs: longitudinal, case, and simulation studies Beyond the major families, the wiki uses **longitudinal** designs that track learners over time ([[ai-lms-middle-school-longitudinal|a longitudinal LMS study]]), **case and in-the-wild** studies of authentic usage ([[ai-in-the-wild-college|large-scale analysis of real student interactions]]), and **simulation** studies in which LLMs stand in for students or patients ([[llm-student-simulation-teacher-insights|LLMs as simulated learners]], [[simulation]]). These trade breadth or control for realism and for access to phenomena that are otherwise hard to observe. ### Expert-consensus methods: the Delphi technique The Delphi method is a structured technique for establishing **expert consensus** on a question where the answer is not yet known empirically — most often used in the wiki to develop frameworks, competency lists, and definitions that practitioners and researchers can agree on. In a Delphi study, a panel of experts responds to successive rounds of questionnaires; after each round, an anonymized summary of the group's responses is fed back, and experts revise their answers until the group converges on agreement (typically defined by a pre-set threshold, e.g., 75%). It is a way to build construct validity and professional consensus through iterative, anonymized consultation rather than a single survey or vote. - **Strengths:** produces consensus from a diverse expert panel without in-person group pressures (anonymity reduces dominance effects); well-suited to defining constructs, competencies, and frameworks when no validated measure exists; iterative rounds let experts refine and converge; feasible where full experiments or large samples are impractical. - **Limitations:** consensus reflects expert judgment, not empirical evidence — it establishes agreement, not effect; results depend on panel composition and the (subjective) consensus threshold; can be slow across multiple rounds; a single panel's judgement may not generalize. - **Exemplars:** [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl|the SAIL framework study]] (three rounds, 17 experts, refining AI-literacy competency levels), [[hcap-human-centric-ai-pedagogy-framework-2026|the HCAP framework study]] (three rounds, 30 teachers, defining 25 AI-teacher competencies), [[ai-literacy-heptagon-2026|the AI Literacy Heptagon]] (which used expert input/consensus alongside a PRISMA-guided review), and [[brookings-ai-students-report|the Brookings students-and-AI report]]. Delphi is often combined with other methods — for example, expert consensus can be used to validate a framework (as in SAIL and HCAP) that is then tested or implemented via design-based research or survey studies. It sits alongside qualitative and expert-judgment approaches and contributes to the [[educational-measurement|validity]] of framework-based instruments. ### Research vs. evaluation: connections and distinctions Research and evaluation are closely related but distinct. **Research** asks generalizable questions about how AI affects learning — "does scaffolding improve learning outcomes?" — and aims to build theory and evidence that transfers beyond the specific study. **Evaluation** (see [[ai-ed-evaluation]]) assesses whether a *specific* AI tool or system works — is accurate, reliable, pedagogically sound, and fit for purpose — against benchmarks, rubrics, or stakeholder-defined criteria. Research emphasizes internal validity and generalization; evaluation emphasizes system quality and local decision-making. The boundaries blur: benchmark studies are evaluation that can feed research, and evaluation instruments (rubrics, ground-truth sets, validity frameworks) depend on the [[educational-measurement]] and [[assessment-validity]] concerns that research clarifies. Conversely, research findings on what supports learning should inform how AI tools are [[ai-ed-evaluation|evaluated]]. The wiki treats them as complementary: computational and benchmark evaluation ([[benchmark]], [[ai-ed-evaluation]]) tells us whether an AI system is technically sound, while efficacy and survey research ([[efficacy-study]], [[rct]]) tells us whether it helps people learn. ### Choosing among methods Method choice follows the research question. Causal-effect questions favor experiments ([[rct]]); mechanism and perception questions favor surveys and qualitative work; system-quality questions favor computational evaluation ([[benchmark]], [[ai-ed-evaluation]]); synthesis questions favor reviews and meta-analyses; design questions favor DBR; and questions about what experts agree a construct, competency, or framework should contain favor expert-consensus methods like the Delphi technique. Given the field's heterogeneity and the speed of AI change, the wiki's corpus reflects a deliberate move toward triangulation — combining computational evaluation with efficacy, qualitative, and expert-consensus evidence to judge both whether a tool works and whether it helps learning. ## Connected Concepts - [[ai-ed-evaluation]] - [[efficacy-study]] - [[rct]] - [[benchmark]] - [[educational-measurement]] - [[assessment-validity]] - [[simulation]] - [[ai-education]] - [[higher-ed]] ## Connected Articles - [[access-not-enough-ai-tutoring-2026]] — Access is Not Enough: Human Support Improves Engagement with AI Tutoring - [[genai-can-harm-teaching-rct-2026]] — Generative AI Can Harm Teaching - [[genai-over-reliance-learning-2026]] — From Enhancement to Over-Reliance: A Mixed-Method Study - [[acceptance-ai-english-tools-2026]] — Acceptance of AI-Assisted English Language Learning Tools - [[ai-tutor-safety-harms]] — AI Tutor Safety and Pedagogical Harms - [[zerkouk-comprehensive-review-its-2025]] — Comprehensive Review of Intelligent Tutoring Systems - [[ai-assisted-collaborative-learning-model-dbr]] — Design-Based Research for an AI-Assisted Collaborative Learning Model - [[teachbench-llm-teaching-evaluation]] — TeachBench: Evaluating LLM Teaching Ability - [[ground-truth-reliability-aied]] — Modernizing Ground Truth: Four Shifts Toward Reliability and Validity - [[llm-student-simulation-teacher-insights]] — Can LLMs Effectively Simulate Human Learners? - [[ai-lms-middle-school-longitudinal]] — AI-Integrated Learning Management System: A Longitudinal Study - [[ai-in-the-wild-college]] — AI in the Wild: Large Scale Analysis of Authentic Interactions - [[same-ai-different-pathways]] — Same AI, Different Pathways: Unpacking Mechanisms - [[t2i-competence-paradox-2026]] — The Competence Paradox: Text-to-Image GenAI in Art and Design --- ## [Scaffolding](https://edtechdev.github.io/aied/concepts/scaffolding/) > **Scaffolding** — structured support that helps learners accomplish tasks they cannot yet complete independently, with support fading as competence grows. In AI in education, scaffolding is the primary design principle for ensuring AI tools support learning rather than replace it. ### How scaffolding appears in AIED - **Prompt-based scaffolding:** [[guided-llm-scaffolding-independent-learning|Guided LLM scaffolding]] teaches structured prompting as a learning intervention. [[scaffolding-critical-engagement-genai-minority-students|Critical engagement scaffolding]] uses culturally responsive approaches. - **Socratic scaffolding:** [[socratic-method|Socratic AI dialogue]] withholds direct answers, using questions to guide discovery — a form of [[desirable-difficulties]] scaffolding. - **Adaptive fading:** [[intelligent-tutoring|Intelligent tutoring systems]] adjust scaffolding based on [[knowledge-tracing]] estimates, providing more support for unmastered concepts and less for known ones. - **Hint systems:** [[correct-answer-trap-ai-tutor|AI tutor hint research]] examines when hints help versus when they encourage over-reliance. - **Conceptual scaffolds:** [[concept-catalyst-engineering-scaffolds|Concept Catalyst]] and [[rethinking-scaffolding-llm-tutors|LLM tutor rethinking]] explore design patterns for cognitive support. ### The ZPD connection [[zone-of-proximal-development|Vygotsky's Zone of Proximal Development]] provides the theoretical foundation: scaffolding targets the space between what learners can do independently and what they can achieve with support. AI tools should operate in this zone — enough support to enable progress, not so much that learning is bypassed. ### Connections Scaffolding connects to [[over-reliance]] (scaffolding that doesn't fade creates dependency), [[cognitive-load-theory]] (scaffolding manages cognitive load), [[feedback-loop]] (scaffolding provides formative feedback), and [[ai-literacy]] (learners must recognize when scaffolding is beneficial vs. when it displaces learning). ## Connected Concepts - [[zone-of-proximal-development]] - [[over-reliance]] - [[cognitive-load-theory]] - [[feedback-loop]] - [[ai-literacy]] - [[intelligent-tutoring]] - [[socratic-method]] - [[metacognition]] - [[personalized-learning]] - [[adaptive-learning]] - [[llm]] - [[higher-ed]] ## Connected Articles - [[guided-llm-scaffolding-independent-learning]] - [[scaffolding-critical-engagement-genai-minority-students]] - [[rethinking-scaffolding-llm-tutors]] - [[concept-catalyst-engineering-scaffolds]] - [[correct-answer-trap-ai-tutor]] - [[critical-thinking-genai-scaffolding]] - [[veriforge-narrative-drafting-scaffolding-2026]] --- ## [Self-Determination Theory](https://edtechdev.github.io/aied/concepts/self-determination-theory/) > **Self-Determination Theory (SDT)** — a psychological theory of human motivation positing that intrinsic motivation and well-being depend on satisfying three basic psychological needs: autonomy, competence, and relatedness. In AI in education, SDT provides a framework for designing AI tools and professional development that support rather than undermine learners' and teachers' motivation. SDT is increasingly used in AI in education research as a theoretical lens for both learner-facing and teacher-facing AI systems. The theory's central claim — that motivation is not simply a quantity learners have but a quality shaped by the social and technological environment — makes it directly relevant to questions about how AI tools affect engagement, persistence, and learning outcomes. The articles in this wiki apply SDT across three main contexts: teacher professional development, AI-mediated learning engagement, and affective computing. ### Key research themes **SDT-based teacher professional development** applies the theory's need-supportive principles to prepare educators for AI. **[[teacher-education-ai-literacy-sdt-2026|Chiu et al.]]** studied 382 secondary school teachers, finding that need-supportive professional development grounded in SDT enhances teachers' AI literacy and fosters sustained behavioral engagement in online professional learning communities. Qualitative analysis identified nine design strategies supporting autonomy, competence, and relatedness — bridging the gap between isolated professional development and professional learning communities. **SDT in AI-mediated learning engagement** examines how generative AI tools shape student motivation. **[[students-engagement-with-generative-ai-in-academic-learning-a-self-determination|Isaeva et al.]]** combined SDT with epistemic network analysis to study students' engagement with generative AI in academic learning. **[[ai-availability-student-motivation]]** explores how AI availability affects student motivation and persistence, connecting to [[over-reliance]] concerns about motivation erosion. **SDT in affective computing** applies the theory to emotionally intelligent AI agents. **[[from-emotion-regulation-to-academic-success-a-self-determination-theory-based-em|Zheng et al.]]** developed EmoAgent, an SDT-based emotional agent that proactively detects students' emotional states and provides emotion regulation strategies. An 8-week quasi-experiment with 173 sixth graders showed the SDT-based approach significantly outperformed conventional agents in academic achievement, engaged students in positive emotional experiences, and moderated the negative effect of negative emotions on success. ### Connections to related concepts SDT connects directly to [[motivation]] as its parent construct, to [[affective-computing]] and [[affective-tutoring]] for emotion-aware AI design, and to [[student-experience]] for how learners experience AI-mediated environments. The theory's emphasis on autonomy connects to [[self-regulated-learning]], while its competence dimension connects to [[self-efficacy-tutoring-learning]] and [[teacher-ai-competency]]. SDT is particularly relevant to [[professional-training]] and [[faculty-development]] because need-supportive design is a transferable principle for preparing educators to use AI. ## Connected Concepts - [[motivation]] - [[student-experience]] - [[affective-computing]] - [[affective-tutoring]] - [[self-regulated-learning]] - [[teacher-ai-competency]] - [[faculty-development]] - [[professional-training]] - [[over-reliance]] - [[engagement-metrics]] - [[ai-education]] ## Connected Articles - [[teacher-education-ai-literacy-sdt-2026]] - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] - [[from-emotion-regulation-to-academic-success-a-self-determination-theory-based-em]] - [[ai-availability-student-motivation]] - [[not-a-universal-benefit-examining-the-differential-effects-of-emotional-ai-on-l2]] --- ## [Self-Efficacy](https://edtechdev.github.io/aied/concepts/self-efficacy/) > **Self-efficacy** — a learner's belief in their capability to successfully perform a task or achieve a goal. Drawing on social cognitive theory (Bandura), self-efficacy shapes motivation, effort, persistence, and learning engagement. In AI in education, self-efficacy matters in two ways: AI tools can build learners' confidence and autonomy (e.g., by providing feedback and scaffolding), and learners' AI self-efficacy — their confidence in using AI technologies — influences how effectively they engage with AI, including how AI-related knowledge translates into career-relevant readiness. Self-efficacy is distinct from actual competence: it is a belief about capability that drives behaviour. It is closely related to — and often used interchangeably with — the everyday notion of *confidence* in one's abilities. Self-efficacy connects closely to [[motivation]], [[self-regulated-learning]], and [[student-experience]]. In the AI context, AI self-efficacy (confidence in working with AI) is a distinct construct from AI literacy, and research shows it plays a crucial role in whether learners actually activate and apply AI-related knowledge. ### How self-efficacy appears in the wiki's research - **AI self-efficacy and career readiness:** [[ai-literacy-career-adaptability-business-2026|Research on AI readiness]] shows that AI self-efficacy moderates the relationship between AI literacy and AI readiness: literacy translates into readiness only when learners have confidence in using AI, and self-efficacy directly predicts career adaptability. - **Robotics and hands-on learning:** [[remind-robot-mediated-roleplay-antibullying-2026|REMind]]'s robot-mediated role-play built children's self-efficacy in anti-bullying intervention; robotics and [[educational-robotics|embodied learning]] generally build confidence by grounding tasks in observable outcomes. - **Teacher self-efficacy:** [[teacher-ai-competency|Teacher AI competency]] research examines how professional development builds teachers' confidence in using AI, which affects adoption and integration. - **Feedback and confidence:** AI feedback and [[ai-tutoring|tutoring]] can build learner self-efficacy by providing actionable, supportive feedback. Self-efficacy connects to [[motivation]], [[self-regulated-learning]], [[student-experience]], [[ai-literacy]], [[agency]], and [[educational-robotics]]. Building self-efficacy is a key mechanism through which AI supports engagement and learning. ## Connected Concepts - [[motivation]] - [[self-regulated-learning]] - [[student-experience]] - [[ai-literacy]] - [[agency]] - [[educational-robotics]] ## Connected Articles - [[ai-literacy-career-adaptability-business-2026]] — AI Literacy, AI Readiness, and Career Adaptability - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[teacher-education-ai-literacy-sdt-2026]] — Teacher Education for AI Literacy (SDT) - [[social-robot-study-companions]] — Social Robots as Study Companions - [[hcap-human-centric-ai-pedagogy-framework-2026]] — HCAP Framework --- ## [Self-Regulated Learning](https://edtechdev.github.io/aied/concepts/self-regulated-learning/) > Self-regulated learning (SRL) describes learners as active participants who can shape and develop their cognitive and behavioral actions in a successful way. AI tools can either scaffold SRL development or inadvertently short-circuit it by removing the regulatory demands that build expertise.^[[scheu-mobile-chatbot-journaling-motivation-2026]]^[[stanford-evidence-base-ai-k12-2026]] SRL is the process whereby learners actively manage their own learning through three interrelated phases: 1. **Forethought:** Goal setting, strategic planning, self-efficacy beliefs 2. **Performance:** Strategy deployment, self-observation, attention focusing 3. **Self-reflection:** Self-evaluation, causal attribution, adaptation Proficient self-regulated learners employ cognitive strategies to improve success and utilize [[metacognition]] to refine their learning processes continuously.^[[scheu-mobile-chatbot-journaling-motivation-2026]] ## Digital Support for SRL ### Learning Journals Learning journals are a promising SRL intervention: by reflecting on their learning processes, students increase awareness of cognition and strengthen regulatory capacity. Key design considerations: - **Structure matters:** Open-ended journals often produce shallow entries; guided prompts and example models improve depth - **Motivation decay:** Mobile journaling apps commonly see rapid engagement decline after a few days - **Scaffolding trade-off:** AI assistance that writes reflections for students undermines the SRL practice; assistance that structures prompts without authoring content preserves it ### Scheu et al.'s 2×2 Experiment (2026) In a randomized field experiment with 179 students over 22 days, two design principles were compared: | Principle | Mechanism | Effect on SRL | Effect on Motivation | Effect on Engagement | |---|---|---|---|---| | **Example-based course** | 7-day curriculum teaching reflective journaling via modeled responses | Increased perceived competence and enjoyment | **Positive** | Constant positive | | **LLM journaling assistant** | GPT-3.5 summarizes drafts, asks clarifying questions, suggests reformulations | No direct SRL skill effect measured | **No effect** | Increasing over time (feedback loop) | **Key insight:** The course improved SRL skills *and* intrinsic motivation through skill transfer, while the assistant improved engagement without affecting motivation.^[[scheu-mobile-chatbot-journaling-motivation-2026]] ## AI Tools and the SRL–Motivation Reciprocal Loop A foundational principle of SRL theory is that self-regulation skills and motivation form a **reciprocal relationship**: - Better SRL → more successful learning → higher self-efficacy → stronger motivation - Higher motivation → more effortful engagement → better SRL practice AI tools can enter this loop at different points: - **SRL-first design** (e.g., structured courses, graduated hints, reflection prompts): Strengthens the loop by building genuine skill - **Engagement-first design** (e.g., autocomplete, content generation): May boost behavioral engagement without entering the motivation loop, risking tool dependence ## Relationship to Tutoring-Specific Design [[tutoring-specific-vs-general-ai|Tutoring-specific AI]] aligns with SRL-first design: it provides graduated scaffolds that preserve learner agency and require strategic self-regulation. General-purpose AI often removes the regulatory demands entirely.^[[stanford-evidence-base-ai-k12-2026]] For example: - Bastani et al.'s tutoring-specific chatbot preserved step-by-step reasoning (SRL demand) - The general-purpose GPT variant simply provided answers (SRL bypass) ## Implications - **For journaling/chatbot tools:** Combine SRL instruction (course-based) with optional writing support to get both motivation and engagement gains - **For AI policy:** Procurement criteria should ask whether a tool develops or displaces self-regulation - **For researchers:** Long-term studies measuring SRL outcomes (not just immediate performance) are essential ## Connected Concepts - [[metacognition]] - [[ai-literacy]] - [[scaffolding]] - [[over-reliance]] - [[intelligent-tutoring]] - [[student-experience]] - [[adaptive-learning]] - [[formative-assessment]] - [[llm]] - [[teacher-role]] - [[higher-ed]] - [[generative-ai]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[ai-guided-learning-audiovideo-2026]] - [[agents-that-teach-incidental-learning]] - [[ai-fatigue-academic-contexts]] - [[ai-learning-assistants-higher-ed-large-scale]] - [[ai-learning-companions-framework]] - [[ai-making-us-stupid]] - [[ai-tutor-safety-harms]] - [[aied-unfinished-mission-bypass]] - [[authentic-assessment]] - [[bloom-aligned-educational-control-llms]] - [[codify-socratic-programming-tutor]] - [[cognitive-offloading-llm-synthesis-writing]] - [[critical-thinking-genai-scaffolding]] - [[cross-subject-validity-delayed-start]] - [[curiobot-llm-tutoring-exploratory-learning]] - [[ecnuclaw-k12-personalized-companion]] - [[learnity-graphs-lifelong-learning-framework-2026]] - [[ai-feedback-enactment-workflow-2026]] --- ## [Simulating Students](https://edtechdev.github.io/aied/concepts/simulating-students/) > **Simulating students** — using LLM-based agents to model learner behavior, cognition, and social dynamics for educational research, design, and training. Simulated students let researchers evaluate pedagogical approaches, model diverse learner profiles, test educational AI before deployment, and train teachers — tasks that are difficult, slow, or ethically constrained to do systematically with real learners. Simulated students are a methodological tool: agents that stand in for real learners so that tutoring systems, curricula, and instructional strategies can be evaluated and iterated without recruiting cohorts of human students. Large language models have made this paradigm far more scalable and linguistically realistic than the rule-based simulated learners that preceded them, while also introducing new validity challenges. ### Why simulate students - **Evaluating pedagogy:** testing instructional approaches across many learner profiles in a controlled, repeatable way. - **Modeling diverse learners:** capturing variation in cognitive levels, learning styles, prior knowledge, and misconceptions that is hard to assemble in a real cohort. - **Testing educational AI:** validating tutoring and assessment systems before live deployment, and generating training data. - **Teacher training:** letting instructors practice tutoring and classroom management with simulated, often imperfect, learners. ### The core challenge: realistic imperfection The defining difficulty of student simulation is that LLMs are trained to be "helpful assistants" that produce correct, polished answers. Yet real students are imperfect — they make characteristic mistakes, hold misconceptions, and learn gradually. A simulated student that answers perfectly (or too randomly) is not a valid model of a learner. Research frames this as the **competence paradox**: broadly capable LLMs asked to emulate partially knowledgeable learners produce unrealistic error patterns and learning dynamics. Addressing it requires constraining the simulation so it reflects a genuine epistemic state — what the learner knows, how errors are structured, and how state evolves — rather than the model's full competence. Techniques include cognitive prototypes grounded in [[knowledge-graph]] or [[knowledge-tracing]] models, explicit epistemic state specifications, and state-transition models of learning rather than simple persona-conditioned role-play. ### Fidelity over surface realism Validity is the central concern: a simulated student is only useful if its behavior is **epistemically faithful** — reflecting the intended learner's knowledge state — not merely linguistically plausible. Research warns against **sycophancy**, where a "simulated student" simply agrees with the tutor rather than exhibiting the misconceptions it was meant to embody. This connects to [[trust-calibration]] and to the broader problem of evaluating whether an agent genuinely models a construct rather than reproducing surface behavior. ### Connection to the wiki Simulating students sits at the intersection of [[simulation]], [[student-modeling]], and [[knowledge-tracing]]. It is a distinct use of [[generative-ai]] in education (modeling learners rather than tutoring them) and an application of [[agentic-ai]] multi-agent systems. It supports [[intelligent-tutoring]], [[adaptive-learning]], [[personalized-learning]], and [[teacher-role]] development, and it overlaps with patient simulation for professional training (e.g., [[special-education]] and medical education contexts). ## Connected Concepts - [[simulation]] - [[student-modeling]] - [[knowledge-tracing]] - [[cognitive-diagnosis]] - [[agentic-ai]] - [[pedagogical-agent]] - [[intelligent-tutoring]] - [[adaptive-learning]] - [[personalized-learning]] - [[generative-ai]] - [[llm]] - [[teacher-role]] ## Connected Articles - [[llm-student-simulation-teacher-insights]] — Can LLMs Simulate Human Learners? Teachers' Insights - [[llm-student-simulation-misconception-faithfulness]] — Simulating Students or Sycophantic Problem Solving? - [[history-aware-student-simulation]] — History-Aware Profiles for Student Simulation - [[llm-educational-simulation-adhd]] — LLM-Based Educational Simulation and Student Persona Stability - [[simulating-students-java-programming-errors-llms]] — Simulating Students' Java Programming Errors - [[adaptive-virtual-patient-psychotherapy-training]] — Adaptive Virtual Patients for Psychotherapy Training - [[medeasy-ai-standardized-patients]] — MedEasy: AI Standardized Patients - [[simulating-students-diverse-cognitive-levels-2025]] — Embracing Imperfection: Simulating Diverse Cognitive Levels - [[simulating-students-llm-review-2026]] — Simulating Students with LLMs: A Review - [[valid-student-simulation-llm-2026]] — Towards Valid Student Simulation - [[agentschool-multi-agent-simulation-education-2026]] — AgentSchool: Multi-Agent Simulation for Education - [[inside-llm-student-simulator-reasoning-2026]] --- ## [Simulation](https://edtechdev.github.io/aied/concepts/simulation/) > **Simulation** — the use of modeled environments, agents, or scenarios to support learning through practice and feedback in contexts that are safe, repeatable, and often otherwise inaccessible. Simulations let learners act, make errors, and see consequences without real-world cost, and are increasingly powered by AI and agent-based modeling. Simulation sits at the core of experiential and [[active-learning]] pedagogies. It provides the deliberate practice, productive failure, and feedback loops that build skill and judgment. AI has transformed simulation in two ways: it powers more realistic and adaptive simulated environments, and it generates simulated learners, patients, or interlocutors that make practice scalable. ### AI and simulation - **AI-powered environments:** adaptive simulations adjust difficulty and scenarios to a learner's state, linking to [[adaptive-learning]] and [[reinforcement-learning]]-based coaching. - **Simulated agents:** AI can simulate patients (for medical training), students (for teacher practice), or conversation partners, making high-stakes interpersonal practice accessible and repeatable. - **Simulated learners:** models of student behavior let researchers and designers test tutoring systems and curriculum before live deployment, grounding [[student-modeling]] and [[knowledge-tracing]]. - **Trust and fidelity:** the value of a simulation depends on how faithfully it models the real context — and on the learner's awareness of its limits, connecting to [[trust-calibration]]. ### Connections Simulation connects to [[active-learning]], [[adaptive-learning]], and [[pedagogical-agent]]. It is a mechanism for experiential and [[constructivist]] learning and is amplified by AI's ability to generate adaptive, realistic practice environments. ## Connected Concepts - [[active-learning]] - [[adaptive-learning]] - [[pedagogical-agent]] - [[reinforcement-learning]] - [[student-modeling]] - [[constructivist]] - [[trust-calibration]] - [[professional-training]] ## Connected Articles - [[adaptive-virtual-patient-psychotherapy-training]] — Adaptive Virtual Patients for Psychotherapy Training - [[ai-enabled-serious-games]] — AI-Enabled Serious Games - [[anvil-ai-educational-animations]] — ANVIL: Analogies and Videos for Lecturers - [[astra-atco-training-simulator]] — ASTRA: ATCO Training Simulator - [[supplynet-visual-exploratory-learning]] — SupplyNet: Visual Exploratory Learning - [[medeasy-ai-standardized-patients]] — MedEASY: AI Standardized Patients - [[hdr-brachytherapy-agentic-ai-simulation-2026]] - [[residencyrl-clinical-rl-training-2026]] --- ## [Social-Emotional Learning](https://edtechdev.github.io/aied/concepts/social-emotional-learning/) > **Social-emotional learning (SEL)** — the process of developing the competencies that enable individuals to synchronize thoughts, emotions, and actions to foster positive interactions with oneself and others: self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (the CASEL framework). In AI in education, SEL is increasingly recognized as critical because the rapid integration of generative AI into learning raises questions about students' well-being, sociability, empathy, and trust — and because technical AI literacy alone is insufficient for navigating AI-mediated learning environments. Social-emotional learning is closely related to, but distinct from, emotional intelligence (EI): SEL/SEC (social-emotional competencies) encompasses the ability to synchronize thoughts, emotions, and actions for positive interactions, while EI is an individual's capacity to process emotional information (conceptualized through ability models — reasoning and problem-solving — or trait models — emotional dispositions and behaviours). In the AI era, SEL matters because AI can reshape learning in ways that affect students' relational and emotional development, and because educators need both technological skill and emotional intelligence to support learners effectively. ### How SEL appears in the research - **Integrating SEC into AI literacy:** [[sec-ai-literacy-narrative-review-2026|The narrative review by Palmquist et al.]] proposes an integrated framework that combines AI literacy with social-emotional competencies, arguing that technical proficiency alone is insufficient — educators and students need both technological and emotional intelligence to navigate AI-mediated learning environments, fostering personalized learning, collaboration, and ethical engagement. - **Teachers and relational practice:** Research on [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl|AI literacy frameworks]] and [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai|teacher-student trust]] emphasizes that SEL supports the relational dimensions of learning (teacher-student and student-student relationships), which AI must complement rather than replace. - **Well-being and AI's affective impact:** [[brookings-ai-students-report|Reports on AI and students]] examine how the increasing use of generative AI affects students' socio-emotional skills, well-being, sociability, and sense of trust and empathy — concerns that motivated the OECD's call for AI literacy grounded in humanistic, social, and emotional values. - **Affective dimensions of AI:** SEL connects to [[affective-computing]] and [[well-being]] research, examining how AI systems can support or undermine emotional and relational learning. ### SEL as a complement to AI literacy A recurring theme in the wiki's research is that AI literacy should not be reduced to technical skill. Integrating social-emotional competencies into AI literacy supports a holistic approach where digital and relational competencies are equally valued: AI-driven personalisation can enhance relational dynamics if designed carefully, educators can focus more on students' emotional and cognitive development as AI automates administrative tasks, and ethical literacy (data privacy, bias, equity, responsible decision-making) overlaps with SEL's emphasis on social awareness and responsible decision-making. ### Connections to related concepts SEL connects to [[ai-literacy]] (as a complement that makes AI literacy relational and ethical), [[affective-computing]] and [[well-being]] (the affective dimensions of AI), [[self-regulated-learning]] (self-management), [[ethics]] (responsible decision-making), [[teacher-ai-competency]] (educators' capacity to support SEL), and [[student-experience]] (well-being and belonging). It relates to [[higher-ed]] and [[k-12]] as the settings where SEL-infused AI literacy is cultivated. ## Connected Concepts - [[ai-literacy]] - [[affective-computing]] - [[well-being]] - [[self-regulated-learning]] - [[ethics]] - [[teacher-ai-competency]] - [[student-experience]] - [[higher-ed]] ## Connected Articles - [[sec-ai-literacy-narrative-review-2026]] — Integrating Social-Emotional Competencies Into AI Literacy - [[mind-the-trust-gap-teacher-student-views-control-agency-k12-classroom-ai]] — Mind the Trust Gap: Teacher-Student Views - [[brookings-ai-students-report]] — Brookings Report on AI and Students - [[the-scaffolded-ai-literacy-sail-framework-results-of-a-delphi-study-for-equitabl]] — The Scaffolded AI literacy (SAIL) framework - [[teacher-education-ai-literacy-sdt-2026]] — Teacher Education for AI Literacy (SDT) --- ## [Social Robots](https://edtechdev.github.io/aied/concepts/social-robots/) > **Social robots** — robots designed to engage people through social interaction, using human-like cues such as speech, gesture, facial expression, and personality to communicate, teach, assist, or accompany. In education, social robots (humanoids like iCub, Pepper, Reachy, and companion robots) are used for tutoring, storytelling, role-play, language support, and as study companions. Their social presence is the key differentiator from software-based AI agents, enabling relationship-building and emotional engagement. Social robots are distinct from [[educational-robotics|educational robots]] used purely for programming practice: they are designed for *social* interaction rather than just mechanical manipulation. Advances in [[llm|large language models]] have dramatically expanded what social robots can say and do, enabling fluent, adaptive conversational tutoring. However, this also introduces risks such as misinformation, overreliance, and privacy violations, motivating knowledge-based design approaches. ### How social robots appear in the wiki's research - **LLM-powered tutoring:** [[knowledge-based-design-generative-social-robots-2026|Knowledge-based design research]] and [[teachy-mini-generative-social-robot-higher-ed-2026|Teachy Mini]] develop generative social robots (GSRs) that tutor higher-education students, using knowledge-based design requirements to ensure responsible, effective behaviour. - **Storytelling and engagement:** [[robobuddy-llm-social-robots-classroom-2025|RoboBuddy]] lets teachers create LLM-powered scenario-based storytelling activities, [[motibo-digital-storytelling-robots-motivation-2026|MotiBo]] uses a human-like robot for interactive digital storytelling to boost motivation, and [[icub-humanoid-storytelling-llm-hri-2025|the iCub study]] explores co-creative narrative interaction. - **Social-emotional learning:** [[remind-robot-mediated-roleplay-antibullying-2026|REMind]] uses social robots to rehearse anti-bullying bystander intervention through role-play. - **Sign language and inclusion:** [[pepper-robot-sign-language-lis-2025|Pepper sign-language research]] explores robots communicating in Italian Sign Language to support Deaf users. - **Trust and autonomy:** [[task-context-trust-educational-hri-2026|Research on trust]] shows task context shapes trust in social robots more than appearance; [[human-autonomy-agency-hri-review-2025|a systematic review]] examines how HRI affects human autonomy and sense of agency. Social robots connect to [[human-robot-interaction]], [[educational-robotics]], [[embodied-learning]], and [[student-engagement|engagement]]. They are increasingly relevant to [[social-emotional-learning]], [[equity-in-ai-education|inclusive education]], and [[higher-ed]]/[[k-12]] tutoring, with design grounded in [[agency]], [[trust]], [[self-efficacy]], and [[ethics]]. ## Connected Concepts - [[human-robot-interaction]] - [[educational-robotics]] - [[embodied-learning]] - [[llm]] - [[student-engagement]] - [[social-emotional-learning]] - [[agency]] - [[trust]] - [[ethics]] - [[higher-ed]] ## Connected Articles - [[knowledge-based-design-generative-social-robots-2026]] — Knowledge-Based Design for Generative Social Robots - [[teachy-mini-generative-social-robot-higher-ed-2026]] — Teachy Mini - [[robobuddy-llm-social-robots-classroom-2025]] — RoboBuddy - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[pepper-robot-sign-language-lis-2025]] — Pepper and Sign Language - [[icub-humanoid-storytelling-llm-hri-2025]] — iCub Narrative HRI - [[social-robot-study-companions]] — Social Robots as Study Companions - [[enhancing-creative-writing-with-robot-llm-integration-the-interplay-of-embodimen]] — Robot-LLM Integration in Creative Writing --- ## [Socratic AI Dialogue](https://edtechdev.github.io/aied/concepts/socratic-ai-dialogue/) > Socratic dialogue — asking structured questions rather than providing answers — is one of the strongest pedagogical scaffolds for deep learning. When automated via AI, it produces measurable reasoning gains but also requires careful calibration to avoid frustrating learners or displacing human mentorship.^[[hashmi-socratic-physics-chatbot-2025]]^[[favero-critical-ai-tutors-empower-enslave-2025]] Deployed a custom Socratic AI chatbot in a large-enrollment introductory mechanics course: | Metric | Result | |---|---| | **Sample** | 150 first-year STEM majors | | **Knowledge-based skills rating** | Median **4.0/5** | | **Overall effectiveness rating** | Median **3.4/5** (notable gap) | | **Question specificity (first turn)** | ~10–15% | | **Question specificity (final turn)** | **100%** | | **Specificity × grade correlation** | Pearson **r = 0.43** | **Interpretation:** Students began with vague, generic questions but progressively sharpened them through Socratic interaction — a clear indicator of developing expert-like reasoning. The positive correlation between question specificity and self-reported expected grade suggests that learning to ask better questions is itself a domain skill. ## The Effectiveness Gap The gap between "knowledge-based skills" (4.0/5) and "overall effectiveness" (3.4/5) suggests a tension: students recognize that the Socratic bot improved their reasoning, yet do not fully endorse it as a complete tutoring solution. Possible reasons: - Socratic dialogue is effortful; students may prefer direct answers for efficiency - The chatbot cannot provide the relational support of a human tutor - Some students may get stuck in Socratic loops without resolution ## Relationship to Pedagogical Training The Socratic approach directly embodies the principle from [[pedagogical-llm-training|EduQwen]]: **reward "guiding" over "answering."** However, real-time Socratic calibration is harder than paper-bench pedagogy: - EduQwen optimizes for correct guiding on a multiple-choice benchmark - A live Socratic tutor must decide *when* to guide, *when* to hint, and *when* to answer — based on real-time student signals - [[affective-tutoring|Affective state]] is a critical moderator: a frustrated student may need a brief direct answer before returning to Socratic mode ## Agency and Critical Use Favero et al. (2025) caution that even Socratic AI can undermine agency if students become dependent on the questioning structure rather than internalizing it. The goal is not permanent Socratic scaffolding but **scaffolded transfer** — students eventually Socratize themselves. ## Open Questions 1. Does Socratic dialogue transfer across domains, or is physics-specific reasoning non-transferable? 2. How does Socratic specificity correlate with *actual* (not self-reported) course performance? 3. Can Socratic AI be combined with [[ai-peer-feedback-systems|peer feedback]] for social amplification? ## Connected Concepts - [[scaffolding]] - [[metacognition]] - [[intelligent-tutoring]] - [[formative-assessment]] - [[ai-literacy]] - [[cognitive-offloading]] - [[faculty-development]] - [[automated-grading]] - [[personalized-learning]] - [[over-reliance]] - [[llm]] - [[self-regulated-learning]] ## Connected Articles - [[codify-socratic-tutoring-programming]] - [[critical-thinking-genai-scaffolding]] - [[generative-ai-guardrails-harm-learning]] - [[pedagogy-ai-mistakes]] - [[prober-ai-inquiry-writing]] - [[retrieval-augmented-tutoring-algorithm-kite]] --- ## [Socratic Method](https://edtechdev.github.io/aied/concepts/socratic-method/) > **Socratic Method** — a pedagogical approach rooted in guided questioning and dialogue rather than direct instruction, now being adapted for generative AI tutoring systems. In AI in education, the Socratic method is operationalized through LLMs that ask probing questions, scaffold reasoning, and withhold direct answers — aiming to promote deeper understanding and productive struggle rather than answer-fetching. The Socratic method is one of the oldest pedagogical techniques — originating with Socrates in ancient Athens — and it has found new relevance in the age of generative AI. In AI education research, the Socratic method refers to AI systems that engage learners through guided dialogue, posing questions that lead students to discover answers rather than providing them outright. ### How it works in AI tutoring Unlike direct-instruction AI tutors that give answers, Socratic AI tutors use question sequences that: - **Elicit prior knowledge** — asking what the student already knows about a topic - **Probe reasoning** — "Why do you think that?" or "What if the situation were different?" - **Surface misconceptions** — through carefully chosen counterexamples - **Guide toward insight** — without giving the answer away ### Research in the wiki The **[[hashmi-socratic-physics-chatbot-2025|Socratic Physics Chatbot]]** provides empirical evidence that the Socratic method can be operationalized through generative AI at scale, serving simultaneously as a teaching tool and data-collection instrument for [[learning-analytics]]. Unlike rule-based Socratic systems of the past, LLM-based approaches can adapt question sequences dynamically based on student responses. **[[codify-socratic-tutoring-programming|Codify]]** applies the Socratic method specifically to programming education, building an intelligent tutoring system that guides students through problem-solving with incremental questions rather than code solutions — connecting to [[scaffolding]] and [[computational-thinking]]. **[[ai-agents-constructive-conflict-design-education-2026|Adversarial AI agents]]** enact constructive conflict — a Socratic variant — prompting novice designers to reconsider their assumptions, leading to more design iterations and higher-rated final work. This connects Socratic questioning to [[design-thinking]] and [[critical-thinking]]. **[[syal-multimodal-dialogue-stem-2026|Multimodal dialogue systems]]** extend Socratic tutoring to visual domains, using a zero-retraining intervention protocol that asks models to describe, reason, and self-correct — a multimodal Socratic scaffold. ### Connections to other concepts The Socratic method is closely tied to [[scaffolding]] (providing just enough support), productive-struggle (letting students wrestle with difficulty), and [[intelligent-tutoring]] (adaptive question sequencing). It contrasts with [[over-reliance]] — students who receive direct answers may bypass learning, while Socratic guidance maintains cognitive engagement. The approach also connects to [[over-reliance]], as students must learn when to trust AI-generated questions versus when to question them. ## Connected Concepts - [[scaffolding]] - [[intelligent-tutoring]] - [[learning-analytics]] - [[stem-education]] - [[student-modeling]] - [[student-experience]] - [[agentic-ai]] - [[metacognition]] - [[knowledge-tracing]] - [[adaptive-learning]] - [[generative-ai]] - [[over-reliance]] ## Connected Articles - [[ai-agents-constructive-conflict-design-education-2026]] - [[genai-performance-vs-learning]] - [[hashmi-socratic-physics-chatbot-2025]] - [[structured-llm-feedback-programming]] - [[syal-multimodal-dialogue-stem-2026]] - [[zerkouk-comprehensive-review-its-2025]]- [[physics-chatbot-epistemological-beliefs-2026]] - [[embodied-inquiry-ai-facilitator-physics-2026]] --- ## [Special Education](https://edtechdev.github.io/aied/concepts/special-education/) > **Special Education** — the design and delivery of instruction for learners with disabilities, spanning cognitive, physical, sensory, and neurodevelopmental differences. AI in education research in this wiki explores how AI tools can support diverse learner needs through personalization, adaptive scaffolding, and accessible interfaces — while also examining the risks of AI systems that overlook or marginalize disabled learners. Special education is a domain where AI's capacity for personalization and adaptation offers particular promise. Unlike one-size-fits-all instruction, AI tutors can theoretically adapt to individual cognitive profiles, communication needs, and learning paces. The articles in this wiki span AI for specific disability profiles, neurodivergent learner experiences, and critical perspectives on AI and disability. **Disability-specific AI tutoring** tailors AI to particular learner needs. **[[special-r1-rl-special-education|Special-R1]]** extends reinforcement learning to model cognitive and communicative diversity across five disability profiles, using persona-aware prompts and thinking rewards to shape tutor responses for each learner. **[[dyslexlens-dyslexic-learners-ai|DysLexLens]]** analyzed how dyslexic learners experience AI tools, revealing both the value of AI for literacy support and persistent accessibility barriers. **[[embodied-string-learning-blindness-low-vision-musicians]]** developed non-visual learning strategies with blind and low-vision musicians, centering disability-led embodied design. These connect to [[accessible-learning]] and [[neurodiversity]]. **Neurodivergent learner experiences** center autistic and ADHD students. **[[neurodivergent-computing-students|Zastudil et al.]]** found neurodivergent computing students need structured assignments, small consistent teams, and explicit role definitions — design requirements that [[collaborative-learning]] tools must address. **[[adhd-video-segmentation-computing-education]]** demonstrated that AI-segmented videos eliminated the ADHD performance gap. Both connect to [[instructional-design]] and [[universal-design-for-learning]]. **Critical perspectives** examine how AI can marginalize disabled learners. **[[genai-minoritized-knowledges-disability|Tali-Otmani]]** argues that AI systems actively marginalize disability-centered knowledge due to Western-centric training data — connecting to [[equity-in-ai-education]] concerns about epistemic justice. ## Connected Concepts - [[accessible-learning]] - [[equity]] - [[equity-in-ai-education]] - [[neurodiversity]] - [[universal-design-for-learning]] - [[instructional-design]] - [[student-experience]] - [[ai-literacy]] - [[k-12]] - [[higher-ed]] - [[cs-education]] - [[generative-ai]] ## Connected Articles - [[special-r1-rl-special-education]] - [[dyslexlens-dyslexic-learners-ai]] - [[neurodivergent-computing-students]] - [[adhd-video-segmentation-computing-education]] - [[genai-minoritized-knowledges-disability]] - [[embodied-string-learning-blindness-low-vision-musicians]] --- ## [STEM Education and AI](https://edtechdev.github.io/aied/concepts/stem-education/) > **STEM Education** — science, technology, engineering, and mathematics education is the most common domain for AI in education research in the wiki. STEM's structured knowledge, clear right/wrong answers, and computational nature make it an ideal testbed for AI tutoring and assessment. ### STEM as the primary AIED domain - **Mathematics:** [[math-education|Math education]] research spans [[generative-ai-reduced-study-time-math|GenAI impact on math learning]], [[ai-powered-personalized-learning-elementary-fractions-2026|elementary fraction tutoring]], and [[student-math-competence-clustering|competence clustering]]. - **Physics:** [[physics-education|Physics education]] includes [[becker-chatgpt-typology-physics-2026|ChatGPT typology studies]], [[hashmi-socratic-physics-chatbot-2025|Socratic physics chatbots]], and [[ai-scoring-language-bias-physics|scoring bias analysis]]. - **Computer science:** [[cs-education|CS education]] is the most-researched STEM subfield — [[code-review-genai-cs1|code review]], [[debugtracker-classroom-debugging|debugging tools]], and [[prompt-problems-nl-programming-mistakes|prompting studies]]. - **Engineering:** [[concept-catalyst-engineering-scaffolds|Engineering scaffolds]], [[structured-ai-demonstrations-engineering-mechanics|mechanics demonstrations]], and [[ai-engineering-education-balancing-act|curriculum balancing]] bring AI to engineering education. ### Why STEM dominates STEM's structured knowledge representation, verifiable answers, and computational thinking alignment make it the most natural fit for AI tutoring. [[computational-thinking|Computational thinking research]] explores this alignment explicitly. ### Connections STEM education connects to [[cs-education]], [[math-education]], [[physics-education]], [[computational-thinking]], [[k-12]], [[higher-ed]], and [[ai-tutoring]] — it is the domain context for much of the wiki's tutoring and assessment research. ## Connected Concepts - [[cs-education]] - [[math-education]] - [[physics-education]] - [[computational-thinking]] - [[k-12]] - [[higher-ed]] - [[ai-tutoring]] - [[automated-grading]] - [[formative-assessment]] - [[personalized-learning]] - [[llm]] ## Connected Articles - [[ai-acceptance-preservice-science-teachers-2026]] - [[becker-chatgpt-typology-physics-2026]] - [[ai-powered-personalized-learning-elementary-fractions-2026]] - [[concept-catalyst-engineering-scaffolds]] - [[generative-ai-reduced-study-time-math]] - [[ai-metacognition-stem-review]] --- ## [Storytelling in Education](https://edtechdev.github.io/aied/concepts/storytelling-in-education/) > **Storytelling in education** — the use of narrative as a pedagogical tool to engage learners, convey meaning, and support knowledge construction, creativity, and emotional connection. Storytelling is a natural and motivating way for learners to make sense of the world, and it is increasingly combined with technology — including AI and [[social-robots|social robots]] — to create interactive, adaptive narrative experiences. Digital and robot-mediated storytelling can add interactivity, personalization, and embodiment that conventional (paper-based or slide-based) storytelling lacks. Storytelling is grounded in [[motivation]], [[student-engagement]], and constructivist theories of learning. It supports [[language-learning]], [[creativity]], [[social-emotional-learning]], and comprehension. In the AI era, [[llm|LLM-powered]] and robot-mediated storytelling enables co-creation, where learners and AI agents build stories together, and interactive narrative that responds to the learner. ### How storytelling appears in the wiki's research - **Robot-mediated storytelling:** [[motibo-digital-storytelling-robots-motivation-2026|MotiBo]] uses a human-like interactive digital storytelling robot, finding significant gains in behavioural and cognitive engagement over paper and PowerPoint methods; [[robobuddy-llm-social-robots-classroom-2025|RoboBuddy]] lets teachers create LLM-powered scenario-based storytelling activities from curriculum content. - **Co-creative narrative HRI:** [[icub-humanoid-storytelling-llm-hri-2025|The iCub narrative study]] explores human-robot co-creation of stories, integrating generative models for contextually appropriate interaction. - **Narrative and creativity:** Storytelling supports [[creativity]] and [[language-learning|language]] development, and is used to enhance motivation and engagement in [[k-12]] settings. Storytelling connects to [[student-engagement]], [[motivation]], [[creativity]], [[social-robots]], [[language-learning]], [[social-emotional-learning]], and [[educational-robotics]]. ## Connected Concepts - [[student-engagement]] - [[motivation]] - [[creativity]] - [[social-robots]] - [[language-learning]] - [[social-emotional-learning]] - [[educational-robotics]] ## Connected Articles - [[motibo-digital-storytelling-robots-motivation-2026]] — MotiBo - [[robobuddy-llm-social-robots-classroom-2025]] — RoboBuddy - [[icub-humanoid-storytelling-llm-hri-2025]] — iCub Narrative HRI - [[remind-robot-mediated-roleplay-antibullying-2026]] — REMind - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education --- ## [Student Engagement](https://edtechdev.github.io/aied/concepts/student-engagement/) > **Student engagement** — the degree and quality of a learner's active involvement in the learning process, most often decomposed into behavioral, cognitive, and affective dimensions. In AI-education research, student engagement is both a key outcome (does an AI tool keep students engaged?) and a mechanism (does engagement mediate between AI design and learning?). It is conceptually distinct from the specific [[engagement-metrics]] used to measure it, and from learning itself — engagement is participation in learning, not proof of cognitive gain. Engagement is a multidimensional construct rooted in educational psychology. **Behavioral engagement** refers to participation, effort, persistence, and on-task activity. **Cognitive engagement** refers to the depth of mental processing — elaboration, critical analysis, self-regulation, and the investment of mental effort. **Affective engagement** refers to emotional reactions such as interest, enjoyment, anxiety, and identification with learning. These dimensions can diverge: a student may be behaviorally active (clicking, spending time) while cognitively shallow (passively accepting output), or affectively interested while behaviorally distracted. This multidimensionality is why engagement must not be equated with any single observable behavior. ### How student engagement appears in the research - **Engagement as an outcome of AI design:** [[genai-motivation-engagement-2026|GenAI motivation research]] shows that engagement in generative-AI-supported learning follows the satisfaction of basic psychological needs ([[self-determination-theory|autonomy, competence, relatedness]]) — engagement is the downstream result of motivational support, not of technology availability alone. - **Quality over quantity:** [[critical-engagement-code-completion|Critical engagement in AI code completion]], [[icap-cognitive-engagement-llm-agents|cognitive-engagement discourse analysis]], and [[scaffolding-critical-engagement-genai-minority-students|scaffolding critical engagement]] show that *deep* (cognitive) engagement with AI predicts learning, while *shallow* (behavioral) engagement predicts the [[over-reliance]] and learning displacement that dominate the wiki's risk literature. - **Fragile and context-dependent:** [[polished-artifacts-fragile-engagement-2026|Polished artifacts, fragile engagement]] and [[genai-tutor-engagement-patterns|multi-institution engagement patterns]] find engagement varies by task, context, and learner — an AI tool that engages one student deeply may produce shallow, output-chasing behavior in another. - **Motivational antecedents:** [[ai-availability-student-motivation|AI availability and motivation]] shows that knowing AI is available can reduce the perceived value of effortful engagement, particularly for novice learners — engagement is shaped by expectancy, value, and perceived competence as much as by tool features. ### Engagement vs. learning A central theme in the wiki's research is that engagement and learning must be distinguished. AI tools that generate high engagement (time on task, interaction volume) may not produce learning if that engagement is passive or substitutes for the cognitive work of understanding — see [[genai-performance-vs-learning|performance vs. learning]]. Conversely, productive struggle and desirable difficulty can produce learning even when surface engagement feels lower. Engagement is therefore best treated as a *mechanism* — valuable insofar as it reflects or enables meaningful cognitive processing — rather than a terminal outcome. ### Connections to related concepts Student engagement connects to [[motivation]] and [[self-determination-theory]] as its psychological drivers, and to [[student-experience]] as the lived context. Its measurement relies on [[engagement-metrics]] and [[learning-analytics]]. The distinction between deep and shallow engagement ties directly to [[self-regulated-learning]] (self-regulated learners engage strategically), [[cognitive-offloading]] and [[over-reliance]] (shallow reliance as the failure mode), and [[metacognition]]. In system design, engagement signals feed [[student-modeling]] and [[adaptive-learning]], and engagement outcomes feature in [[efficacy-study]] evaluations of AI-education interventions. ## Connected Concepts - [[engagement-metrics]] - [[motivation]] - [[self-determination-theory]] - [[student-experience]] - [[learning-analytics]] - [[self-regulated-learning]] - [[over-reliance]] - [[cognitive-offloading]] - [[metacognition]] - [[higher-ed]] ## Connected Articles - [[genai-motivation-engagement-2026]] — Impact of Generative AI on Student Motivation and Engagement - [[critical-engagement-code-completion]] — To Tab or Not to Tab: Measuring Critical Engagement in AI Code Completion - [[icap-cognitive-engagement-llm-agents]] — Measuring Cognitive Engagement in Collaborative Discourse - [[genai-tutor-engagement-patterns]] — Not All Students Engage Alike: Multi-Institution Patterns - [[polished-artifacts-fragile-engagement-2026]] — Polished Artifacts, Fragile Engagement - [[ai-availability-student-motivation]] — "Why Put in This Much Effort?": How AI Availability Shapes Motivation - [[genai-performance-vs-learning]] — Distinguishing Performance Gains From Learning - [[scaffolding-critical-engagement-genai-minority-students]] — Scaffolding Critical Engagement With GenAI - [[engagement-intensity-learner-modeling]] — Engagement Intensity as a Learner-Modeling Signal --- ## [Student Experience with AI](https://edtechdev.github.io/aied/concepts/student-experience/) > **Student experience with AI** — how learners perceive, interact with, and are affected by AI tools in educational settings. With over 85 articles in the wiki, student experience is one of the most-researched dimensions of AI in education. ### How student experience is studied - **Large-scale surveys:** [[ai-in-the-wild-college|AI in the Wild]] analyzes authentic interactions of thousands of college students, while [[genai-availability-grades-satisfaction|availability and satisfaction studies]] correlate AI access with student outcomes. - **Interaction patterns:** [[tracing-genai-literacy-interaction-patterns|Tracing GenAI literacy]] maps how students engage with AI across assignments. [[misiejuk-cognitive-offloading-prompting-2026|Prompting analysis]] reveals cognitive engagement levels through prompt structure. - **Motivation and agency:** [[ai-availability-student-motivation|AI availability and motivation]] examines whether knowing AI is available changes student effort. [[aied-unfinished-mission-bypass|AIED's unfinished mission]] frames agency and motivation as central challenges. - **Perceptions and attitudes:** [[genai-usage-design-students-survey|GenAI usage surveys]] and [[student-mental-models-genai|mental model studies]] investigate how students understand and trust AI. - **Equity dimensions:** [[student-experience]] intersects with [[equity]] — AI access and effectiveness vary across student populations. ### Connections Student experience connects to [[over-reliance]] (excessive AI dependence), [[ai-literacy]] (skills for effective use), [[cognitive-offloading]] (how AI changes cognitive work), and [[engagement-metrics|engagement]] (how AI systems measure and respond to student behavior). ## Connected Concepts - [[generative-ai]] - [[llm]] - [[higher-ed]] - [[ai-literacy]] - [[over-reliance]] - [[cognitive-offloading]] - [[equity]] - [[k-12]] - [[scaffolding]] - [[metacognition]] - [[self-regulated-learning]] ## Connected Articles - [[genai-student-experiences-uk-he-survey-2026]] - [[metacognitively-discordant-completion-genai-2026]] - [[ai-generated-interactive-fiction-education-2026]] - [[ai-in-the-wild-college]] - [[genai-availability-grades-satisfaction]] - [[tracing-genai-literacy-interaction-patterns]] - [[misiejuk-cognitive-offloading-prompting-2026]] - [[ai-availability-student-motivation]] - [[aied-unfinished-mission-bypass]] - [[student-mental-models-genai]] - [[genai-usage-design-students-survey]] - [[spritz-ai-disciplinary-mediation-student-teams-2026]] --- ## [Student Misconceptions about AI](https://edtechdev.github.io/aied/concepts/student-misconceptions-ai/) > **Student misconceptions about AI** — the inaccurate beliefs students hold about what AI systems are, what they do, and what using them means for learning, especially in academic contexts. Misconceptions are not a single falsehood but a family of calibration errors that cluster around two core mistakes: misjudging what the model is (authority vs. tool, neutral vs. biased, understanding vs. generating) and misjudging what learning requires (output vs. process). Misconceptions about AI matter because they are the cognitive precursor to the harmful behaviors the wiki documents under [[over-reliance]], [[cognitive-offloading]], and [[academic-integrity]] concerns. Students rarely set out to misuse AI; they do so because inaccurate mental models lead them to misplace trust, skip verification, and treat output as understanding. ### What AI misconceptions are A misconception here is not mere ignorance of how a model works — it is an actively held, often self-reinforcing belief that produces systematic errors in how students interact with AI. They are directly analogous to the domain misconceptions studied in learning science: stable, plausible, and resistant to correction until confronted. Correcting them is a core aim of [[ai-literacy]] and [[trust-calibration]] education. ### Common misconceptions in academic contexts - **The authority fallacy** — treating LLM output as verified fact rather than a probabilistic completion. Drives uncritical acceptance and the answer-seeking-over-understanding pattern documented in AI-tutoring research, where learners accept a model's answer without checking it against [[hallucination-risk]]. - **Learning-equals-output** — believing that producing work *with* AI is the same as having learned it. This is the exact error behind [[over-reliance]]: the drafting, recall, and revision processes that build durable knowledge get outsourced. - **The neutrality illusion** — assuming AI is objective and unbiased. Students often miss that models encode training-data biases and that in [[writing-education]] contexts this produces idea homogenization across a cohort. - **The integrity gray zone** — misjudging whether AI use is acceptable. Some students see AI output as "not copying a person" and therefore permissible; others over-correct and think *any* use is cheating. Institutional inconsistency feeds both errors. - **Anthropomorphism** — believing the model has intent, memory, and understanding of *their* context. This over-trust is especially risky academically, because students may rely on plausible-sounding explanations the model cannot actually ground. - **The determinism error** — expecting one query to be enough and not realizing output is non-deterministic and prompt-sensitive. Underestimating this produces the "prompting gap," where students mistake shallow results for the tool's ceiling. - **The detection miscalibration** — underestimating both institutional detection and, more importantly, the self-harm of submitting work they cannot later explain or defend. - **The efficiency illusion** — treating time saved as pure gain, missing that unexercised foundational skills decay and that novices cannot yet tell good output from bad. ### Why misconceptions matter for learning Misconceptions translate directly into the behaviors that cause learning harm. The belief that "AI is always right" suppresses verification; the belief that "using AI is learning" suppresses effortful processing; the belief that "it's not cheating" bypasses the metacognitive review that consolidates understanding. In this sense misconceptions are upstream of the [[ai-misuse-learning-harm]] documented across the wiki's evidence base. ### Correcting misconceptions Correction is not a one-time disclosure but an ongoing [[ai-literacy]] process that develops [[metacognition]] and [[self-regulated-learning]]: helping students monitor their reliance, calibrate when to trust and when to question a model, and see the cost of bypassing their own cognitive work. Because misconceptions are resistant, they are best addressed through direct confrontation with evidence — including the finding that students often *do not perceive* the learning harm of AI misuse. ## Connected Concepts - [[ai-literacy]] - [[trust-calibration]] - [[over-reliance]] - [[cognitive-offloading]] - [[metacognition]] - [[self-regulated-learning]] - [[academic-integrity]] - [[hallucination-risk]] - [[generative-ai]] - [[student-experience]] ## Connected Articles - [[student-rationalization-ai-writing]] — Student Rationalization of AI Writing - [[genai-skill-bypass-literacy]] — GenAI Skill Bypass and Literacy - [[trust-reliance-ai-education-2026]] — Trust and Reliance in AI Education - [[contextual-sycophancy-ai-literacy]] — Contextual Sycophancy and AI Literacy - [[sycophantic-ai-social-interaction-2026]] — Sycophantic AI in Social Interaction - [[llm-fallacy-misattribution]] — LLM Fallacy Misattribution - [[kim-llm-fallacy-misattribution-2026]] — LLM Fallacy Misattribution (Kim et al.) - [[generative-ai-guardrails-harm-learning]] — GenAI Without Guardrails Can Harm Learning --- ## [Student Modeling](https://edtechdev.github.io/aied/concepts/student-modeling/) > **Student modeling** — the broad practice of representing learner characteristics including knowledge, skills, affective states, engagement, and preferences in computational form. Student modeling is the foundation upon which adaptive and personalized AI in education systems are built. Student modeling is the umbrella term encompassing all ways of representing learners in AI systems. Where [[knowledge-tracing]] specifically tracks cognitive knowledge over time, student modeling includes affective modeling, engagement tracking, learning style inference, and preference learning. A complete student model answers: *who is this learner, what do they know, how do they feel, and what do they need next?* ### Dimensions of student modeling - **Cognitive modeling:** [[knowledge-tracing]] and [[cognitive-diagnosis]] systems track what students know and their skill mastery - **Affective modeling:** [[affective-computing]] and [[affective-tutoring]] detect and respond to student emotions and motivation - **Behavioral modeling:** [[learning-analytics]] systems infer engagement patterns from clickstream and interaction data - **Proficiency modeling:** [[personalized-learning]] and [[adaptive-learning]] platforms use student models to sequence content ### LLM-era student modeling Recent advances use [[llm|LLMs]] for richer student modeling. The [[xie-hillm-cd-2026|HiLLM-CD framework]] represents students as proficiency trees, while [[multimodal-knowledge-graph-educational-reasoning|multimodal approaches]] construct evidence-grounded knowledge representations from diverse data sources. LLMs enable automated model construction from educational text, reducing reliance on expert annotation. ### Connections to other concepts Student modeling feeds into [[intelligent-tutoring]] (for problem selection), [[personalized-learning]] (for content sequencing), and [[learning-analytics]] (for dashboards and interventions). It connects fundamentally to [[knowledge-tracing]] for cognitive tracking and to [[adaptive-learning]] for system adaptation. ## Connected Concepts - [[learning-analytics]] - [[knowledge-tracing]] - [[adaptive-learning]] - [[intelligent-tutoring]] - [[personalized-learning]] - [[formative-assessment]] - [[k-12]] - [[affective-tutoring]] - [[llm]] - [[higher-ed]] - [[ai-education]] - [[ai-tutoring]] ## Connected Articles - [[haiml-human-centered-ai-metacognitive-model-2026]] - [[ai-guided-learning-audiovideo-2026]] - [[multimodal-item-parameter-estimation-2026]] - [[at-risk-students-ml-prediction]] - [[cold-start-knowledge-tracing-safeinsights]] - [[correct-answer-trap-misconceptions]] - [[cross-subject-validity-delayed-start]] - [[educlaw-bench-pedagogical-llm-agents-2026]] - [[edumirror-educational-social-dynamics]] - [[huang-interpretable-knowledge-tracing-2026]] - [[kar-mathbuddy-affective-math-tutoring-2025]] - [[knowledge-gap-detection-ai-tas]] - [[llm-item-difficulty-prediction]] - [[multimodal-knowledge-graph-educational-reasoning]] - [[proprl-prerequisite-relation-learning]] - [[simulating-students-java-programming-errors-llms]] - [[skill-acquisition-without-temporal-info]] - [[xie-hillm-cd-2026]] - [[learnity-graphs-lifelong-learning-framework-2026]] - [[inside-llm-student-simulator-reasoning-2026]] --- ## [Teacher AI Competency](https://edtechdev.github.io/aied/concepts/teacher-ai-competency/) Teacher AI competency encompasses the knowledge, skills, and dispositions required for effective AI integration in educational contexts. Emerging frameworks identify three competency dimensions: ### 1. Technical Proficiency - **Prompt Engineering:** Crafting effective prompts for educational objectives (Xiao et al. 2026: 4-hour training → 2.3x integration increase) - **Tool Evaluation:** Assessing AI tools for pedagogical fit and safety - **Troubleshooting:** Addressing AI-system failures in real-time classroom contexts ### 2. Pedagogical Integration - **Curriculum Alignment:** Mapping AI use to learning objectives - **Scaffolding Design:** Using AI to support student metacognition and self-regulation - **Assessment Literacy:** Evaluating AI-generated content and student AI outputs ### 3. Ethical & Critical Use - **Bias Recognition:** Identifying cultural and representational gaps in AI outputs (Wang et al. 2025) - **Data Privacy:** Protecting student data in AI-enhanced learning environments - **Equity Advocacy:** Ensuring AI tools serve diverse learners ### The Competency Gap Zhang et al. (2026) documented a **40% gap** between self-reported and performance-based AI competency. Teachers confident in AI skills often lack foundational prompting and evaluation abilities—highlighting the need for performance-based assessment. ## References Zhang, S., Xiao, R., et al. (2026). *How to Assess AI Literacy: Misalignment Between Self-Reported and Performance*. arXiv:2601.06101. Xiao, R., Ye, R., et al. (2026). *Transforming GenAI Policy to Prompting Instruction: An RCT*. arXiv:2602.16033. Wang, J., Xiao, R., et al. (2025). *LLMs to Support K-12 Teachers in Culturally Relevant Pedagogy*. arXiv:2505.08083. ## Source - https://arxiv.org/abs/2601.06101 - [https://arxiv.org/abs/2602.16033](https://arxiv.org/abs/2602.16033) ## Connected Concepts - [[faculty-development]] - [[ai-literacy]] - [[academic-integrity]] - [[equity]] - [[assessment-validity]] - [[equity-in-ai-education]] - [[agentic-ai]] - [[over-reliance]] - [[instructional-design]] - [[human-in-the-loop-ai]] - [[student-experience]] - [[regulation]] ## Connected Articles - [[ai-acceptance-preservice-science-teachers-2026]] - [[ai-education-global-capacity]] - [[ai-tpack-teacher-multi-agent-workflow]] - [[civic-education-ai-lesson-plans]] - [[cost-of-ethics-crisis-cs-ethics-education]] - [[post-covid-ict-career-aspirations]] - [[sangwa-epiq-ai-faculty-readiness-2026]] - [[universities-ai-era-rethinking]] - [[white-wu-robotics-ai-education-2026]] - [[genai-pd-ai-pck-learning-gain-2026]] --- ## [Teacher Role in AI-Enhanced Education](https://edtechdev.github.io/aied/concepts/teacher-role/) > **Teacher role** — how AI reshapes the work, identity, and agency of educators. With 50+ articles examining this dimension, the wiki documents a fundamental transformation: from sole knowledge authority to orchestrator of human-AI learning environments. ### How AI transforms teaching - **From instructor to orchestrator:** [[teacher-ai-teaming-five-levels|Five levels of teacher-AI teaming]] and [[teacher-student-agency-orchestration|agency orchestration research]] map the spectrum from AI as tool to AI as teaching partner. - **Workflow transformation:** [[ai-changing-teaching-workflows]] documents how AI shifts teacher time from content delivery to higher-value activities like individual support and curriculum design. - **Competency demands:** [[teacher-ai-competency|Teacher AI competency frameworks]] define what educators need to know. [[teacher-ai-adoption-confidence|Adoption studies]] identify barriers: confidence, institutional support, and workload concerns. - **Co-design and agency:** [[teacher-authored-prompts-student-ai-dialogue|Teacher-authored prompts]] and [[gaide-vibe-coding-k12-teachers|vibe coding for teachers]] show educators as active AI designers, not passive consumers. - **Preservice preparation:** [[ai-tpack-preservice-math-teachers|TPACK-based training]] and [[faculty-development]] programs prepare future teachers for AI-augmented classrooms. ### The orchestration metaphor The dominant metaphor in the wiki is *orchestration*: teachers coordinate human learners, AI tutors, and curriculum resources. This contrasts with replacement narratives — AI augments rather than substitutes for human teaching. ### Connections Teacher role connects to [[faculty-development]] (how teachers are prepared), [[ai-literacy]] (teacher AI competency), [[k-12]] and [[higher-ed]] (context-specific implications), and [[scaffolding]] (how teachers scaffold AI use for students). ## Connected Concepts - [[faculty-development]] - [[teacher-ai-competency]] - [[ai-literacy]] - [[k-12]] - [[higher-ed]] - [[scaffolding]] - [[instructional-design]] - [[ai-tutoring]] - [[llm]] - [[professional-training]] ## Connected Articles - [[ai-acceptance-preservice-science-teachers-2026]] - [[ai-interior-design-malaysia-2026]] - [[critical-media-literacy-education-2026]] - [[teacher-ai-teaming-five-levels]] - [[teacher-student-agency-orchestration]] - [[ai-changing-teaching-workflows]] - [[teacher-ai-adoption-confidence]] - [[gaide-vibe-coding-k12-teachers]] - [[teacher-authored-prompts-student-ai-dialogue]] - [[ai-tpack-preservice-math-teachers]] --- ## [Transfer of Learning](https://edtechdev.github.io/aied/concepts/transfer-of-learning/) > **Transfer of Learning** — the extent to which knowledge or skills acquired in one context (e.g., practice with an AI tool) persist and apply in a different context (e.g., independent performance without the tool). In AI in education, transfer is the central open question: whether performance gains students show *with* AI tools translate into durable learning they can demonstrate *without* them. Transfer of learning is a foundational concern in education research, and AI tools have made it urgent. The defining empirical pattern documented across AI in education studies is a **transfer paradox**: students using AI typically show immediate, measurable gains on tasks where AI is available, but those gains often fail to persist — or even reverse — when AI is removed and students must demonstrate understanding independently. This pattern implicates [[over-reliance]], [[cognitive-load-theory]], and [[metacognition]] as the mechanisms at work, and connects directly to debates about [[ai-tutoring]] design. ### The transfer paradox Students using AI typically show **immediate, measurable gains** on the tasks where AI is available. Yet when AI is removed: - Effects become **mixed or negative** - Gains often **fail to transfer** to unassessed settings - Students may become **dependent on the tool** at the expense of independent reasoning The evidence base, synthesized in the [[stanford-evidence-base-ai-k12-2026|Stanford Evidence Base on AI in K-12]] review, is consistent across domains: | Study | Context | Immediate Effect | Transfer Effect | Mechanism | |---|---|---|---|---| | Bastani et al. (2025) | High school math | Higher practice grades | **~17% worse** on closed-book finals | General-purpose chatbot did the work | | Chen et al. (2025) | Programming homework | Higher homework scores | **No improvement** on unassisted exams | LLM-Tutor solved problems for students | | Lehmann et al. (2025) | Programming | More topics covered | **Harmed understanding**; widened gaps | General AI for low-prior learners | | Stadler et al. (2024) | Academic research | Faster task completion | **Lower-quality reasoning** vs. search | Reduced cognitive engagement | | Kosmyna et al. (2025) | Essay writing | Higher essay quality | **83% failed to recall** their own quotes | Outsourced authorship | All five studies show a **negative or null transfer** pattern when general-purpose AI is the intervention. ### Mechanisms undermining transfer **Metacognitive displacement.** AI completing reasoning reduces opportunities for students to monitor their own understanding and select strategies. Students who used AI were less able to explain their answers when queried. This connects to [[metacognition]] research on self-monitoring and the [[vibe-compiler-metacognition-genai-agency-2026|evidence that structured courses increase metacognitive competence while raw LLM assistants do not]]. **Germane load suppression.** General-purpose AI reduces not just extraneous (distracting) cognitive load but also *germane* load — the productive mental effort that encodes durable knowledge. Easier practice feels better but stores weaker traces. See [[cognitive-load-theory]] and the distinction between [[tutoring-specific-vs-general-ai|tutoring-specific vs general AI]]. **Over-reliance / expertise reversal.** Novices given answers do not build schemas. General AI provides answers; effective tutoring provides structured guidance. When novices are given expert-level shortcuts, learning is disrupted — the [[desirable-difficulties]] principle in reverse. **Tool-dependent performance.** Students may optimize for the specific affordances of the AI tool (prompt engineering, reliance on generated code structure) rather than building domain generalization — a form of [[cognitive-offloading-speedup-illusion|cognitive offloading]] that feels productive but displaces durable learning. ### Conditions supporting positive transfer The limited evidence suggests transfer is possible when: - **Pedagogical guardrails are present** — step-by-step hints, misconception targeting, Socratic questioning (Bastani et al., 2025 tutoring variant) - **Traditional strategies are preserved** — note-taking paired with AI use improved retention (Kreijkes et al., 2026) - **AI is used for formative, not summative, practice** — scaffolding during learning, not during assessment - **Learner expertise is calibrated** — the tool adapts support to readiness rather than defaulting to full assistance This aligns with [[ai-tutoring]] research showing that tutoring-specific tools with pedagogical guardrails outperform general-purpose chatbots, and with [[scaffolding]] principles about fading support as competence grows. ### Unanswered questions 1. **Time scale:** Does transfer improve over weeks/months of use, or does dependence deepen? 2. **Domain differences:** Is transfer better in well-structured domains (math) vs. ill-structured domains (writing)? 3. **Individual differences:** Do high-prior-knowledge students suffer less transfer loss than novices? 4. **Skill remediation:** Can explicit "AI-off" practice sessions reverse tool dependence? ### Connections to related concepts Transfer of learning connects to [[metacognition]] (self-monitoring of understanding), [[cognitive-load-theory]] (germane vs extraneous load), [[desirable-difficulties]] (productive struggle), [[scaffolding]] (fading support), [[over-reliance]] (tool dependence), and [[zone-of-proximal-development]] (general-purpose AI operates outside the ZPD by completing work for students). It is the bridge between assisted performance and genuine learning — the distinction between [[tutoring-specific-vs-general-ai]] and the central question for [[ai-tutoring]] effectiveness. ## Connected Concepts - [[metacognition]] - [[cognitive-load-theory]] - [[desirable-difficulties]] - [[over-reliance]] - [[scaffolding]] - [[zone-of-proximal-development]] - [[ai-tutoring]] - [[k-12]] - [[self-regulated-learning]] ## Connected Articles - [[stanford-evidence-base-ai-k12-2026]] - [[tutoring-specific-vs-general-ai]] - [[educational-llm-alignment]] - [[cognitive-offloading-speedup-illusion]] - [[vibe-compiler-metacognition-genai-agency-2026]] - [[ai-tutor-safety-harms]] - [[brookings-ai-students-report]] - [[learnity-graphs-lifelong-learning-framework-2026]] --- ## [Trust Calibration](https://edtechdev.github.io/aied/concepts/trust-calibration/) > **Trust calibration** — the metacognitive capacity to align one's confidence in an AI system with its actual reliability in a given context, knowing when to trust and when to question its output. Trust calibration is the direct antidote to [[over-reliance]]: it is the skill of matching trust to evidence rather than to an AI's confident fluency. A language model's fluent, confident prose reads as trustworthy whether or not it is. Trust calibration is the counterweight to that illusion — the practice of evaluating AI output against its verifiability and the stakes of the task, rather than accepting it on the strength of its presentation. ### Why trust needs calibrating Uncalibrated trust takes two forms. **Over-trust** (accepting AI output without verification) produces the uncritical acceptance documented in [[over-reliance]] and [[cognitive-offloading]] research, and compounds the [[hallucination-risk]] of confident errors. **Under-trust** (avoiding AI entirely) forgoes legitimate benefits. Both stem from the same root: trust based on appearance rather than evidence. Research on [[student-misconceptions-ai]] shows students often default to over-trust because they assume an AI that "sounds right" is right. ### How calibration works - **Verification habits:** checking AI claims against primary sources and the "AI proposes, you verify" rule, rather than accepting plausible-sounding output. - **Context awareness:** recognizing that trustworthiness varies by task — a well-trodden topic the model has seen extensively is safer than an obscure, high-stakes, or fast-moving one. - **Stakes adjustment:** applying more scrutiny where errors are costly (submitted work, medical or legal claims) and less where they are benign. - **Metacognitive monitoring:** tracking when and why one over-trusts, which connects calibration to [[metacognition]] and [[self-regulated-learning]]. ### Connections Trust calibration is central to [[ai-literacy]] and sits alongside [[reducing-ai-misuse]] as a skill-based intervention: students misuse AI less when they can judge when its output deserves trust. It is also a design goal — [[pedagogical-safety]] and transparency tools aim to make AI's reliability legible so learners can calibrate more accurately. ## Connected Concepts - [[ai-literacy]] - [[over-reliance]] - [[cognitive-offloading]] - [[hallucination-risk]] - [[metacognition]] - [[self-regulated-learning]] - [[human-ai-collaboration]] - [[student-misconceptions-ai]] - [[reducing-ai-misuse]] - [[pedagogical-safety]] ## Connected Articles - [[trust-reliance-ai-education-2026]] — Trust and Reliance on AI in Education - [[ai-fallibility-warning-help-seeking]] — Warning About AI Fallibility Increases Help-Seeking - [[calibrating-trustworthiness-llm-education-2026]] — Calibrating Trustworthiness: Co-Designing Metrics for LLMs in Education - [[llm-fallacy-misattribution]] — The LLM Fallacy and Misattribution of Competence - [[ai-partner-science-epistemic-vigilance]] — Epistemic Vigilance as the Key to Productive Augmentation --- ## [Trust in AI](https://edtechdev.github.io/aied/concepts/trust/) > **Trust in AI** — the willingness of learners and educators to rely on AI systems for learning, judgment, and decision-making. Trust is a precondition for effective use of AI in education, but it is a double-edged sword: appropriate trust enables productive engagement, while over-trust leads to [[over-reliance]] and under-trust prevents beneficial use. Trust is shaped by the perceived competence, transparency, reliability, and benevolence of the AI system, and by contextual factors such as the task and the user's experience. It connects to [[trust-calibration]] (matching trust to actual reliability) and epistemic trust (trust in AI as a source of knowledge). Trust is central to how learners interact with AI — whether a tutoring chatbot, a [[social-robots|social robot]], or an automated feedback system. It is closely related to [[ai-literacy]] (understanding what to trust), [[critical-thinking]] (evaluating AI output), and the design of responsible AI. In embodied contexts like [[human-robot-interaction]], trust is shaped by robot behaviour, task, and appearance. ### How trust appears in the wiki's research - **Trust in educational robots:** [[task-context-trust-educational-hri-2026|Research on trust in educational HRI]] shows that task context shapes trust in social robots more than appearance, with the highest trust during instructional tasks — what a robot does matters more than what it looks like. - **Epistemic trust in AI-mediated learning:** [[face-value-how-avatar-identity-shapes-epistemic-trust-in-ai-mediated-learning|Research on avatar identity]] examines how the perceived source of AI content shapes epistemic trust. - **Calibration:** [[trust-calibration]] research examines how to help learners calibrate their trust to the actual reliability of AI systems, avoiding both over- and under-trust. - **Trust in tutoring and feedback:** Studies of [[ai-tutoring]] and AI feedback examine when learners appropriately trust (or question) AI-generated guidance and [[hallucination-risk|hallucinations]]. Trust connects to [[ai-literacy]], [[critical-thinking]], [[trust-calibration]], [[over-reliance]], [[face-value-how-avatar-identity-shapes-epistemic-trust-in-ai-mediated-learning|epistemic trust]], [[social-robots]], [[human-robot-interaction]], and [[ethics]]. Building appropriate trust is a core goal of responsible AI design in education. ## Connected Concepts - [[trust-calibration]] - [[ai-literacy]] - [[critical-thinking]] - [[over-reliance]] - [[social-robots]] - [[human-robot-interaction]] - [[ethics]] - [[ai-tutoring]] ## Connected Articles - [[task-context-trust-educational-hri-2026]] — Task Context and Trust in Educational HRI - [[face-value-how-avatar-identity-shapes-epistemic-trust-in-ai-mediated-learning]] — Avatar Identity and Epistemic Trust - [[white-wu-robotics-ai-education-2026]] — Robotics and AI in Education - [[human-autonomy-agency-hri-review-2025]] — Human Autonomy and Agency in HRI - [[social-robot-study-companions]] — Social Robots as Study Companions --- ## [Universal Design for Learning](https://edtechdev.github.io/aied/concepts/universal-design-for-learning/) > **Universal Design for Learning (UDL)** — an educational framework that designs instruction to be accessible and effective for the widest range of learners by proactively building in flexible means of engagement, representation, and action/expression, rather than retrofitting accommodations for individuals. UDL rests on the insight that learner variability is the norm, not the exception. Rather than designing a single path and adding support for those who struggle, UDL designs multiple pathways from the start so that barriers are removed for everyone. It is a core lens for [[accessible-learning]], [[equity]], and [[special-education]]. ### The three principles - **Multiple means of engagement** — the "why" of learning: varied ways to motivate and sustain interest, connect to relevance, and support self-regulation. - **Multiple means of representation** — the "what" of learning: presenting information in varied formats (text, audio, visual, interactive) so all learners can perceive and comprehend it. - **Multiple means of action and expression** — the "how" of learning: offering varied ways for learners to demonstrate what they know (writing, speaking, building, performing). ### UDL in the AI era Generative AI creates new opportunities and new risks for UDL. AI can personalize representation and provide alternative pathways, supporting [[personalized-learning]] and accessibility. But it can also encode bias, assume dominant communication styles, and — if it reduces learner agency — undermine the engagement principle. Research on [[ai-misuse-learning-harm]] and equity shows that AI tools must be designed with inclusive principles or they recapitulate [[equity]] gaps. UDL therefore informs both how AI is deployed and how AI-literacy and assessment are designed to be fair across learner variability. ### Connections UDL connects to [[accessible-learning]], [[equity]], [[special-education]], [[instructional-design]], and [[culturally-relevant-pedagogy]]. In assessment, it intersects with [[authentic-assessment]]'s emphasis on representational fairness and with [[reducing-ai-misuse]] as a guardrail against tools that penalize particular communication styles. ## Connected Concepts - [[accessible-learning]] - [[equity]] - [[special-education]] - [[instructional-design]] - [[personalized-learning]] - [[culturally-relevant-pedagogy]] - [[authentic-assessment]] - [[student-experience]] - [[generative-ai]] ## Connected Articles - [[authentic-products-authenticated-processes-2026]] — From Authentic Products to Authenticated Processes - [[tactile-statistical-graphs-accessibility]] — Tactile Statistical Graphs for Accessibility - [[neurodivergent-computing-students]] — Neurodivergent Computing Students - [[ai-learning-tools-engineering-education-needs]] — Designing Needs- and Attention-Aware AI Learning Tools --- ## [Well-Being](https://edtechdev.github.io/aied/concepts/well-being/) > **Well-being** — the positive state of being mentally, physically, and socially healthy, encompassing emotional, psychological, and social dimensions. In AI in education, well-being has become a central concern because the rapid integration of generative AI into learning environments can affect students' and educators' mental health, motivation, belonging, anxiety, and sense of agency — raising questions about whether AI supports or undermines learners' well-being. Well-being in education is multifaceted: it includes emotional well-being (positive affect, low distress), psychological well-being (purpose, autonomy, competence), and social well-being (belonging, positive relationships). In the AI era, well-being matters because AI can reshape learning in ways that affect these dimensions — from reducing students' confidence and increasing anxiety about academic integrity, to fostering or undermining engagement and belonging. Concerns about AI's impact on students' socio-emotional skills, well-being, sociability, and sense of trust and empathy (raised by the OECD and others) have positioned well-being as a key consideration in responsible AI integration. ### How well-being appears in the research - **AI literacy and social-emotional learning:** [[sec-ai-literacy-narrative-review-2026|Research integrating SEC into AI literacy]] argues that fostering educators' and students' emotional intelligence and well-being is essential for navigating AI-mediated learning environments, connecting to [[social-emotional-learning]] and affective dimensions of AI. - **AI anxiety and student experience:** Studies on students' engagement with AI (e.g., [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination|SDT-based research]]) find AI use is intertwined with anxiety, trust, and confidence, with students' well-being affected by concerns about academic integrity, creativity, and over-reliance. - **Teacher well-being and role:** AI's impact on [[teacher-role|teachers]] — including workload, anxiety about teaching with disruptive technology, and the capacity to provide emotional support — is a recurring concern, connecting to [[teacher-ai-competency]] and professional development. - **Ethics and responsible AI:** Well-being is a core ethical consideration in [[ai-education]], linking to [[ethics]] and the imperative to design AI that supports rather than harms learners' mental health and belonging. ### Well-being as a design consideration A recurring theme is that well-being should be a deliberate design consideration in AI in education, not an afterthought. This means: designing AI to support rather than replace human relationships; ensuring students can maintain agency and confidence rather than experiencing AI-induced anxiety or over-reliance; supporting educators' capacity and well-being as they integrate AI; and evaluating AI systems not only for learning outcomes but also for their effects on students' and teachers' well-being. Research connects well-being to [[motivation]], [[self-regulated-learning]], and [[student-experience]] (belonging and engagement). ### Connections to related concepts Well-being connects to [[student-experience]] (as a dimension of learners' overall experience), [[social-emotional-learning]] and [[affective-computing]] (the emotional competencies AI intersects with), [[ethics]] (as a core ethical consideration), [[motivation]] and [[self-regulated-learning]] (well-being supports and is supported by these), [[teacher-ai-competency]] (educators' capacity and well-being), and [[higher-ed]] and [[k-12]] as the settings where AI shapes well-being. ## Connected Concepts - [[student-experience]] - [[social-emotional-learning]] - [[affective-computing]] - [[ethics]] - [[motivation]] - [[self-regulated-learning]] - [[teacher-ai-competency]] - [[higher-ed]] ## Connected Articles - [[sec-ai-literacy-narrative-review-2026]] — Integrating Social-Emotional Competencies Into AI Literacy - [[students-engagement-with-generative-ai-in-academic-learning-a-self-determination]] — Students' Engagement With GenAI (SDT) - [[teacher-education-ai-literacy-sdt-2026]] — Teacher Education for AI Literacy (SDT) - [[genai-motivation-engagement-2026]] — Generative AI, Motivation, and Engagement - [[ai-chatbot-collective-efficacy-collaborative-learning]] — AI Chatbots, Collective Efficacy, and Collaboration --- ## [AI in Writing Education](https://edtechdev.github.io/aied/concepts/writing-education/) > **AI in writing education** — the use of AI tools for writing instruction, assessment, feedback, and the study of how generative AI reshapes the writing process itself. Writing education is one of the most AI-affected domains, because LLMs excel at the very activities writing instruction centers on — text generation, revision, and evaluation. Research in this area spans automated scoring, AI feedback quality, writing-process support, second-language writing, academic integrity, and the deeper question of how AI changes what it means to write and to be a writer. Writing is not merely output but a cognitive, social, and rhetorical process. This is why AI's impact on writing education is so consequential and contested: AI can be a scaffold that helps students draft, revise, and receive feedback they otherwise wouldn't get, but it can also displace the cognitive work — and the human audience — that make writing a learning activity. The wiki's research consistently frames AI in writing as a *human-centered complement* to, rather than a replacement for, the social and cognitive processes of writing. ### How AI in writing education appears in the research - **Automated essay scoring:** [[automated-essay-scoring]] systems like [[choi-anchor-aes-prompting-2025|anchor-based AES]] and [[aiawe-automated-writing-evaluation|AIAWE]] evaluate student writing at scale, raising questions about [[assessment-validity|construct validity]] and the reduction of writing to measurable features. - **Writing feedback:** [[ai-feedback-quality|AI feedback quality]] research ([[genai-teacher-feedback-comparison|GenAI vs. teacher feedback]], [[care-full-feedback-genai|care-full feedback]], [[repeated-ai-writing-feedback-semester|repeated AI feedback]]) examines whether AI feedback improves writing and how it compares to human feedback. The PAIRR model ([[pairr-ai-peer-review-2025|Peer and AI Review + Reflection]]) combines AI with [[peer-review]] and finds AI feedback is most useful in a human-centered process. - **Writing process support and agency:** [[agency-gap-ai-writing|Agency gap research]] and [[ai-writing-support-stage-ownership-2026|stage-ownership research]] explore how AI changes the writing process from planning to revision, and how students' [[student-experience|agency]] is affected when AI participates at different stages. - **Posthumanist perspectives:** [[posthumanist-ai-literacy-2025|A posthumanist approach to AI literacy]] reframes writing as a human-AI entanglement in which agency is distributed, challenging both uncritical anthropomorphization of AI and its dismissal as a mere tool — a relational rather than transactional view of AI literacy. - **L2 / multilingual writing:** [[self-referential-l2-writing-llm-assessment|L2 writing assessment]], [[genai-linguistic-diversity-academic-writing|linguistic diversity research]], and [[ai-writing-support-stage-ownership-2026|stage-ownership research]] address how AI supports (or constrains) second-language and multilingual writers, including the risk of reinforcing Standard Academic English norms. - **Academic integrity:** [[academic-integrity]] and [[student-rationalization-ai-writing|student rationalization studies]] examine how students justify AI use, moving the conversation from plagiarism policing toward building AI literacy and ethical use. ### Writing as thinking Because writing is a cognitive process, AI-in-writing research connects to [[cognitive-offloading]] (does AI writing support bypass thinking?), [[metacognition]] (does AI feedback improve self-assessment?), [[self-regulated-learning]] (do students regulate their use of AI feedback?), and [[ai-literacy]] (can students evaluate AI-generated writing critically?). The [[critical-thinking-genai-scaffolding|critical-thinking scaffolding]] and [[ai-feedback-critical-thinking-writing-2026|AI feedback for critical thinking]] research show that the pedagogical value of AI in writing depends on whether it prompts reflection and judgment rather than answer-replacement. ### Connections Writing education connects to [[automated-essay-scoring]], [[ai-feedback-quality]], [[academic-integrity]], [[cognitive-offloading]], [[ai-literacy]], [[language-learning]], [[formative-assessment]], [[peer-review]], [[metacognition]], [[self-regulated-learning]], and [[higher-ed]]. It is a domain where AI's capabilities and risks are both highly visible, making it a rich site for studying how AI transforms pedagogy, assessment, and the very nature of authorship and agency. ## Connected Concepts - [[automated-essay-scoring]] - [[ai-feedback-quality]] - [[academic-integrity]] - [[cognitive-offloading]] - [[ai-literacy]] - [[language-learning]] - [[higher-ed]] - [[metacognition]] - [[llm]] - [[generative-ai]] - [[formative-assessment]] - [[peer-review]] - [[self-regulated-learning]] - [[student-experience]] ## Connected Articles - [[pairr-ai-peer-review-2025]] — Peer and AI Review + Reflection (PAIRR) - [[posthumanist-ai-literacy-2025]] — A Posthumanist Approach to AI Literacy - [[choi-anchor-aes-prompting-2025]] — Anchor-Based Automated Essay Scoring - [[aiawe-automated-writing-evaluation]] — AIAWE: Automated Writing Evaluation - [[agency-gap-ai-writing]] — The Agency Gap in AI-Supported Writing - [[ai-writing-support-stage-ownership-2026]] — From Planning to Revision: AI Writing Support at Different Stages - [[genai-teacher-feedback-comparison]] — Comparing Generative AI and Teacher Feedback - [[student-rationalization-ai-writing]] — "It's OK Because...": The Wild West of Student Rationalization - [[care-full-feedback-genai]] — Care-Full Feedback Approaches - [[self-referential-l2-writing-llm-assessment]] — Self-Referential L2 Writing Assessment - [[ai-peer-feedback-systems]] — AI Peer Feedback Systems - [[repeated-ai-writing-feedback-semester]] — Student Evaluation of Repeated AI Feedback - [[elementary-writing-genai-systematic-review-2026]] — Rethinking Elementary Writing Instruction - [[veriforge-narrative-drafting-scaffolding-2026]] — VeriForge: Narrative Drafting Scaffolding - [[ai-feedback-critical-thinking-writing-2026]] — Using AI-Generated Feedback to Improve Critical Thinking --- ## [Zone Of Proximal Development](https://edtechdev.github.io/aied/concepts/zone-of-proximal-development/) > **Zone Of Proximal Development** is a central concept in AI in education research, connected to 8 articles in this wiki. ## Connected Concepts - [[metacognition]] - [[scaffolding]] - [[cognitive-load-theory]] - [[desirable-difficulties]] - [[adaptive-learning]] - [[human-in-the-loop-ai]] - [[rct]] - [[ai-tutoring]] - [[k-12-ai-education]] ## Connected Articles - [[collaborative-ai-tutoring]] - [[finkelstein-principled-ai-education-2025]] - [[stanford-evidence-base-ai-k12-2026]] - [[text-simplification-its]] - [[tutoring-specific-vs-general-ai]] ---