🏷️ automated-grading
71 pages tagged with automated-grading(61 articles, 10 concepts)
📄 Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency
> **Synthesis:** This study developed the Writing Improvement and Smart Evaluation Agent (WISE Agent), an AI feedback tool targeting textual logic and perspective biases in student essays. A three-mon…
🏷️ AI Feedback Quality
> **AI feedback quality** — the accuracy, usefulness, timeliness, and pedagogical value of feedback generated by AI systems for learners. As AI-generated feedback becomes ubiquitous in education, unde…
🏷️ Assessment Validity in AI Education
> **Assessment validity** — whether assessments measure what they claim to measure. AI in education raises fundamental validity questions: do AI-graded assessments assess student learning or AI prompt…
🏷️ Automated Assessment
> **Automated assessment** — the use of AI to evaluate student work, from formative quizzes to high-stakes exams. Automated assessment spans multiple modalities — multiple-choice, short answer, essay,…
🏷️ Automated Essay Scoring
> **Automated Essay Scoring (AES)** — the use of AI to evaluate and score written essays, spanning traditional statistical approaches, fine-tuned language models, and increasingly accessible LLM-based…
🏷️ CS Education and AI
> **CS Education** — computer science education is the most-researched STEM subfield in the wiki, benefiting from natural alignment between AI tools and programming tasks. Code generation, debugging a…
🏷️ Feedback Loop
> **Feedback loop** — the cyclical process where AI systems assess student work, deliver feedback, observe the student's response, and adapt subsequent instruction. Effective feedback loops close the …
2026-08-09 · formative-assessment, ai-feedback-quality, scaffolding, self-regulated-learning, ai-tutoring
🏷️ Large Language Models (LLMs)
> **Large Language Models (LLMs)** — neural network models trained on vast text corpora that generate human-like text, powering most modern AI in education applications. LLMs are the computational bac…
🏷️ STEM Education and AI
> **STEM Education** — science, technology, engineering, and mathematics education is the most common domain for AI in education research in the wiki. STEM's structured knowledge, clear right/wrong an…
📄 Using LLMs to Detect Growth in Computational Thinking in Introductory Physics
> **Synthesis:** Savage, Shanker, Michlitsch & Rebello (2026) investigate using LLMs to evaluate students' written explanations of computational physics problems at scale. Establishing a human-coded b…
📄 What out-of-the-box LLMs can(t) do in law? A Turing test in Italian exams for lawyers, judges and notaries
> **Synthesis:** This paper reports a blind Turing Test evaluating leading LLMs on three Italian professional legal examinations: the Bar exam, Judges exam, and Notary exam. LLMs generated full writte…
📄 Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module
> **Synthesis:** An LLM-based interactive module teaches K-12 students prompting literacy through scenario-based deliberate practice with an AI auto-grader providing immediate, detailed feedback. Depl…
📄 Comparative Validation of GPT-4o-mini and Teacher Mean Scores for Automated Scoring of Music Analysis Responses: Single-Pass Deployment, Repeatability, and Strategy-Specific Bias
> **GPT-4o-mini can produce stable rubric-based scores for open-ended music analysis responses, with few-shot chain-of-thought prompting agreeing most strongly with teacher means while RAG systematica…
📄 CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedback
> 1. **Evidence-Centered Design (ECD)** — assessments and rubrics aligned to curriculum goals from the start 2. **Human-in-the-loop prompt engineering** — labelled examples and prompts refined iterati…
2026-08-03 · formative-assessment, human-in-the-loop, prompt-engineering, benchmark, ai-ed-evaluation
📄 Responsible Assessment in the AI Era: Key Insights from a Future-Focused Conference
> **Responsible assessment in the AI era** — assessment grounded in learners' sociocultural contexts and designed to generate valid, trustworthy, context-specific inferences from accumulated evidence,…
📄 The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations
> **Ilya Mikhelson** — Submitted to Computers and Education: Artificial Intelligence (2026).…
📄 Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education
Presents **StructRAG**, a pattern-aware framework that improves how AI tutoring systems interpret **complex engineering diagrams** (circuit schematics, network topologies, block flowcharts) in STEM. C…
📄 Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis
> **Authors:** Kamila Misiejuk, Sonsoles López-Pernas, Eduardo Araujo Oliveira, Brendan Eagan, Mohammed Saqr **Source:** Computers and Education: AI, Vol 11 — Open Access (CC BY 4.0)…
2026-08-01 · educational-theory, ground-truth-reliability-aied, learning-analytics, llm, llm-in-education
📄 AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics
> **Authors:** Markus S. Feser, Paul L. Tschisgale (Leibniz Institute for Science and Mathematics Education, Kiel, Germany)…
📄 ICLE++: Modeling Fine-Grained Traits for Holistic Essay Scoring
Introduces ICLE++, a new annotated corpus of persuasive student essays that addresses critical limitations of the dominant ASAP benchmark in [[automated-essay-scoring]] research. Unlike ASAP — used by…
2026-07-31 · automated-essay-scoring, benchmark, educational-measurement, formative-assessment, higher-ed
📄 Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models
Pre-registered study auditing whether general-purpose helpfulness rubrics can distinguish direct answer-giving from pedagogical guidance in LLM tutors. Uses deterministic detectors for answer leakage …
📄 Confidence-Aware Automatic Short Answer Grading
> **Confidence-Aware ASAG** — A hybrid confidence estimation framework for Automatic Short Answer Grading with LLMs that fuses model-based confidence signals (verbalized, latent, consistency-based) wi…
📄 The Effect of High-Frequency, Automatically-marked Formative Assessments on Student Outcomes in A-Level Sciences
This quasi-experimental mixed-methods longitudinal study (N=142) deploys a fully automated marking pipeline for handwritten mock examinations in A-Level sciences, removing the human-marking bottleneck…
🏷️ Open Source
> **Open-source** AI in education is studied in [[lata-ferpa-compliant-local-llm-autograder]], [[vismatic-secure-sandbox-cs-education]], and [[open-source]] (tag) pages: local open models address [[pr…
📄 Knowledge Distillation for Automated AI Tutor Evaluation
Addresses the lag between LLM integration into K-12/higher education and reliable methods for evaluating pedagogical quality. The authors introduce a knowledge-distillation approach to automate AI-tut…
📄 A Durability and Cross-Language Transfer Benchmark for a Validated Teaching-Feedback Classification Protocol
> Extends a prior validated protocol for classifying open-ended teaching-evaluation feedback by thematic category and sentiment, introducing a durability and cross-language transfer benchmark. Institu…
2026-07-14 · feedback-loop, teacher-role, formative-assessment, higher-ed, faculty-development-genai
📄 Automated Grading of Linux/Bash Examinations Using Large Language Models
**Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez, Lorena Otero-Cerdeira (2026)** This paper presents an [[llm]]-based grading system for Linux/bash com…
📄 Constructing Epistemic AI Literacy: Detecting Epistemic Aims and Processes in Student-AI Co-Programming
**Mengqian Wu (2026)** Epistemic thinking — understanding how knowledge is constructed and justified — plays a central role in [[ai-literacy]], particularly when students co-program with generative AI…
📄 Data Comics for Education: Evaluating Effectiveness, Benefits, and the Ethics of AI-Assisted Creation
Data comics combine sequential visual narratives with data visualization to improve student engagement with [[generative-ai]] in educational settings. This paper evaluates the effectiveness of AI-assi…
📄 Evaluating Interactivity: Toward Automated Assessment of AI-Generated Explorable Explanations
While [[llm]]s now enable rapid generation of learning materials like [[generative-ai]], evaluating the pedagogical quality of these materials remains an open challenge. This paper proposes an automat…
📄 From Answer Generators to Reasoning Facilitators: Designing AI Tutors for Mathematical Reasoning in High-Stakes Environments
The rapid integration of [[llm]]s into [[intelligent-tutoring]] threatens to reduce mathematical learning to mere answer generation. This paper presents a design framework for AI tutors that act as re…
📄 Mind the Trust Gap: Identifying (Mis)alignments in Teacher-Student Views Toward Control and Agency in K-12 Classroom AI
**Tomohiro Nagashima, Lisa Siegrist, Niklas Scholz, Shintaro Sato, Martina Vincoli, Man Su (2026)** As AI technologies enter [[k-12]] classrooms, understanding how different stakeholders perceive thes…
📄 The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions
Imran and Bulathwela (2026) identify the 'correct answer trap' — automated feedback systems that judge only answer correctness reinforce rather than address misconceptions when students reach the righ…
📄 Why Machines Misread Pedagogical Quality: Human-Machine Alignment in LLM-Based Pretest Question Evaluation
Tseng et al. (2026) investigate human-machine alignment in LLM-based pretest question evaluation — a critical bottleneck for scalable AI-assisted assessment. Their AI-assisted workflow combines automa…
📄 Test-Driven, AI-Assisted Learning: Replacing Lectures with Weekly Closed-Book Tests
Liu et al. (2026) report on a 13-week Test-Driven, AI-Assisted (TDAA) redesign of a Theory of Computation course at HKUST (Guangzhou). The course replaced all lectures with self-directed, AI-assisted …
📄 Confidence-Aware Automated Assessment of Student-Drawn Scientific Models
> **Luyang Fang, Yingchuan Zhang, Jongchan Park, Zhaoji Wang, Ping Ma, Xiaoming Zhai** (2026). arXiv cs.AI preprint…
2026-06-19 · stem-education, formative-assessment, k-12, efficacy-study, multi-representational-tools
📄 PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback
> **Wei Xia, Jin Wu, Haoran Shi, Xiangyu Wang, Chanjin Zheng** (2026). East China Normal University / arXiv cs.CL preprint…
📄 AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice
AI-driven assessment of human tutor training performance correlates with real-life tutoring quality; bridges the gap between training metrics and classroom practice. AI-Driven Assessment of Human Tuto…
📄 From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions
LLM-generated educational questions show varying cognitive depth; models excel at factual recall but struggle with higher-order thinking questions per Bloom's taxonomy. From Memorization to Creation: …
📄 Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs
Evaluates cross-dataset generalization of ML/DL methods and LLMs for automatic Bloom's taxonomy classification of assessment questions across five datasets. Supervised ML/DL models degraded substantia…
📄 Are LLM-based Chatbots Good Enough to Support Computer Science Students in Multiple-Choice Exercises?
Investigates LLM chatbots' performance on 70 MCQs for a university CS lecture on interactive visual data analysis, comparing with student performance. GPT-4o and GPT-5 significantly outperformed small…
📄 AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models
Gayed presents **AiAWE**, an open-source [[automated-grading|automated writing evaluation]] (AWE) system that scores argumentative essays using a LoRA-adapted instruction-tuned [[llm|large language mo…
📄 Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations
**Hartwig Grabowski, Michael Canz** — cs.AI, cs.CV, cs.CY This paper identifies the didactic narrowing caused by fully digital e-assessment (overuse of closed question formats) and proposes a hybrid a…
📄 LLM-Generated Feedback in Introductory Programming: A Classroom Study
Presents a **large-scale classroom study** (N=215 students, 6,693 submissions across 17 labs) deploying AI-generated feedback through a randomized protocol in an introductory Python programming course…
📄 AI Assistance for Discretionary Work: Increasing Feedback Provision in Higher Education
This field experiment shows that AI-generated feedback drafts can measurably increase the rate and length of feedback that teaching assistants actually deliver to students, without sacrificing perceiv…
📄 Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation
This large-scale K-12 deployment provides empirical evidence that teacher-authored prompts can reliably shape the cognitive quality of student-AI dialogue at classroom scale. The TASD system lets teac…
📄 The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals
> **Authors:** Shim Jaechang, Unggi Lee (2026) — CIKM 2026…
📄 Agentic Literacy Debt: A Structural Problem the AI Literacy Field Has Not Yet Named
**Agentic Literacy Debt** names a critical gap in the [[ai-literacy]] landscape that has become urgent with the rise of autonomous AI agents. Existing AI literacy frameworks assume humans evaluate AI …
📄 Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams
**AI-Assisted Writing Transforms Research Teams** challenges the longstanding "Big Science" trend toward ever-larger teams, showing that AI writing tools enable smaller, younger research teams to prod…
📄 Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning
**Catching the Correct Answer Trap** — accepted at AIED 2026 — exposes a critical blind spot in [[intelligent-tutoring]] systems: they systematically fail to detect misconceptions when students arrive…
📄 Mathematical Modelling of Ethical AI Use in Higher Education: A Coordination Game Framework for Future-Facing Learning
**Ethical AI Use in Higher Education: A Coordination Game Framework** provides a formal mechanism-level account of why policy statements alone fail to change student AI-use behavior. Reframing student…
📄 KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing
**KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing** bridges two key AI-in-education paradigms: [[personalized-learning]] through question generation and [[learning-analytics]] …
2026-05-28 · adaptive-learning, intelligent-tutoring, learning-analytics, llm, personalized-learning
📄 LLM-assisted sentiment analysis for integrated computational and qualitative mixed methods education research: A case study of students' written reflection assignments
**LLM-Assisted Sentiment Analysis for Mixed-Methods Education Research** demonstrates how LLMs can serve as scalable qualitative research assistants, enabling researchers to investigate multiple demog…
📄 Learning after COVID-19 and the ICT career aspirations: Are students entering the AI era with weaker skills?
**Post-COVID ICT Career Aspirations** uses PISA 2018 and 2022 country-level data to investigate whether students entering the generative AI era have adequate educational foundations. Using a mixed-met…
📄 REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading
**REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models** advances the [[automated-grading]] frontier by solving a fundamental trust problem: even accurate AI graders are unusable if educat…
📄 Generative AI as a Design Variable: An Evidence-Centered Framework for Principled Governance in STEM Assessment
This paper proposes a principled framework grounded in Evidence-Centered Design (ECD) that treats [[generative-ai]] as a design variable within STEM assessment arguments rather than an external threat…
📄 Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education
This paper presents a rigorous empirical comparison between [[llm|LLM]]-based and semantic similarity methods for [[automated-grading|automated assessment]] of student self-explanations in programming…
📄 Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs
Automated grading systems have enabled scalable assessment for many response types, but handwritten mathematics remains a barrier due to the complexity of multi-step solutions. Vision-capable large la…
📄 Codify: An Intelligent Socratic Tutoring System for Programming Education
Codify (also referred to as "AI Tutor") is a web-based [[intelligent-tutoring]] platform for programming education that integrates conversational AI, adaptive assessment, and learning analytics. It le…
📄 LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework
> LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework **Rodríguez (2026)** — Oregon State University. Submitted to Computers & Education.…
📄 AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education
AICoFe orchestrates a multi-LLM pipeline using GPT-4.1-mini, Gemini 2.5 Flash, and Llama 3.1 to synthesize quantitative rubric data and qualitative observations into actionable feedback for higher edu…
📄 Ensuring Reliability in Programming Knowledge Tracing: A Re-evaluation of Attention-augmented Models and Experimental Protocols
This ITS 2026 paper challenges claims about attention-augmented Programming Knowledge Tracing (PKT) superiority. The authors identify three critical protocol flaws: **attention dimension misconfigurat…
📄 AcademiClaw: When Students Set Challenges for AI Agents
> **Yu, Lu, Si et al. (77 authors, 2026)** — Shanghai Jiao Tong University, SII, GAIR. Open-source benchmark.…
📄 Modernizing Ground Truth: Four Shifts Toward Improving Reliability and Validity in AI in Education
> Stop treating κ > 0.8 as a binary stamp of approval.…
📄 LLM-based Multimodal AI Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback
**AI multimodal feedback matches educator feedback for learning while significantly outperforming it on student perceptions.** The authors built a real-time AI-facilitated multimodal feedback system i…
📄 Assessing the Impact and Underlying Pathways of Sequenced AI Feedback on Student Learning
**Sequenced AI feedback harms learning despite boosting engagement and positive perceptions.** In a randomized experiment with 199 participants, the authors compared two types of AI-generated feedback…
📄 AISSA: AI-based Student Slides Analysis Tool for Academic Presentations
> A web-based system that uses LLMs and Learning Analytics dashboards to provide automated, rubric-based feedback on student presentation slides. Developed by Becerra et al. (2026), AISSA addresses th…
📄 Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation
> Schleifer, Ariely & Klebanov (2026) investigate a critical gap in [[automated-grading]]: **how scoring quality degrades for mid-range student responses**. Most ASAS evaluations focus on clearly corr…
🏷️ Automated Question Generation
Automated question generation leverages NLP and LLMs to create educational assessments at scale. Wei & Stamper (2025) introduced the **generate-then-validate** paradigm, reducing hallucination by 62% …
📄 Automatic Short Answer Grading with LLMs
> Automatic Short Answer Grading (ASAG) is never perfect. Upper bounds on accuracy arise from: > Zero-shot LLMs perform strongly on ASAG without task-specific fine-tuning, but **model-based confidence…
📄 Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs
> Bannò, Knill & Gales (2026) propose a paradigm shift in automated essay scoring: from **inter-learner ranking** to **intra-learner profiling**. Instead of asking "how does this essay rank against ot…