🧠 AI Ed Wiki

📄 Research Article

LLM-Generated Feedback in Introductory Programming: A Classroom Study

● high · 2026-06-09llm, formative-assessment, feedback-loop, stem-education, higher-ed, automated-grading, rag

Presents a large-scale classroom study (N=215 students, 6,693 submissions across 17 labs) deploying AI-generated feedback through a randomized protocol in an introductory Python programming course. Students received one of three conditions: natural language hints, AI-generated failing test cases, or no AI feedback (control). The resulting dataset, ProgFeed, captures fine-grained temporal learning trajectories.

Key findings: Natural language feedback is significantly associated with higher completion rates and faster convergence to correct solutions. Test case feedback shows heterogeneous effects that depend critically on feedback validity. The form of AI-generated feedback matters — evaluating feedback quality, not just its presence, is essential for understanding pedagogical impact.

This study provides one of the largest empirical validations of LLM-based automated feedback in authentic programming classrooms, with direct implications for automated grading systems and formative assessment design in CS education.

Connected Concepts

  • AI Feedback Quality
  • Feedback Loop
  • Automated Grading
  • Formative Assessment
  • STEM Education

    • Connected Articles

  • Lata Ferpa Compliant Local LLM Autograder — LaTA: A Drop-in, FERPA-Compliant Local-LLM Autograder for Upper-Division STEM Coursework
  • Learning Engagement Assistant Lea — Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Evaluation of an Agentic AI Tutoring System
  • AI Generated Feedback Higher Ed — Artificial intelligence and feedback in university education: effectiveness and student perceptions
  • LLM Misconception Difficulty Easy Trap — The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty
  • Hybrid E Assessment Semi Automated Grading — Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations
  • LLM Automated Assessment Student Self Explanations — Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education

    • Citation

    Heickal, H., & Lan, A. (2026). A Classroom Study of LLM-Generated Feedback Intervention in Introductory Programming. arXiv:2606.08807. Accepted at IRAISE 2026.