A Machine Learning Approach for Predicting Student Progress in Online Programming Education

Synthesis: A Machine Learning Approach for Predicting Student Progress in Online Programming Education. Zhang, Jeffries and Koprinska propose an Machine Learning approach that predicts student progress at the module level in large-scale online programming courses: an intrinsically interpretable decision-tree model predicts a student's performance on the final problem of a module from content-interaction features derived from their log data. Across four large-scale K-12 programming courses (~35,000 students), the glass-box decision trees delivered competitive accuracy (85–91%) against black-box models such as random forests and SVMs, while producing interpretable tree-based visualizations and slide rankings that educators found useful for intervening with at-risk learners and improving course design.

Key Findings

• Machine Learning classification on course content-interaction features yields accurate module-level predictions of student outcomes, with decision trees reaching 85.2–90.9% accuracy, precision 0.88–0.92, recall 0.78–0.98 and F1 0.83–0.92 across all four courses, substantially exceeding the majority-class baseline.
• Intrinsically interpretable decision trees performed competitively with more advanced black-box models (logistic regression, random forest, SVM), and feature selection (CFS and information-gain ratio) cut tree complexity to just 3–5 leaf nodes without degrading accuracy — supporting glass-box models over post-hoc explainers for online course settings.
• Completion of prior exercises — predominantly problem slides (75% of features selected) — was the strongest predictor of module completion, consistent with the "doer effect"; interactive content slides mattered more in school-level courses where novices lack background knowledge.
• The method flags "No submission" (dropout) outcomes accurately and offers a 7–8 day prediction window before a module's deadline, giving educators a real-time opportunity to intervene with struggling and disengaged, at-risk students.
• Two experienced educators judged the trees and slide rankings genuinely useful for identifying where students struggle and give up, spotting slides with low pedagogical value, and surfacing behavioral differences between courses (e.g., Blockly vs. Python; beginner vs. intermediate).
• Exploratory analysis distinguished three engagement profiles — disengaged at-risk, disengaged-but-successful, and engaged high-performers — revealing that low content-slide completion can signal either positive or negative outcomes depending on student background and course level.

What this means for practice

• Instructors. Act inside the 7–8 day window before a module deadline: read the tree output for the end-of-module problem while students can still submit, and intervene with those flagged as "No submission" rather than waiting for the module to close.
• Instructors. Use the slide rankings as a course-design instrument, not only an early-warning feed: they surface slides where students stall before failing tasks and slides with little pedagogical value that can be revised or cut.
• Instructors. Compare modules across course versions and levels when students stumble, since the same low content-slide completion signals different outcomes for novices in Blockly and for intermediate Python students.
• Designers. Ship the tree alongside the prediction: 3–5 leaf decision trees matched random forest and SVM accuracy while letting educators see which content completions drive the model's judgment.
• Designers. Keep the feature set to content interactions that platforms already log — problem slides accounted for 75% of selected features — so the approach stays portable across programming courses.

Limitations

• The prediction target is performance on a module's last problem (pass, fail, or no submission), and the features come only from log-data content interactions, so the model captures progress on one task rather than learning.
• The "Failed" class is barely represented — failed submissions account for at most 4.3% of submissions for any given module — and the authors state this is why predictions for that class degrade (Intermediate Python precision 0.11, recall 0.00).
• Data come from four courses run by one provider over a five-week period in 2018, with roughly 35,000 students enrolled but only 20–50% active per course (for example, Novice Blockly averaged 1,952 active students out of 9,237), so the usable sample is far smaller than enrollment suggests.
• Usefulness was judged qualitatively by two experienced educators reading the trees and slide rankings; no classroom intervention or measured outcome improvement was tested.

Citation

Zhang, V., Jeffries, B., & Koprinska, I. (2025). A Machine Learning Approach for Predicting Student Progress in Online Programming Education. International Journal of Artificial Intelligence in Education, 35, 3614–3644.