Architecting an AI-Driven Decision Support System for Enhanced Online Learning and Assessment

Synthesis: This technical review synthesizes 2020–2025 research on AI-based decision support systems (AI-DSSs) for online learning and assessment, integrating machine learning, NLP, knowledge-based systems, and deep learning to enable predictive analytics, automated grading, and personalized learning paths. The authors propose a modular four-component architecture (data collection, AI processing, decision engine, user interface) integrated with LMSs via LTI, and report concrete gains including up to 70% faster grading and 12–20% grade increases. It critically assesses technical, ethical, and implementation barriers—interpretability, bias, privacy, cost, and adoption resistance—alongside mitigation strategies and future directions toward generative AI and multimodal integration.

Key Findings

A proposed modular AI-DSS architecture (data collection, AI processing, decision engine, user interface) integrates with existing LMSs via Learning Tools Interoperability (LTI), featuring federated learning for Privacy and a hybrid RL–ML decision engine for dynamic path optimization.
Concrete performance gains reported across the literature: automated NLP essay grading achieves up to 0.85–0.90 correlation with human graders and reduces grading time by up to 70%; adaptive reinforcement learning systems improve student retention and performance by roughly 12–20%.
AI techniques show complementary strengths: supervised ML predicts outcomes with 85–90% accuracy, deep learning reaches up to 92% precision on multimodal assessments, and NLP chatbots answer queries with ~90% accuracy.
Major limitations persist: LLMs hallucinate in 15–25% of responses, reinforcement learning suffers sample inefficiency and reward-design instability (20–30% outcome variability), and deep learning incurs high computational costs (raising costs 40–50% in resource-limited settings).
Evaluation relies on a balanced mix of quantitative metrics (accuracy, precision, recall, response time, scalability) and qualitative frameworks (user satisfaction, engagement), with benchmarking against non-AI LMS baselines showing NLP grading up to 80% faster and ML recommendations improving retention 15–20%.

Study Design & Method

This is a technical review (not a meta-analysis) following PRISMA guidelines for rigor and transparency. A systematic literature search covered IEEE Xplore, Scopus, and Web of Science using keywords including "AI-based decision support," "online learning," "automated assessment," and "adaptive learning." The search targeted peer-reviewed journal articles, conference papers, and technical reports published between January 2020 and July 2025. Inclusion prioritized studies with empirical evaluations, novel AI-DSS frameworks, or real-world implementations; exclusion removed non-peer-reviewed sources and studies lacking technical depth. The review synthesizes findings through case studies of prominent platforms (e.g., a MOOC using NLP, an adaptive learning system using reinforcement learning) and comparative analyses of ~29 summarized studies (Tables 5–6), and proposes an implementation/validation roadmap (prototype → controlled pilot → A/B trials → multi-institution deployment).

Implications for AI in Education

The paper positions AI-DSS as a cornerstone of modern educational technology that can address scalability, personalization, and assessment fairness simultaneously. For practitioners, it offers a concrete architecture and validation roadmap, emphasizing that AI systems must be co-designed with stakeholders and grounded in user-centric principles (accessibility, multilingual support, bias-aware algorithms). For researchers, it underscores persistent gaps—model interpretability, algorithmic bias, data privacy (GDPR/FERPA compliance), and cost—and points to future work in generative AI, multimodal integration, and cross-cultural studies for global accessibility. The review cautions that ethical safeguards, human oversight, and regular fairness audits are essential if AI is to deliver equitable rather than amplified outcomes in education.

Connected Concepts

Connected Articles

Citation

Mahamad, S., Chin, Y.H., Zulmuksah, N.I.N., Haque, M.M., Shaheen, M., & Nisar, K. (2025). Architecting an AI-Driven Decision Support System for Enhanced Online Learning and Assessment. Future Internet, 17(9), 383.