An Exploratory Machine Learning Approach to Understanding Determinants of Future ChatGPT Use in Higher Education

Synthesis: This study uses an exploratory machine learning (ML) framework to identify which educational, perceptual, and demographic factors are associated with university students' intended future use of ChatGPT for academic work. Survey data were collected from 166 students (final analytical sample of 111 after conservative outlier removal), and the analysis deliberately prioritizes educational interpretation over raw algorithmic performance. The authors position their contribution not in new algorithms but in a pedagogically grounded operationalization of established adoption theories.

The work integrates four complementary theoretical lenses: the Technology–Organization–Environment (TOE) framework as the primary contextual structure; the Technology Acceptance Model (TAM) and the Unified Theory of Acceptance and Use of Technology (UTAUT) to operationalize individual perception constructs such as perceived usefulness, ease of use, and facilitating conditions; and Self-Determination Theory (SDT) with connectivist principles to interpret learner Motivation, engagement, and networked knowledge practices. Survey items were specified a priori and mapped to these theories before analysis.

Methodologically, the study applies exploratory factor analysis (EFA; KMO = 0.68, Bartlett's χ² = 5291.6, p < 0.05, average communality 0.57) to examine latent structure, then Principal Component Analysis (PCA) to derive 22 latent components explaining 57.1% of cumulative variance for use as model inputs. Five classifiers — Logistic Regression (LR), Support Vector Classifier (SVC), Random Forest (RF), CatBoost, and XGB — were compared under stratified 10-fold cross-validation with SMOTENC for class imbalance. SHAP (Shapley Additive Explanations), partial dependence plots, and interaction analyses were used to interpret model outputs in educational terms, and calibration was assessed via Brier score and Expected Calibration Error (ECE).

Key Findings

• Pedagogical factors dominate adoption. Across both statistical group comparisons and SHAP-based explanations, Learning Support & Accessibility and Academic Assistance & Engagement were the most influential predictors of future ChatGPT use, while usability and performance perceptions showed comparatively weaker and more context-dependent effects.
• Significant perceptual group differences. Future users and non-users differed significantly on Learning Support & Accessibility (U = 1225, p < 0.001) and Academic Assistance & Engagement (t = 3.28, p = 0.01), whereas most other perception components (e.g., adoption/satisfaction, efficiency, verification behavior) showed no significant differences.
• Demographic moderators. Age (χ² = 68.80, p < 0.001, Cramér's V = 0.45), gender (χ² = 176.97, p < 0.001, V = 0.72), and geographic region (χ² = 53.87, p < 0.001, V = 0.58) were significantly associated with future use, with users concentrated in the 18–21 age range and higher male adoption (71.1% of males vs. 51.1% of females favoring use). Academic specialization, study level, year of study, and AI experience were not significant.
• Overall adoption intent. A majority of respondents (65.7%) expressed interest in using ChatGPT for academic purposes in the future. The sample was largely male (72.9%), with mean age 21.4 years, mostly bachelor's students (77.1%), and over half (57.2%) educated in Asia.
• Model performance. SVC achieved the highest balanced performance (MCC 0.54 ± 0.19, F1 0.88, accuracy 0.82, AUC 0.85); LR reached MCC 0.52 with the highest precision (0.90), and RF the highest recall (0.91). Clustered MCC values indicate that outcomes depend more on feature representation than on algorithm choice.
• Nonlinearity and interaction. SHAP interaction heatmaps, PDP/ICE curves, and 2D partial dependence surfaces reveal non-additive relationships — notably between learning support, reliability/accuracy concerns, and overall satisfaction — supporting the use of ML over purely linear models.
• Calibration and robustness. SVC showed the best calibration (Brier 0.14, ECE 0.10). Sensitivity analyses across seeds, sampling strategies, and PCA thresholds, plus an ablation study, showed performance was stable and that PCA benefits linear models but can reduce ensemble-model performance.

What this means for practice

• Instructors. Embed ChatGPT in guided activities with explicit instructions on appropriate use, verification strategies, and reflective engagement instead of leaving adoption to independent student experimentation, since Learning Support & Accessibility was the most influential predictor.
• Instructors. Tie AI use to Feedback, explanation, and formative support, because Academic Assistance & Engagement carried the strongest association with intended future use.
• Administrators. Adopt balanced AI Governance — academic-integrity guidelines plus support for pedagogical innovation — rather than purely restrictive controls, which do not map onto what predicts adoption.
• Policymakers. Target training and support by student group: adoption intent differed sharply by gender (71.1% of males vs. 51.1% of females favoring use), age, and region, while integrity concerns did not separate users from non-users.
• Administrators. Require interpretable, calibrated modeling — SHAP explanations plus Brier score and Expected Calibration Error — whenever predictive insight is used for educational decisions.

Limitations

• Convenience sampling and a modest sample: 166 responses were collected and only 111 remained in the final analytical sample after conservative outlier removal, which the authors note may introduce bias and limit generalizability.
• No external validation: despite stratified 10-fold cross-validation with SMOTENC, the models were never tested on an independent population, restricting their applicability beyond this sample.
• The design is cross-sectional and self-report, predicting intended future use rather than observed behavior, and SHAP explanations describe model behavior rather than causal mechanisms.
• The sample is skewed — 72.9% male, mean age 21.4 years, 77.1% bachelor's students, and 57.2% educated in Asia — so group contrasts rest on uneven cells.

Citation

Verma, C., & Kumar, D. (2026). An Exploratory Machine Learning Approach to Understanding Determinants of Future ChatGPT Use in Higher Education. Computers and Education: Artificial Intelligence, 10, 100613.