Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts

Synthesis: Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts

Key Findings

• A mixed-methods study of undergraduates across two discipline–institution contexts in Thailand — Design students at Raffles International College Bangkok (n = 221) and Business (BBA) students at Siam University (n = 222), recruited from Years 1–3 — modeled how AI literacy translates into coursework performance through two behavioral routes and two mediating mechanisms.
• AI literacy significantly predicted both prompting proficiency and verification behavior (p < .001), confirming the proximal-behavior hypotheses; prompting proficiency was in turn positively associated with AI-supported calibration accuracy, while verification behavior was negatively associated with perceived task-related cognitive load.
• Bootstrapped indirect effects of AI literacy on assignment quality were supported in both cohorts (bias-corrected 95% CIs, 5000 resamples): the prompting route (e.g., Design estimate .072, p < .001; Business .051, p = .001) and the verification→lower-load route (e.g., Design .034, p < .001; Business .024, p = .001).
• The most performance-proximal route differed by discipline: verification behavior directly predicted task quality in the Design cohort, whereas AI-supported calibration accuracy — plus a direct AI-literacy path (p < .01) — predicted subject-specific assignment quality in the Business cohort, consistent with each discipline's "task ecology" (open-ended ideation vs. audit-like analytic standards).
• Measurement invariance across the two cohorts was supported (ΔCFI ≤ .010), and the multigroup models showed acceptable fit (χ2/df below 3; CFI/TLI/IFI above .90; residual-based indices in acceptable ranges).
• In 20 semi-structured interviews (thematic saturation reached by the 18th), Design students described using ChatGPT mainly to accelerate ideation and refinement, while Business students used it for structured analysis and to make their work more auditable; across both contexts, verification functioned as a metacognitive safeguard that reduced overload and stabilized accuracy.

Study Design & Method

The quantitative strand surveyed students immediately after course-embedded assignments, measuring AI literacy, prompting proficiency, verification behavior, AI-supported calibration accuracy (CAL), and perceived task-related cognitive load (ECL) via self-report, with rubric-scored performance outcomes (task quality for Design; subject-specific assignment quality for Business). Multigroup structural equation modeling with measurement invariance testing and bootstrapped indirect effects was used; for example, the configural invariance model reported χ2(170) = 356.1, CFI = .958, TLI = .942, RMSEA = .059, SRMR = .048. The qualitative strand consisted of 20 semi-structured interviews with a purposive, maximum-variation subsample, analyzed thematically in NVivo until saturation at the 18th interview; integration used joint displays mapping SEM paths to interview themes, with an audit trail and explicit attention to discrepant cases.

What this means for practice

• Instructors. Teach prompting and verification as two separate, assessable skills: AI literacy predicted both prompting proficiency and verification behavior (p < .001), and each reached assignment quality through a different mediator.
• Make verification routine on high-stakes work; it acted as a metacognitive safeguard that was negatively associated with perceived task-related cognitive load and directly predicted task quality in the Design cohort.
• Match the emphasis to the discipline's task ecology: AI-supported calibration accuracy predicted Business assignment quality, while verification predicted Design task quality, so assessment criteria should follow the kind of work being assessed.
• Embed short prompting practice in regular teaching time — 10–15 minutes at a stretch — paired with rubric-anchored Prompt Engineering instruction, and ask students where they relied on, rejected, or revised AI output.
• Learners. Submit evidence traces (an appendix of links/DOIs plus a 2–3 sentence rationale) with high-stakes assignments so documented judgment, not the volume of AI use, carries the credit.

Limitations

• The design is non-experimental, so causal claims remain tentative despite theory-consistent directional modeling.
• Mediators — prompting proficiency, verification behavior, AI-supported calibration accuracy, and perceived task-related cognitive load — were all assessed by post-task self-report on Likert scales, and residual common-method bias cannot be fully ruled out despite the survey drawing N = 443 students (Design 221; Business 222).
• CAL captured students' self-assessed assignment quality and rubric fit rather than system-level trust calibration, and ECL may blend extraneous load with intrinsic task difficulty and time pressure.
• The sample covers two programs within a single national context, and the single-author mixed-methods design lets a strong prior model shape qualitative coding; thematic saturation was reached at the 18th of 20 interviews.

Citation

Shen, Q. (2026). Same AI, different pathways: Unpacking mechanisms of AI-mediated learning across discipline-institution contexts.