Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics

Experimental study comparing Guided vs. Unrestricted LLM access. Explicit training in reasoning-focused scaffolding (stepwise hints, verification) led to significantly better independent performance and self-assessment calibration compared to uncritical reliance. This work emphasizes that AI Literacy is a developmental capacity requiring structured Scaffolding and Prompt Engineering discipline. It connects to the need for Curriculum Design that targets Metacognition and Agentic AI rather than just syntax mastery.

Key Findings

In a four-week quasi-experimental summer program in an undergraduate Probability and Statistics course, students were organized into three balanced conditions: no LLM access, unrestricted LLM access, and guided LLM access.

The guided condition used the same LLM platform as the unrestricted condition, but students received explicit training and rules intended to promote reasoning-focused help-seeking, stepwise hints, verification, and ethical use.

Guided use was associated with a clearer learning-oriented interaction pattern than unrestricted access, especially in prioritizing reasoning over final answers and requesting stepwise support.

Guided-LLM students showed a promising pattern of stronger no-help quiz performance during the intervention phase, while unrestricted access appeared more useful for assisted practice completion than for consistently improving independent performance.

All quizzes and the delayed final exam were completed without LLM or external assistance, separating AI-supported practice performance from independent learning; available time measures did not support a simple duration-based explanation, and self-assessment calibration suggested better alignment between perceived and demonstrated understanding in the guided condition.

Study Design & Method

The design distinguishes between assigned LLM access and the quality of students' actual interaction with the model. The three balanced conditions isolate the effect of guidance: because the guided and unrestricted groups used the same platform, differences can be attributed to training and usage rules rather than tool availability. The use of LLM-free quizzes and a delayed final exam provides a no-help measure of whether AI-supported practice transferred to independent performance.

Implications for AI in Education

The central conclusion is that LLM access alone is an incomplete educational intervention: for AI Tutoring and Curriculum Design, the design challenge is to scaffold how students use LLMs so that these systems function as partners in reasoning rather than answer-getting tools. The findings support investing in Prompt Engineering-style training and help-seeking guidance, and align with concerns about Over Reliance when access is unrestricted — while noting that the study's modest scale and single-course context warrant replication.

Connected Concepts

Connected Articles

Tracing GenAI Literacy Interaction Patterns — Tracing GenAI Literacy: Student-AI Interaction Patterns in Academic Writing

Ase 26 Agentic Software Engineering Curriculum — ASE-26: A Curriculum for Agentic Software Engineering as a Discipline

AI Adoption Training Public Sector — The Main Barrier to AI Adoption in the Public Sector is Lack of Training

Agentic AI Education Scoping Review — Agentic AI in Education: A Scoping Review of Research Landscape, Capabilities, and the Frontier Agent Paradigm

Learning By Chatting GenAI Impact — Learning by Chatting? Investigating the Impact of Generative AI on Information Seeking and Learning

Slidesqaqa Pedagogical Question Generation — Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation

Citation

Amanlou, M., Amou-Jafari, Y., Livani, M., Boloukazari, F., Bagheri, F., & Bahrak, B. (2026). Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics. Proceedings of the 34th International Conference on Computers in Education.