Research Article
Teacher-regulated generative AI in structured group tasks: independent performance and metacognitive judgements in higher education
Synthesis: Zhang and colleagues (2026) asked whether AI support in a classroom group task changes not only what students can do but how accurately they judge what they know. In a group-randomized, post-test-only design, 126 undergraduates in two intact classes worked a structured case discussion in Rain Classroom, with the platform's built-in AI learning companion available to half the pre-existing groups and withheld from the other half; students then predicted their own score before sitting an individual post-test. AI-supported groups scored higher on the independent assessment (B = 0.506, SE = 0.188, p = 0.013) with a small standardized difference (d = 0.196), but showed no detectable difference in predicted performance, confidence, calibration bias, or absolute calibration error. The authors read the pattern as better performance without better metacognitive monitoring.
Key Findings
- AI support raised independent post-test scores. Students whose groups could use the platform's built-in AI companion scored higher on the individual post-test than students in non-AI groups, though the standardized difference was small.
- Metacognitive monitoring did not move with it. Predicted performance, confidence judgments, calibration bias, and absolute calibration error showed no statistically significant between-condition differences.
- Group-level randomization protected the classroom task. Condition was assigned to 28 pre-existing groups rather than to individuals, and the final sample retained 27 analysable groups covering 126 students.
- AI was bounded, not open. The AI condition used the platform's built-in learning companion inside a teacher-designed structured task, not an unrestricted chatbot, which limits how far the result generalizes to open AI use.
- No baseline means change cannot be read. With no pre-intervention outcome measures, the design supports between-condition description only, not within-student growth.
What the study separated
Most classroom AI studies collapse three different questions into one: did students produce better work, did they learn more independently, and did they know how much they had learned. This design pulls them apart by measuring an individual post-test after a group activity, then asking each student to predict their score on that post-test before taking it. The mixed models, adjusting for class and group clustering, put the AI-supported condition at B = 0.506 (cluster SE = 0.188, p = 0.013, 95% CI [0.117, 0.894]) on the individual post-test. That performance result is real but modest — d = 0.196 sits at the boundary of what a 126-student sample can detect. The monitoring result is the more consequential one: students in AI-supported groups were no better and no worse at anticipating their own performance, which means the AI did not inflate their sense of competence even though it improved their output. That is a different finding from the automation bias worry that fluent AI answers breed false confidence.
Why fluency and calibration come apart
The authors' interpretive frame is metacognition for the outcomes and distributed cognition for the setting, and they are careful to say the latter was not measured. That caution matters: in a structured group task, a student can draw confidence from the group's shared answer, from a persuasive AI suggestion, or from the fluency of the discussion, and none of those pathways was observed directly. What the data show is that the net effect of all of them on calibration was null. A plausible reading is that the teacher-regulated design kept the AI inside a task whose answer still had to be produced individually, so the AI supported the discussion without standing in for the student's own judgment. Human-in-the-loop framing fits: the teacher set the task and the platform bounded what the companion could do.
The ceiling on the claim
Two structural features cap the inference. The design is post-test-only, with no pre-intervention measures of performance or calibration, so baseline equivalence between the randomized groups is assumed rather than shown and no within-student change can be estimated. And the setting is narrow: two intact classes in one course at one Chinese university, groups formed by student choice and ranging from one to eight members, with complete-case analysis and no imputation after 13 participants were excluded. The group-level randomization is a genuine strength for ecological validity, but with 27 analysable groups the study is powered for a moderate performance effect and not for a small one on calibration — a true but small monitoring difference would be invisible here.
What this means for practice
- Instructors. Do not treat a stronger AI-supported product as evidence that students know more; ask for a prediction before the individual check so the gap between confidence and performance becomes visible.
- Learning analytics designers. A platform that reports task completion should also capture prospective judgments, because the performance gain here arrived with no measurable improvement in self-monitoring.
- Researchers. Power calibration and confidence outcomes separately from achievement, and report the minimum detectable effect; a d of 0.196 on performance leaves little room for a small monitoring effect to register in 126 students.
Limitations
- Post-test-only design with no pre-intervention measures, so baseline equivalence and within-student change cannot be assessed.
- Two intact classes in one course at one Chinese university, with groups formed by student choice and ranging from one to eight members.
- Complete-case analysis with no imputation after 13 participants were excluded for withdrawal or incomplete data.
- Distributed cognition framed the interpretation but was not measured, so peer uptake of AI suggestions is inferred rather than observed.
Citation
Zhang, W., Yang, M., Bai, L., Liu, Y., & Dong, X. (2026). Teacher-regulated generative AI in structured group tasks: Independent performance and metacognitive judgements in higher education. Frontiers in Psychology, 17, 2007654.