Computational Thinking Development in AI Agent Creation: A Mixed-Methods Study

Synthesis: This mixed-methods study examines how 93 pre-high school students develop Computational Thinking skills through a five-day AI agent creation workshop using CocoFlow, a no-code platform. The study integrates pre-post assessments, behavioral logs, and interviews to trace learning trajectories.

Key quantitative findings:

• Abstract thinking: significant improvement (d = 0.71)
• Algorithmic thinking: significant improvement (d = 0.70)
• Iterative testing engagement predicted Self-Efficacy gains (β = 0.20, p = 0.05)

The Optimal Development Zone effect (η² = 0.55): Students with moderate initial CT levels showed substantially greater gains than both high-CT and low-CT peers. Qualitative analysis revealed why:

• Moderate-CT students exhibited adaptive expertise — flexible, effective Problem Solving
• High-CT students risked over-engineering — creating unnecessarily complex solutions
• Low-CT students struggled with task decomposition — breaking problems into manageable parts

These findings challenge linear learning assumptions in K-12 and provide direct evidence for differentiated Scaffolding in CT education. The Optimal Development Zone concept parallels Mapping the Scaffolding of Metacognition and Learning by AI Tools in STEM Classrooms: A Bibliometric-Systematic Review findings on tailoring AI support to student readiness. The study also contributes to the AI Literacy evidence base by showing that no-code AI agent creation platforms can effectively develop CT in young learners, complementing work on Agentic Education with AI Coding Assistants with older students using coding assistants.

What this means for practice

• Instructors. Differentiate the task, not just the tool. Moderate-CT students gained most from open-ended challenges, high-CT students needed constraint-based tasks to curb over-engineering, and low-CT students needed structured pathways with agent modification tasks.
• Instructors. Reward iterative testing over first-try polish. Iterative testing engagement predicted Self-Efficacy gains (β = 0.20, p = .050) after controlling for initial self-efficacy, so the quality of interaction with the AI system mattered more than initial design accuracy.
• Learners. Build in the test loop yourself — the real-time chat simulator that let you try a design and see it fail is the behavior tied to confidence gains, and complexity added before testing does not pay off.
• Instructors. Do not assume a no-code platform removes the need for decomposition support. Low-CT students' primary struggle was breaking problems into manageable parts, not operating the tool.

Limitations

• The intervention lasted only five days — nine hours on Days 1-3 and twelve hours on Days 4-5 — a short window that cannot speak to persistence or transfer of Computational Thinking gains.
• No post-intervention objective CT assessment was administered. The headline results are self-perception measures from paired-samples t-tests on N = 84 of the 93 participants, so reported gains may reflect confidence rather than competence.
• Single site, no control group: 93 incoming high school freshmen at one public high school in southern China, recruited through voluntary enrollment without prior screening for programming experience.
• Pattern recognition and generalization improved only modestly (d = 0.35), which the authors read as a higher-order capability that a five-day workshop may not be long enough to cultivate.

Citation

Sun, Y., Xin, H., Niu, Q., Li, S., Huang, L., & Chen, G. (2026). Computational thinking development in AI agent creation: A mixed-methods study.