The AI Writes the Code, the Student Writes the Model: A Theory and Measurement Programme for Learning by Construction with Generative AI

Synthesis: This theoretical paper develops Code-to-Learn with Generative AI (CtL-GenAI), a framework for upper-secondary students (roughly fifteen to eighteen) who build software with generative AI and are meant to learn the subject the software models — physics, chemistry, a real problem — rather than to learn coding itself. Its organising claim is that the AI may write the code while the student must write the model. To make that slogan testable, the paper defines a model authorship construct (four facets × four ordered levels, with observable indicators and a partial-credit measurement model), partitions the outcome space into supported performance, unsupported performance and delayed transfer (defining the assistance-removal gap), recasts verification of AI output as a signal-detection problem (separating sensitivity from response criterion), audits twenty-four studies, and states six hypotheses with designs, measures and falsification criteria.

Key Findings

The performance–learning dissociation. Three recent randomised studies (Bastani et al., 2025; Fan et al., 2025; Liu et al., 2026) converge on the same shape: AI assistance raises measured performance while available, then lowers (or fails to raise) it once withdrawn — in mathematics, writing, and reasoning/reading, across roughly 2,300 participants. The manipulation that raises supported performance and lowers unsupported performance is a pattern the field has not named; the paper calls it the performance–learning dissociation.
The unit of analysis should be the task, not the tool. Whether generative AI helps or harms learning turns on how the task is designed, since the same tool yields opposite outcomes under different task structures. The central prediction is a moderation of the assistance-removal gap by task design — not a main effect of the tool.
The field has been measuring the wrong outcome. Across the twenty-four audited studies, the overwhelming majority measured performance while the AI remained available; one reported a transfer-oriented sub-measure; none reported performance after assistance was withdrawn. Most of the corpus therefore cannot in principle detect the dissociation.
Model authorship is a measurable construct. Defined as the degree to which a learner, rather than the system, originates and retains responsibility for the domain-level content of an artifact, it has four facets (specification, conceptual model, verification, interpretation), four ordered levels (delegated, shepherded, co-constructed, authored) with observable indicators, and a specified partial-credit measurement model whose unidimensionality, cumulativity and invariance are refutable assumptions.

The performance–learning dissociation

The paper opens from the phenomenon it is built against. Writing a small program is no longer a barrier for most students — a learner describes what they want and a large language model returns running code. Schools have responded mostly with permission rules, and the AI-literacy literature asks what students should understand about the systems. Neither tells a teacher how to arrange a lesson so that building with AI leaves the student knowing more physics or more about a local environmental problem.

The empirical record has grown blunt about the risk. In a field experiment with nearly a thousand Turkish high-school mathematics students, access to a standard GPT interface raised practice performance sharply (a gain on the order of 48% relative to control) but left the same students about 17% worse on an exam taken without the tool; a guardrailed version removed the deficit without removing the benefit (Bastani et al., 2025). A randomised study of 117 university writers found the ChatGPT group improved essay scores most while gaining no more knowledge and transferring no better, a discrepancy the authors traced to bypassed metacognitive processes they named metacognitive laziness (Fan et al., 2025). A series of RCTs with 1,222 participants found assistance improved in-the-moment performance yet left people significantly worse and less persistent once it was withdrawn, with effects emerging after roughly ten minutes (Liu et al., 2026). Three populations, three domains, one shape: what a learner can do with the machine and what they can do afterwards have come apart.

The paper is careful about what the dissociation does not show. It is a finding about particular assistance designs, not about generative AI as such — in each study a differently structured condition reduced or avoided it. Supported performance is not worthless: there are legitimate goals (motivation, access for students who would otherwise produce nothing, authentic participation in tool-mediated practice) for which performance with the tool is the right outcome; the error lies in reporting it and drawing conclusions about learning. And none of the three studies examined construction tasks of the kind this framework addresses, where the artifact is a model of something and the learning target lies in the domain rather than in the artifact.

Theoretical foundations

The framework rests on six theories, each introduced with the specific design question it answers and the prediction it licenses — rather than cited decoratively. Constructionism (Papert) supplies the premise that understanding is built most robustly when learners make something public and personally meaningful, and answers why students should write code rather than converse with a chatbot; generative AI changes the economics of construction by lowering entry cost, provided construction stays with the learner. Productive failure (Kapur) provides the warrant for placing specification before prompting — a meta-analysis of 166 experimental comparisons (Sinha & Kapur, 2021) found problem-solving-before-instruction outperformed instruction-first designs on conceptual understanding and transfer (mean d = 0.36), and the number of distinct solutions a learner generates predicts learning. Cognitive load theory yields a sorting rule — delegate the incidental, protect the essential — since offloading syntax frees working memory while offloading the processing that constitutes learning is corrosive; the author deliberately avoids quantifying load numerically because the intrinsic–extraneous–germane partition has proved hard to measure. Self-regulated learning and metacognitive monitoring frame prompting, judging output and deciding to revise as forethought, monitoring and reflection, with the key mechanism plausibly monitoring accuracy rather than motivation: fluent, well-formed output invites learners to mistake processing ease for comprehension. The ICAP model (Chi & Wylie) ranks engagement from passive through active, constructive to interactive, giving a vocabulary for the difference between copying AI output unchanged and critiquing or arguing with the model. Finally, scaffolding theory (Vygotsky's zone of proximal development, Wood, Bruner & Ross) frames the AI as a more capable partner whose help must be calibrated and withdrawn — support that never recedes produces dependence rather than competence — and the evaluative judgement literature names what fading is meant to leave behind: the capability to make decisions about the quality of work against standards.

The synthesis states the organising principle: the AI writes the code, the student writes the model. Writing the model means retaining the specification, the conceptual structure, the verification against the world, and the interpretation of results. Constructionism explains why retaining authorship matters, productive failure why specification precedes the artifact, cognitive load theory which parts are safe to delegate, SRL and ICAP how to keep the learner cognitively active and why fluent output threatens that, and evaluative judgement plus scaffolding why the AI's contribution must shrink over time.

Model authorship and the measurement model

Model authorship is defined deliberately narrowly in two respects. It concerns domain-level content — a student who writes every line of code while holding no view about the physics scores low — and it is a property of a particular artifact and task (an episode of work), not a trait of a person, which distinguishes it from dispositional constructs like AI self-efficacy or trust in automation and from Molenaar's (2022) automation levels, which characterise the configuration of an environment rather than the content of a learner's contribution within it. The four facets are specification, conceptual model, verification, and interpretation, each scored on four ordered levels — delegated, shepherded, co-constructed, authored — where the level names inherit vocabulary from Prather et al. (2023), and each cell carries an observable indicator. Level 3 (authored) requires a verifiable specification that precedes and constrains the first prompt, rejection of output on domain grounds with a stated reason, and interpretation that goes beyond the artifact's own report of itself.

The paper confronts the framework's most obvious vulnerability head-on: is a natural-language specification really modelling? The objection is that the value of programming in the constructionist tradition is bound up with the unforgiving nature of the machine — a sentence can be vague in ways an executable program cannot. The answer has three parts: the specification must be verifiable (it must generate a prediction) rather than merely descriptive; the confrontation with precision is relocated, not removed, from compile-time to a validation phase where a prediction meets the artifact's behaviour; and, empirically, comprehension does not in fact arrive with the natural-language framing — novices explaining LLM-generated code succeeded on roughly a third of tasks, with automation bias among the causes (Nguyen et al., 2025). This both supports the framework's insistence on glass-box prompting and verification and sets a real limit, treated as a boundary condition.

Treating each facet as a polytomous item, the natural measurement model is Masters' (1982) partial credit model, and specifying it makes three assumptions explicitly refutable: unidimensionality (the facets claim to measure one thing — a principal-components analysis of residuals could split authoring facets from evaluating facets, refining rather than destroying the framework), cumulativity (the four levels are ordered — checked via ordered thresholds or Mokken scalability, H ≥ 0.30 weak, H ≥ 0.50 strong), and invariance across domains and tasks (via differential item functioning). Reliable person estimates require multiple tasks (three tasks per learner give twelve observations), coding from at least three sources by two raters on ≥20% of cases with a target weighted kappa ≥ 0.70, or else a sum-score composite from 0 to 12 with the caveat that its assumptions are then untested.

The outcome space and the assistance-removal gap

The paper formalises the outcome the field has been conflating. Domain competence is assessed under three regimes: supported performance (with the AI available), unsupported performance (an equivalent assessment immediately after, without the AI), and delayed transfer (related but non-identical problems after at least two weeks, without the AI). The assistance-removal gap is their difference, expressed in a common standardised metric — a quantity the literature has almost never reported because the two terms are rarely both collected. Three claims follow precisely: the dissociation is a sign pattern (positive effect on supported performance, null or negative on unsupported) that reporting a single coefficient makes undetectable; the central prediction is that the gap decreases as model authorship increases — an interaction between design condition and assessment regime, not a main effect; and the direction matters, because the gap could in principle be reduced by depressing supported performance, whereas the framework predicts it is reduced from below (by raising unsupported performance and delayed transfer). Estimation uses a mixed model with regime as a within-subject factor and design condition between-subject, with counterbalanced, equivalent assessment forms, pre-test as covariate, and classroom as a random intercept.

Verification as a signal-detection problem

The framework's fourth principle asks students to judge whether an artifact is correct on domain grounds, which is formally a detection task: over a series of outputs, some flawed, the learner accepts or rejects, and sensitivity (d′) measures the ability to discriminate correct from flawed output while the criterion (c) measures where the learner sets the threshold for rejection, independently. The decomposition resolves several things at once. It separates two phenomena the literature runs together: over-reliance factors into a discrimination problem (d′ near zero — the learner genuinely cannot tell) and a criterion problem (d′ adequate but the learner accepts anyway), which have different remedies — exhortations, checklists and hallucination warnings shift c, while domain instruction, worked comparisons and seeded-error practice raise d′. It gives a precise reading of the prior-knowledge boundary: verification instruction is only educative where d′ can exceed zero, which requires the learner already to command enough of the domain to distinguish correct from incorrect output — below that threshold the Validate phase degenerates into criterion-setting and the framework should not be expected to work. And it connects the framework to the appropriate-reliance literature without importing its ambiguity: sensitivity and criterion are exactly the two parameters appropriate reliance requires, and trust measures do not deliver them. Appendix C supplies a seeded-error verification task (twenty items, eight containing a domain-level error) suitable for estimating both parameters in a classroom.

Design principles, the cycle, and the evidence audit

Six design principles operationalise the mechanism: problem or concept first (every artifact serves a prior question — "build an app" is never the task); student as specifier (the learner articulates the logic in their own words, in a form from which a prediction can be derived, before any code — with the solution-count finding making more than one candidate specification a dosage variable); glass-box prompting (prompts require explanation as well as code, enforced because self-explanation cannot be assumed); verify against the world (the artifact is judged by domain criteria rather than by whether it runs); productive friction (reserve moments where the AI is withheld — predict before you run, change a parameter by hand, form a hypothesis about a bug — which is especially valuable in physics where fast intuitive heuristics override knowledge); and scaffold fading and reflection (AI support is reduced across a unit, with each cycle closing in structured reflection on what the learner did, the AI did, and what is now understood). These are enacted through an iterative five-phase cycle — Frame, Specify, Co-construct, Validate, Reflect and Share — where validation routinely sends a class back to re-specify, and an ICAP-based check lets teachers read engagement in the moment. A worked illustration uses a ninth-grade hydrangea-colour chemistry activity (Chen, 2025): the AI wrote the simulation code while the students held the chemistry and specified and verified the behaviour — Level 3 on the rubric — and a physics analogue makes the fading explicit across a unit on Newton's second law.

The evidence audit of twenty-four studies (2021–2026, Scopus, integrative-review method with best-fit framework synthesis) is deliberately positioned as a gap-locating audit, not a systematic review: it searched a single database, excluded conference proceedings, screened single-author with twenty per cent double-screened, and conducted no formal quality appraisal — adequate for locating gaps, inadequate for estimating effects, so no pooled estimate is offered. Three findings emerge. First, the field has been measuring supported performance, and a large share of screened records concerned what the AI can produce rather than what students learn — the dominant research question has been about the tool. Second, the same tool yields opposite outcomes and task design decides which: a ChatGPT group with a conventional setup ended significantly lower in achievement, self-efficacy and flow (Yang, Hsu & Wu, 2025), while a different configuration with question-chain task sheets, verification requirements and error-reflection modules showed markedly higher engagement and stronger higher-order thinking (Zhang, Jiang, Xiong & Zhao, 2025), and a Socratic-constrained chatbot outperformed both a conventional ChatGPT interface and a no-chatbot control (Kholis et al., 2026). Third, where studies protected student cognition — prohibiting the AI from performing the algorithmic thinking, requiring causal self-explanation, externalising a plan before generating code — engagement and understanding rose; where assistance was unstructured, the shepherding and drifting that ICAP predicts will not produce learning appeared. The science strand is small but telling, showing the mechanism operating outside computer science: students whose engineering-design or physics-experiment projects were LLM-supported, with the AI mapped onto each practice while students retained the disciplinary reasoning, significantly outgained controls in scientific knowledge, computational thinking and problem-solving (Guo et al., 2025; Wang et al., 2026). The clearest instantiation of the whole cycle comes from engineering, where buggy AI-generated code became the occasion for elaboration and students reported feeling they were teaching the model — while learners without sufficient prior knowledge struggled to debug, marking the boundary (Vieira et al., 2025). Every design principle has a theoretical warrant and most have descriptive support, but the column that matters for distinguishing learning from supported performance is empty throughout; scaffold fading (Principle 6) is the least tested, with a single suggestive signal from a progressive mind-map condition.

Implications

For Assessment, the paper's central implication is that if the AI can produce the artifact, the artifact cannot be the assessment. Evaluation relocates to the specification, the validation reasoning and the interpretation — precisely the facets the AI cannot supply on the student's behalf. This favours specification documents, predict-then-test records, oral defences and process portfolios over submitted code, and defuses much of the Academic Integrity anxiety around generative AI because what is assessed is what could not have been delegated. The point is not abstract: a benchmark found generative AI answering secondary-level physics items at over eighty per cent accuracy while often sounding plausible when wrong (Yeadon & Hardy, 2024), so non-invigilated artifacts are both easy to fake and unsafe to trust, and the capability that protects a student is evaluative judgement rather than prohibition. Relocating assessment also changes what Feedback can do — a specification, a prediction record and a validation log give a teacher the task/process/self-regulation surfaces that feedback is most effective on, which submitted code does not — and, because the authorship rubric is scored from materials classrooms already generate (specifications, interaction logs, short interviews), the same instrument serves as both formative assessment and a research measure.

For Curriculum Design and computing education practice, the framework relocates the design problem from "how much AI to allow" to "which parts of a construction task must remain with the learner, and why," positioning programming as the medium through which a science concept or real problem is learned rather than an end in itself. It connects to Computational Thinking (decomposition, abstraction, modelling, systematic debugging as the dispositions involved) and to teacher capacity: the framework reassigns the teacher from syntax instructor to designer of specification and validation, a more demanding role that requires professional learning, with prompt-engineering skill emerging as a precondition for adoption.

For Educational Measurement and theory development in AIED, the paper's contribution is a measurement programme: a construct with facets, levels and a refutable partial-credit model; an outcome space that separates what a learner can do with the machine from what they can do without it; a detection-theoretic account of what verification builds and when it cannot; and six hypotheses (specification-before-prompting, dosage of specification, predict-before-run, fading and the assistance-removal gap, verification sensitivity and its prerequisite, and authorship predicting durable learning), each with designs, measures, competing explanations and explicit falsification criteria. Five boundary conditions are stated as part of the theory rather than caveats about it: domains must furnish an external criterion (physics, chemistry, ecology and epidemiology supply one; history, literature and ethics largely do not); prior domain knowledge must let d′ exceed zero; the fifteen-to-eighteen age range is best read as a statement about required scaffolding rather than capability; access to devices must be reliable or the framework can widen gaps; and teacher capacity is an assumption rather than a demonstrated result. As a theoretical proposal, none of its six hypotheses has yet been tested in the form stated, and the measurement model is specified but not yet calibrated.

Connected Concepts

Connected Articles

Competency Based Education GenAI Production 2026 — Knowledge, skills, attitudes, production: competency-based education after generative AI
AI Assessment Scale Reform — AI assessment scale reform
Agency Gap AI Writing — The agency gap in AI writing
Pchl He Framework GenAI Content Creation 2026 — A framework for generative AI content creation in higher education

Citation

Gousopoulos, D. (2026). The AI writes the code, the student writes the model: A theory and measurement programme for learning by construction with generative AI. Preprints.org.

View Source View Changes Edit