Structured ChatGPT-Assisted Writing Instruction and Academic Writing Performance Among Jordanian EFL University Students

Synthesis: What does a semester of structured generative AI support do to academic writing when it is randomized rather than self-selected? Wardat and Akour ran an eight-week individually randomized pre-test/post-test trial with 300 EFL undergraduates across four Jordanian universities: 150 students wrote with ChatGPT plus their instructor's regular guidance, 150 received conventional writing instruction. The ChatGPT group gained 12.13 points against 2.14, and an ANCOVA adjusting for baseline put the difference at 9.964 marks out of 100 (partial η2 = 0.545). A PLS-SEM model adds that students' perceptions of the tool's usefulness tracked their reported critical thinking (β = 0.456), self-regulated learning (β = 0.341) and motivation (β = 0.270) — associations measured only after the intervention, so the trial establishes a writing effect and the model describes who reported what, not why the writing improved.

Key Findings

1. Randomized writing gains were large. Post-test means were 71.87 (SD = 6.88) for the ChatGPT group and 62.00 (SD = 6.51) for control, t(298) = 12.775, p < 0.001, Cohen's d = 1.475, from nearly identical baselines (59.74 versus 59.86, p = 0.876).
2. The baseline-adjusted effect held. ANCOVA put adjusted means at 71.916 and 61.952, an adjusted difference of 9.964 points (95% CI [8.925, 11.003]), F(1, 297) = 356.059, p < 0.001, partial η2 = 0.545.
3. Perceived support correlated most with critical thinking. ChatGPT-assisted learning reached β = 0.456 with critical thinking (R2 = 0.208), β = 0.341 with self-regulated learning (R2 = 0.116) and β = 0.270 with academic motivation (R2 = 0.073), all p < 0.001.
4. Those constructs also predicted writing. Critical thinking (β = 0.276), self-regulated learning (β = 0.186) and academic motivation (β = 0.210) each related to writing performance, with the model explaining 36.3% of its variance.
5. Interviews described a shift in how the tool was used. In 20 semi-structured interviews six themes emerged; 19 of 20 participants described using ChatGPT for idea generation and 18 of 20 for feedback and revision, with critical evaluation of suggestions as a distinct theme.
6. Over-reliance was a named concern, not an afterthought. One of the six interview themes was students' own worry about depending on the tool, which the authors read as evidence that structured instruction, not access alone, shaped how it was used.

What the study did

Three hundred undergraduates were recruited purposively from four Jordanian universities and then individually randomized into two groups of 150. Both groups shared objectives, weekly writing tasks, instructional schedule and assessment procedures, and where feasible the same instructor taught both conditions after a training workshop; the only difference was that the ChatGPT group's writing instruction incorporated the tool alongside the instructor's regular guidance.

Outcomes were measured before and after the eight-week period with an academic writing test: essays were scored on four IELTS criteria by two trained raters blind to condition and test occasion, with inter-rater reliability of ICC = 0.917. A questionnaire administered after the intervention measured ChatGPT-assisted learning, critical thinking, self-regulated learning and academic motivation, and PLS-SEM with 5,000 bootstrap resamples estimated the paths among them. The qualitative strand added 20 interviews and four classroom observations drawn from the experimental group.

The design separates two kinds of claim that are easy to conflate. The randomized comparison supports a causal statement about writing scores; the structural model reports post-intervention associations among self-reported constructs, and the authors say so explicitly in their limitations rather than treating the mediation as a mechanism.

Where the motivation and strategy claims stop

The paper is careful about what the SEM can carry, and readers should be too. Because critical thinking, self-regulated learning and motivation were measured once, after the intervention, the model cannot show that ChatGPT raised them, only that students who rated its support higher also rated themselves higher on those constructs. The writing effect is the experimental finding; Feedback utilization, revision and autonomy appear in the qualitative themes as students' accounts of their own process.

That makes the study a stronger test of writing performance than of the self-regulation or motivation claims that AI-writing studies often lead with. It also means the useful question for a course team is not "does ChatGPT raise motivation" but "what instructional structure around the tool produces the writing gain without the over-reliance students themselves flagged" — which the authors locate in the prompt-based guidance and trained instructors rather than in the model.

What this means for practice

• Instructors: Pair the tool with explicit instruction rather than open access. The trial's experimental condition embedded ChatGPT in guided writing instruction, and the gains were measured against conventional instruction, not against no writing support.
• Instructors: Keep the writing task constant across conditions when you evaluate your own change. Both groups here wrote the same assignments on the same schedule, which is what makes the 9.964-point adjusted difference attributable to the instructional use of the tool.
• Curriculum designers: Budget for revision and feedback cycles in the syllabus. Idea generation and revision were the two most-reported uses, so a unit that only asks for a finished draft leaves most of the reported value unused.
• Researchers: Treat post-only self-report constructs as description. The mediation paths are associations in one measurement occasion; a pre/post design on critical thinking, SRL and motivation is the missing piece.

Limitations

• Participants were undergraduates at four Jordanian universities, so the authors state the findings should not be generalized beyond those EFL learners, systems and proficiency levels.
• No delayed post-test was administered, so the study cannot say whether the writing difference persisted after the eight-week intervention.
• Critical thinking, self-regulated learning and academic motivation were measured only after the intervention by self-report, so the structural paths describe post-intervention associations rather than change.
• The same ChatGPT-assistance items were given to both groups, but control students answered from general impressions while experimental students answered from direct experience, a comparability issue the authors acknowledge.
• Random assignment followed purposive recruitment from four universities, so the design supports a causal claim about these volunteers rather than a population estimate for Jordanian EFL students.

Citation

Wardat, S., & Akour, M. (2026). Structured ChatGPT-Assisted Writing Instruction and Academic Writing Performance Among Jordanian EFL University Students: Experimental Findings and Associations with Critical Thinking, Self-Regulated Learning, and Academic Motivation. Education Sciences, 16(11), 1922.