Can AI make reading more engaging? A motivational framework for AI-assisted reading assessment in EFL/ESL classrooms: a practitioner case study

Synthesis: Almirabi documents a practitioner-built workflow, the AI-Mediated Motivational Reading Assessment (AMMA) framework, and runs it for six multiple-choice quizzes with 53 Arabic L1 EFL undergraduates at one Middle Eastern university (Section 1 n = 33, Section 2 n = 20, B2 on the CEFR) across one semester. Quiz 1 used a textbook passage online and unproctored; Quizzes 2 and 3 AI-generated passages, also online and unproctored; Quizzes 4 to 6 AI-generated passages on paper and invigilated. Means ran from 69.4% (SD 25.1) on the textbook quiz to 84.8% (SD 22.4) on Quiz 2, with a cycle mean of 76.5%; the AI aggregate of 77.8% sat 8.4 points above the textbook quiz and 4.2 points above it once unproctored quizzes were excluded. No comparison isolates passage source from delivery, proctoring, sequence or instrument difficulty, no inferential test was run, motivation was never measured, and the behavioral record comes from its designer. The author's conclusion governs: the data are consistent with, but do not establish, a motivational or comprehension benefit.

Key Findings

1. Six quizzes went to 53 students in two sections (33 and 20): Quiz 1 textbook, online and unproctored; Quizzes 2 and 3 AI-generated, online and unproctored; Quizzes 4 to 6 AI-generated, paper and invigilated.
2. AI-generated quizzes averaged 77.8% (SD 19.8, pass rate 84.0%) against 69.4% (SD 25.1, pass rate 77.6%) on the single textbook quiz: 8.4 percentage points, with dispersion narrower by 5.3 SD points.
3. Online quizzes averaged 78.9% (SD 23.7, pass rate 86.6%) against 73.6% (SD 17.7, pass rate 79.1%) for paper, a 5.3 point gap and 7.5 points on pass rate; among AI quizzes alone the online lead widened to approximately 10 points (84.4% against 73.6%).
4. Restricting the AI aggregate to the three invigilated quizzes gave 73.6%, which is 4.2 points above the textbook quiz and still confounded with sequence position.
5. The cycle mean was 76.5% with 83.0% passing at the 60% threshold; quiz means ranged from 69.4% to 84.8%, N per quiz fell between 41 and 49, and absence ranged from 7.5% to 22.6%.

The six-stage cycle, as run

The framework pairs four motivational conditions, interest, achievement, practice and focused attention, with six stages that run from prompt to published grade inside one class session: generate a passage at a specified CEFR level, check it for factual accuracy, cultural appropriateness, language quality and curriculum alignment, then generate and review items for distractor plausibility, key accuracy and ambiguity. Items were revised or replaced at a rate consistent with the 15% to 30% Kurdi and colleagues report, with key-distractor ambiguity the most frequent defect, though revision records were not kept. Item count follows session length; students may leave when finished; ZipGrade with Blackboard returns scores within 15 to 20 minutes of completion, with preparation of 25 to 45 minutes per cycle. Both sections completed six cycles unchanged.

Why no comparison isolates an effect

The textbook quiz sits first, so improvement is equally consistent with practice, with familiarization with the instructor's conventions and with cohort composition: Quiz 1 had 49 students and Quiz 3 had 41. The baseline is one passage of uncalibrated difficulty. Proctoring is the sharper problem: every online quiz was unproctored, so dictionaries, translation tools and AI assistants were within reach for the two highest means in the cycle. The author treats the paper means of 73.7%, 70.3% and 77.0% as the defensible estimates of independent comprehension. No inferential test was run, so nothing in the tables can be compared against chance.

What the observations can carry

The qualitative contrast was the study's original impetus: peer discussion, grade anticipation, students reviewing rather than leaving, one student calling Quiz 6 "interesting". The observer designed the framework, taught the course and wrote the paper, the least independent position available; no coding scheme or second rater existed, so no reliability can be reported; recording was retrospective; and the comparison is remembered textbook sessions rather than contemporaneous observation. Two readings are withdrawn: flow from early-finisher reviewing, and intrinsic motivation from continued presence, the second because the quizzes were graded. The departure protocol alters autonomy, social comparison, enforced waiting and surveillance at once.

Relevance against transfer

The Interest pillar selects for demographic fit, which schema research predicts will ease comprehension. The same research predicts a cost: a reader fed only schema-supported text is not practicing unfamiliar material, and generated prose tends toward explicit structure and controlled vocabulary unlike authentic academic prose. The paper takes the objection seriously, noting that its widest dispersion came on the textbook passage, which is what one would expect if authentic prose discriminates sharply between readers. On that reading the compressed AI distributions may mean reduced diagnostic value rather than better comprehension.

What this means for practice

Instructors should treat the review protocol as load-bearing, since the achievement claim rests on item quality: verify facts rather than editing prose, read each item adversarially, and log revisions. Publish scores the same session, and use invigilated or proctored delivery wherever scores are read as comprehension evidence; unproctored online administration is for low-stakes practice. Where the instructor does not share the students' background, review by a colleague or by students is the only real check on stereotyped content.

Limitations

• The design is a single-cohort descriptive case study with no control group, no random assignment, no counterbalancing and no pre-test; the author does not analyze it as a quasi-experiment.
• Passage source, delivery, proctoring, sequence and cumulative exposure vary together, and the baseline is one quiz in first position, so no difference can be attributed to AI-generated content.
• Motivation was never measured. No validated instrument was administered, terms such as intrinsic motivation function as framing, and the behavioral record came from the framework's designer without coding or a second rater.
• Absence reached 22.6% on one quiz and was not modeled, so means may overstate performance; revision records were not kept; and one instructor and one semester cannot separate novelty from durable effect.

Citation

Almirabi, M. (2026). Can AI make reading more engaging? A motivational framework for AI-assisted reading assessment in EFL/ESL classrooms: a practitioner case study. Frontiers in Education, 11, 1908499.