Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS)

Synthesis: As generative AI (GenAI) becomes embedded in undergraduate academic writing, how students rely on these tools — not merely whether they use them — has emerged as a core question for Academic Integrity, Student Experience, and educational equity. This study develops and validates the Generative AI Reliance Types Scale (GenAI-RTS), a 20-item instrument measuring four theoretically derived reliance types: Strategic, Instrumental, Dependent, and Dialogic. Confirmatory factor analysis supports a five-factor structure in which Strategic Reliance splits into Deliberate Use and Critical Evaluation (CFI = .92, RMSEA = .08; DWLS CFI = .98, RMSEA = .07), with subscale reliability (ω) ranging .75–.90.

Validation follows the multi-source framework of the Standards for Educational and Psychological Testing, combining a survey of 382 undergraduates at a U.S. Minority-Serving Institution with interviews with 14 purposively sampled students. The Critical Evaluation facet — students scrutinizing GenAI output rather than accepting it — directly operationalizes AI Literacy in writing contexts, while the Dependent type maps onto documented patterns of Over-Reliance on Generative AI tools. The instrument gives researchers and instructors a validated way to measure reliance modes in Writing and across Higher Education, moving beyond binary “use/no-use” measures toward nuanced, equity-aware assessment of GenAI integration.

What this means for practice

• Instructors. Measure how students rely on GenAI rather than whether they use it: score reliance on the four validated types (Strategic, Instrumental, Dependent, Dialogic) with the 20-item GenAI-RTS instead of asking binary use/no-use questions.
• Instructors. Expect Dependent reliance to be under-reported. Interviews showed students describing dependent use cautiously, sometimes out of fear of institutional consequences, so corroborate self-report with drafts, revision histories, or short conferences rather than treating the scale as proof of behavior.
• Faculty developers. Use the five-facet structure — Deliberate Use and Critical Evaluation split out of Strategic reliance — when targeting AI Literacy instruction, so an intervention aims at scrutinizing output rather than tool familiarity.
• Researchers. Adopt the instrument where a pre/post or correlational measure of reliance mode is needed, since subscale reliability ranges from .75 to .90, and pair it with behavioral measures rather than relying on the self-report scores alone.
• Researchers. Use the Strategic composite as a companion outcome alongside an AI-literacy measure: it correlated r = .61 with the AI literacy composite and r = .55 with prior AI exposure in this sample.

Limitations

• Every measure is self-report from a single cross-sectional survey. The authors note the study lacks the AI interaction logs that would anchor response-process claims, and that the criterion variables covered self-reported AI-assisted writing behaviors rather than independently assessed writing quality.
• The structural evidence rests on one validation sample: N = 382 undergraduates at a single U.S. R1 Minority-Serving Institution, recruited over six weeks with an estimated 26% response rate from roughly 1,500 students reached, and the five-factor structure still needs replication in an independent sample.
• No formal content-validity indices were computed; expert review of item content was informal, and the non-binary, genderqueer, and transgender subgroup (n = 25) was too small for full measurement-invariance testing, so only two-group comparisons were possible.
• Test-retest temporal stability of the GenAI-RTS has not been established, and the interview accounts of movement from dependent toward strategic reliance are retrospective and cannot establish developmental change.

Citation

Shahin Hossain, Tukhbita Afroz Nawmi (2026). Measuring How Students Rely on Generative AI in Academic Writing: Development and Multi-Source Validation of the Generative AI Reliance Types Scale (GenAI-RTS).