Language teachers’ AI literacy: A psychometric study based on the ED-AI framework

Synthesis: Nabhan and Habók develop and validate the Teachers' AI Literacy Scale (TAILS), a self-report instrument grounded in the ED-AI literacy framework (Allen & Kendeou, 2024), which conceptualizes AI Literacy across six dimensions: knowledge, evaluation, collaboration, contextualization, autonomy, and ethics. Arguing that most existing instruments target students, general users, or nonexperts, the authors position TAILS as a discipline-specific measure that reframes AI literacy as a pedagogical and ethical capacity—rather than a purely technical one—for English language teachers. Scale development followed a two-phase design: item generation mapped to a single target construct and reviewed by seven experts (content validity, with CVI and modified kappa), a 30-person pilot for face validity, then a two-stage psychometric evaluation—exploratory factor analysis (EFA) on 165 preservice English language teachers in Indonesia and confirmatory factor analysis (CFA) on a separate sample of 227. The EFA confirmed a six-factor structure explaining 63.965% of variance (KMO = 0.906; Bartlett's test significant), and a refined CFA model retained a 27-item version of the scale with acceptable fit (χ²/df = 1.766, RMSEA = 0.058, SRMR = 0.054, TLI = 0.908, CFI = 0.919). The study bridges a theoretical and empirical gap by operationalizing a framework that foregrounds teacher agency, contextual adaptation, and ethical responsibility in AI-mediated language teaching.

Key Findings

• Six-factor structure. EFA (N = 165) recovered six distinct factors matching the ED-AI framework—Knowledge, Evaluation, Collaboration, Contextualization, Autonomy, and Ethics—accounting for 63.965% of total variance. A robustness check using principal axis factoring with promax rotation replicated the same structure, indicating stability independent of extraction method.
• Refined 27-item CFA model. The initial 30-item CFA underperformed (CFI = 0.893, TLI = 0.881). Three weakly loading or conceptually ambiguous items were removed on theoretical grounds—CON2 (supporting students with special needs, which overlapped with knowledge/autonomy), EVA5 (fairness in AI-based assessment, more central to ethics), and COL2 (general teamwork rather than human–AI collaboration)—and one residual covariance pair (ETH1–ETH2, privacy and ethical tool selection) was allowed. The final model reached acceptable fit (CFI = 0.919, TLI = 0.908, RMSEA = 0.058), with Knowledge, Autonomy, and Ethics retaining five items each and the other three dimensions four each.
• High reliability. All six dimensions showed Cronbach's α > 0.90 (Contextualization 0.949, Ethics 0.945, Collaboration 0.925, Evaluation 0.920, Knowledge 0.920, Autonomy 0.903), with composite reliabilities (rho_c) ranging 0.925–0.949. The authors interpret the high alphas as construct coherence rather than item redundancy, consistent with newly developed, theory-driven multidimensional measures.
• Convergent and discriminant validity. AVE values ranged from 0.454 to 0.538 (Ethics and Contextualization exceeding 0.50), with high CRs supporting adequate convergent validity. HTMT ratios (0.613–0.874) all fell below the 0.90 threshold, and bootstrapped 95% confidence intervals never exceeded 1.00, confirming empirically distinct dimensions.
• Concurrent criterion validity. TAILS subscales correlated positively and significantly (p < 0.01) with established AI self-efficacy (r = 0.476–0.661) and attitude-toward-AI measures (r = 0.491–0.690), and intercorrelations among dimensions (0.530–0.695) reflected large, theoretically coherent associations while preserving distinctiveness.
• Strong content validity. I-CVI for all 30 items ranged 0.86–1.00, S-CVI/Ave was 0.98, and modified kappa values (0.86–1.00) exceeded the 0.75 threshold, supplemented by qualitative expert Feedback that led to rewording six items for clarity without adding or deleting any.

Implications for Practice

• Diagnostic and evaluative use in teacher education. TAILS can be administered at program entry to diagnose preservice teachers' strengths and needs across the six dimensions, and evaluatively after AI-focused coursework to monitor learning gains and guide professional development and curriculum refinement.
• Data-informed program and policy design. Aggregated TAILS data support data-driven decision-making for embedding AI literacy within teacher education frameworks and for tracking trends across cohorts, institutions, or demographic groups, aligning with educational policy priorities.
• Scaffolding ethical competencies explicitly. The Ethics dimension's salience reinforces that ethical judgment does not emerge organically from AI use; programs should systematically and explicitly scaffold fairness, accountability, bias, and student privacy into preparation rather than assume it develops.
• Supporting balanced, human-centered integration. Findings underscore that AI should enrich rather than replace human interaction in communicative language learning, guiding teachers to weigh automation against authentic interaction and to contextualize tools for diverse learners, cultural settings, and subject-specific goals.

Connected Concepts

• AI Literacy
• Teacher AI Competency
• Professional Development
• Language Learning
• English Education (EAP / EFL / ESL)
• Educational Measurement
• AIEd in the Disciplines
• Ethics
• Self-Efficacy
• Learner Agency
• Workplace Learning
• Psychometrically Aware AI

Connected Articles

• Assessing AI-TPACK readiness in mathematics teacher education: The role of self-efficacy and teaching beliefs — Like Nabhan & Habók, develops and psychometrically validates a discipline-specific teacher AI competency instrument (AI-TPACK for mathematics), sharing the construct-development and EFA/CFA methodology.
• Unpacking ethics-domain of intelligent-TPACK scale in relation to in-service teachers' trust and distrust — Examines teacher trust and distrust toward AI alongside an ethics-oriented teacher competency lens, complementing TAILS' Ethics and Autonomy dimensions.
• Towards AI literacy: A proposal of a framework based on the Episodes of Situated Learning — Approaches teacher AI literacy from a situated, episode-based perspective, offering a complementary qualitative framing to TAILS' self-report measurement.

Citation

Language teachers’ AI literacy: A psychometric study based on the ED-AI framework — Nabhan, S., & Habók, A. (2026). Computers and Education: Artificial Intelligence, 10, 100583.