Language teachers’ AI literacy: A psychometric study based on the ED-AI framework

Synthesis: Nabhan and Habók develop and validate the Teachers' AI Literacy Scale (TAILS), a self-report instrument grounded in the ED-AI literacy framework (Allen & Kendeou, 2024), which conceptualizes AI Literacy across six dimensions: knowledge, evaluation, collaboration, contextualization, autonomy, and ethics. Arguing that most existing instruments target students, general users, or nonexperts, the authors position TAILS as a discipline-specific measure that reframes AI literacy as a pedagogical and ethical capacity—rather than a purely technical one—for English language teachers. Scale development followed a two-phase design: item generation mapped to a single target construct and reviewed by seven experts (content validity, with CVI and modified kappa), a 30-person pilot for face validity, then a two-stage psychometric evaluation—exploratory factor analysis (EFA) on 165 preservice English language teachers in Indonesia and confirmatory factor analysis (CFA) on a separate sample of 227. The EFA confirmed a six-factor structure explaining 63.965% of variance (KMO = 0.906; Bartlett's test significant), and a refined CFA model retained a 27-item version of the scale with acceptable fit (χ²/df = 1.766, RMSEA = 0.058, SRMR = 0.054, TLI = 0.908, CFI = 0.919). The study bridges a theoretical and empirical gap by operationalizing a framework that foregrounds teacher agency, contextual adaptation, and ethical responsibility in AI-mediated language teaching.

Key Findings

• Six-factor structure. EFA (N = 165) recovered six distinct factors matching the ED-AI framework—Knowledge, Evaluation, Collaboration, Contextualization, Autonomy, and Ethics—accounting for 63.965% of total variance. A robustness check using principal axis factoring with promax rotation replicated the same structure, indicating stability independent of extraction method.
• Refined 27-item CFA model. The initial 30-item CFA underperformed (CFI = 0.893, TLI = 0.881). Three weakly loading or conceptually ambiguous items were removed on theoretical grounds—CON2 (supporting students with special needs, which overlapped with knowledge/autonomy), EVA5 (fairness in AI-based assessment, more central to ethics), and COL2 (general teamwork rather than human–AI collaboration)—and one residual covariance pair (ETH1–ETH2, privacy and ethical tool selection) was allowed. The final model reached acceptable fit (CFI = 0.919, TLI = 0.908, RMSEA = 0.058), with Knowledge, Autonomy, and Ethics retaining five items each and the other three dimensions four each.
• High reliability. All six dimensions showed Cronbach's α > 0.90 (Contextualization 0.949, Ethics 0.945, Collaboration 0.925, Evaluation 0.920, Knowledge 0.920, Autonomy 0.903), with composite reliabilities (rho_c) ranging 0.925–0.949. The authors interpret the high alphas as construct coherence rather than item redundancy, consistent with newly developed, theory-driven multidimensional measures.
• Convergent and discriminant validity. AVE values ranged from 0.454 to 0.538 (Ethics and Contextualization exceeding 0.50), with high CRs supporting adequate convergent validity. HTMT ratios (0.613–0.874) all fell below the 0.90 threshold, and bootstrapped 95% confidence intervals never exceeded 1.00, confirming empirically distinct dimensions.
• Concurrent criterion validity. TAILS subscales correlated positively and significantly (p < 0.01) with established AI self-efficacy (r = 0.476–0.661) and attitude-toward-AI measures (r = 0.491–0.690), and intercorrelations among dimensions (0.530–0.695) reflected large, theoretically coherent associations while preserving distinctiveness.
• Strong content validity. I-CVI for all 30 items ranged 0.86–1.00, S-CVI/Ave was 0.98, and modified kappa values (0.86–1.00) exceeded the 0.75 threshold, supplemented by qualitative expert Feedback that led to rewording six items for clarity without adding or deleting any.

What this means for practice

• Teacher educators. Administer TAILS at program entry to diagnose preservice teachers' strengths and needs across the six ED-AI dimensions, and again after AI-focused coursework to monitor gains and guide professional development and curriculum refinement.
• Teacher educators. Scaffold the Ethics dimension explicitly rather than assuming it develops from AI use: ethical judgment operated as a distinct factor (α = 0.945) alongside Knowledge and Autonomy, each retaining five items.
• Assessment professionals. Treat the six dimensions as separate scales — HTMT ratios of 0.613–0.874 stayed below the 0.90 threshold and intercorrelations of 0.530–0.695 supported discrimination — rather than collapsing them into one AI literacy score.
• Administrators. Use aggregated TAILS data for embedding AI Literacy within Teacher Education frameworks and for tracking trends across cohorts, institutions, or demographic groups.
• Instructors. Keep AI as an enrichment of communicative language learning: weigh automation against authentic interaction and contextualize tools for diverse learners, cultural settings, and subject-specific goals.

Limitations

• Validation rests on 422 preservice English language teachers in Indonesia (30 in the pilot, 165 in EFA, 227 in CFA), recruited by convenience sampling, so the sample is single-country and preservice only.
• All data are self-report and, as the authors state, may be subject to social desirability bias and capture respondents' self-perceptions of AI Literacy rather than observed instructional practices.
• No marker-variable test was conducted to quantify potential common method variance, and the study assessed neither predictive validity nor test–retest stability; the authors call for in-service validation and longitudinal designs.
• Convergent validity is partly dependent on composite reliability, since AVE values ranged from 0.454 to 0.538 and only Ethics and Contextualization exceeded 0.50, while Cronbach's alphas above 0.90 required the authors to argue construct coherence rather than item redundancy.

Citation

Nabhan, S., & Habók, A. (2026). Language teachers’ AI literacy: A psychometric study based on the ED-AI framework. Computers and Education: Artificial Intelligence, 10, 100583.