Conceptualizing and developing second language teachers' multimodal assessment literacy in the GenAI era

Synthesis: Jiang, Xin, Lam, Young and Lo conceptualize multimodal assessment literacy (MAL) for second language teachers as three dimensions — knowledge, value and competence — and then test the framework by running a year-long, Hong Kong Education Bureau-funded professional development program with 30 primary and secondary English teachers across six core schools. Multimodal assessment matters to the authors because GenAI has made the traditional language-centric essay increasingly gameable and because meaning-making itself has become multimodal; MAL is the literacy teachers need to design, implement and evaluate assessment tasks that elicit responses across linguistic, visual, aural, gestural and spatial modes. The program moved knowledge and competence with large effects — knowledge of strategies rose from 3.13 to 4.15 out of 5 (partial η² = 0.571) and competence in supporting students from 3.05 to 3.98 (0.482) — while the value subdimensions did not shift significantly, which the authors read as evidence that beliefs take longer to change than skill. Practice then pushed two additions into the framework: a disposition to keep multimodal assessment alive against competing curricular priorities, and the competence to mobilize GenAI critically across key learning stages.

Key Findings

1. Knowledge and competence moved; values did not. Four of five measured knowledge and competence subdomains rose significantly with large effects (partial η² from 0.427 to 0.571), while all three value subdomains stayed put under a Bonferroni-corrected alpha of .006.
2. The largest single gain was in strategies, not in beliefs. Knowledge of multimodal assessment strategies climbed from 3.13 to 3.79 to 4.15 out of 5 across the three survey points, a change of +1.02, the biggest in the study.
3. Competence in supporting students gained almost as much. Self-rated competence in offering support rose from 3.05 to 3.62 to 3.98 (+0.93), and competence in designing tasks from 3.11 to 3.91 (+0.80), both significant at p < .001.
4. Teachers began positive but not knowledgeable. All 30 valued multimodal assessment going in, yet most defined it as e-learning — recording speaking and submitting it to Google Classroom or Padlet — and rated their own guidance competence below the scale midpoint.
5. GenAI was familiar but almost never used for assessment. Teachers reported fluency with GenAI tools, used mostly for lesson planning; few had used GenAI for multimodal assessment, though all wanted to explore it.
6. The framework grew two subdimensions from practice. The refined MAL adds a disposition to sustain multimodal assessment against competing curricular priorities and a competence to mobilize GenAI critically across learning stages.
7. The instrument itself split one construct in two. A negative Cronbach's alpha on the subdomain about assessment principles forced the researchers to separate items about multimodal representation (α = .54) from items emphasizing linguistic modes (α = .70), evidence that the two commitments are empirically distinguishable.

The construct the study proposes

MAL is defined as the knowledge, value and competence a second language teacher needs to design, select, implement, evaluate and use high-quality multimodal assessment. The knowledge dimension covers the nature of multimodal assessment (tasks that elicit responses in several modes, digital or not), its purposes, and strategies for designing tasks with scaffolding and formative feedback. The value dimension covers belief about non-linguistic modes, the principles governing their use, and the roles teachers and students take. Competence covers task design and the support teachers give students while they compose. The authors build on prior assessment literacy frameworks and on multimodal composing research, and they insist the payoff is a shift from psychometric testing toward a multidimensional practice that includes affective and identity factors.

What the year-long program did

Thirty teachers — 13 secondary and 17 primary, from six core schools — worked through three phases: a needs analysis and 12 hours of workshops, a first cycle of task design and classroom implementation, a second cycle with eBook and video tasks, and then dissemination to ten network schools. Two cluster meetings and in-school co-planning created the collective space the authors consider decisive. Teachers named the six-step design framework as the practical anchor: 23 of them called it "handy," and others said co-planning closed "the distance between the concepts we learned in the workshop and the classroom implementation." Nearly all kept their positive stance and half planned to extend the work, often by bringing in teachers from other subjects.

GenAI as a catalyst the teachers had not yet used

The study's premise is that GenAI destabilizes language-centric assessment and simultaneously makes multimodal assessment more feasible, since models generate images, audio and video that students must select, arrange and evaluate. Yet before the program few teachers had used GenAI for assessment at all. The refined framework answers with a competence subdimension for critical and responsible GenAI use, including the capacity to help students detect, evaluate and counter-narrate biased model outputs — the authors warn that ignoring this dimension risks "automating and reproducing inequalities" in assessment. Their own critique of the program points the same way: more scaffolding on how GenAI prompts work would better prepare teachers for students' self-directed composing. The teachers themselves raised cross-curricular support, competing curricular priorities and uneven student readiness, including AI literacy, as the constraints that decide whether any of it survives the year.

What this means for practice

• Instructors. Design multimodal tasks that extend print literacy rather than replace it, and teach students to inspect and contest GenAI-generated visuals and text before those outputs enter their work.
• Faculty developers. Copy the program's mechanics rather than its content: a 12-hour workshop block, in-school co-planning, cluster meetings across schools, and a design framework teachers can hold in one page.
• Administrators. The binding constraint teachers named was curricular space, so treat cross-subject coordination — and time to negotiate it — as the resource that makes multimodal assessment sustainable.
• Researchers. Measure values separately from knowledge and competence, because this study found them on different clocks, and expect instrument splits when teachers hold both multimodal and linguistic commitments.

Limitations

• One context. All 30 participants were English teachers in Hong Kong primary and secondary schools, and the authors state that MAL in content classrooms may require a different emphasis on content understanding.
• Small sample with per-subdomain tests. Twenty-eight teachers gave complete responses across three time points, and the eight subdomains were tested with separate repeated-measures ANOVAs at a Bonferroni-corrected alpha of .006, so a real change can miss significance.
• Self-report and confidential data. Knowledge, value and competence were measured by teacher survey, supplemented by interviews, observation and artifacts; the authors state the underlying data are confidential and unavailable.
• One subdomain barely reliable. The two-item subdomain on assessment principles scored α = .54, which the authors describe as mathematically expected for a brief scale but which leaves that value weakly measured.
• Program design limits. The authors acknowledge the program under-scaffolded GenAI prompt knowledge, and the study reports no follow-up after the dissemination seminar that would show whether the gains held.

Citation

Jiang, L., Xin, J. J., Lam, P., Young, P. K., & Lo, Y. Y. (2026). Conceptualizing and developing second language teachers' multimodal assessment literacy in the GenAI era. Teaching and Teacher Education, 183, 105790.