📄 Research Article
Multimodal Learning with Generative AI
The guide adopts a middle way between "techno-fixing" and rejecting AI as an existential threat. It argues that:
A comprehensive educator's guide to integrating Generative AI into multimodal teaching, learning, and assessment across higher education. Built on Kress's social semiotic theory of multimodality, the guide positions GenAI as a 'cyber-social' partner that complements—but cannot replace—human meaning-making. It proposes the MMLD-AI unifying model (UDL + ABC Learning Design) and the Dual-Track Cyber-Social Learning Model for designing effective human-AI collaboration.^varga-atkins-educators-guide-multimodal-learning-genai-2025
Core Position: Pragmatic, Not Uncritical
The guide adopts a middle way between "techno-fixing" and rejecting AI as an existential threat. It argues that:
The Four Costs of GenAI
| Cost Domain | Key Concern | Educational Response |
|---|---|---|
| Individual | Privacy, data protection, equity of access, mental health, over-reliance | Transparent authorship; approved tools list; scaffolded critical engagement |
| Environment | Image/video/audio generation uses significantly more energy than text | Mindful use; limit iterations; group demonstrations; digital decluttering |
| Knowledge | Removes sourcing process integral to retention; short-term gains may displace deep learning | Students clarify own understanding before consulting AI; metacognitive scaffolding |
| Future Jobs | Entry-level white-collar roles vulnerable to automation | Focus on human strengths: contextual reasoning, ethical judgement, craftsmanship |
AI Literacy in Multimodal Contexts
Three Levels
1. Basic literacy — Awareness of multimodal GenAI platforms, capabilities, and appropriate uses (creating prompts, generating visual outputs)
2. Intermediate literacy — Co-create multimodal content, critically evaluate AI outputs, scaffold uses (transform lecture notes into visuals or podcasts)
3. Advanced literacy — Design activities/assessments incorporating multimodal GenAI; lead ethical and philosophical discussions
Four Implementation Scales
| Scale | Strategies |
|---|---|
| Individual | Workshops on creative multimodal tasks; prompt crafting practice; reflective assignments documenting AI use |
| Module | Embed GenAI literacy into learning outcomes; optional multimodal tasks with clear rubrics; creative/reflective critique components |
| Programme | Cross-module policies; consistency and transparency via workshops and discussion; alignment with graduate attributes (criticality, creativity, digital fluency) |
| Institutional | Clear policies with checklists; vetted tools; data privacy protocols enforced; avoid rigid mandates in favor of flexible guidance |
The MMLD-AI Unifying Model
The Multimodal Learning Design with GenAI model merges:
Six Multimodal Engagement Types (adapted from ABC)
1. Acquisition of information
2. Investigation and/or research
3. Collaboration with others
4. Production of artefacts (learning, teaching, or assessment)
5. Practice of approaches/theories/principles/skills
6. Discussion/discourse, including critique/evaluation
For each engagement type, educators decide on multimodal affordances of GenAI and explore respective cyber-social strengths.
The Dual-Track Cyber-Social Learning Model (Galla et al., 2025)
This complementary model maps human vs. AI strengths across Bloom's taxonomy processes:
| Process | Human Strengths | AI Strengths | Cyber-Social Approach |
|---|---|---|---|
| Knowledge & framing | Contextual understanding; embodied knowledge; critical verification | Rapid data retrieval; pattern recognition; broad topical coverage | Humans define purpose and frame problems; AI generates background data and inspiration; humans filter and verify |
| Interpretation & analysis | Causal reasoning; cultural/ethical awareness; implicit meaning | Correlation analysis; theme identification; feature extraction at scale | AI identifies statistical patterns; humans determine causality, relevance, deeper significance |
| Application & prototyping | Situated judgement; adaptive problem-solving; ethical decision-making; craftsmanship | Rapid simulation; consistent rule application; code/digital artefact generation | AI generates digital prototypes; humans adapt for real-world complexity, apply physical craft, ensure ethics |
| Synthesis & creation | Novel conceptual blending; purpose-driven integration | Cross-domain pattern integration; combinatorial exploration | AI explores possible combinations; humans evaluate, refine, and integrate meaningfully |
Practical Integration: Three Strands
Teaching (Educator-Created Content)
Learning (Student-Created Content)
Assessment and Feedback
Relationship to Existing Research
| Guide Principle | Wiki Connection |
|---|---|
| Cyber-social partnership (complementary strengths) | Principled AI Education — "AI must augment, not displace" aligns perfectly |
| Four costs framework (individual, environment, knowledge, jobs) | AI Tutor Safety Harms — Costs to knowledge overlap with cognitive offloading; environmental costs are a new dimension |
| AI literacy levels and scales | AI Literacy — ICAP framework; collaborative learning; this guide adds institutional scaling and multimodal specificity |
| MMLD-AI model (UDL + ABC + six engagement types) | Adaptive Learning — Multi-resolution personalization; Agentic Workflows Education — Planning and reflection paradigms |
| Dual-Track Cyber-Social Model | Pedagogical LLM Training — Reward "guiding" over "answering"; Human In The Loop AI — Human verification of AI outputs |
| Multimodal assessment redesign | Authentic Assessment — Six-dimensional framework; Formative Assessment — AI-generated feedback with human validation |
| Scaffolding and metacognition | Self Regulated Learning — UDL's emphasis on student agency; Metacognition — Cyber-social metacognitive awareness |
| Faculty development across four scales | Faculty Development — CTL pragmatic transition model; this guide adds module-level and programme-level strategies |
Case Study Themes from the Guide
The guide includes 15+ educator case studies spanning:
Open Questions
1. Environmental cost awareness: How can educators and students make informed trade-offs between the pedagogical value of multimodal GenAI artefacts and their energy costs?
2. Transfer across modalities: Does competence in AI-assisted multimodal creation in one domain (e.g., visual design) transfer to another (e.g., audio production)?
3. Assessment validity: When students use GenAI to create multimodal assessment artefacts, how can assessors distinguish genuine human meaning-making from AI-generated polish?
4. Scaling the MMLD-AI model: Can the six engagement types be operationalized as automatic learning design recommendations, or does human pedagogical judgment remain essential?
Connected Concepts
Connected Articles
Citation
AI, G., Original, T., Multimodal, A.E.G.T., AI, L.A.G., Investigators:, P., Varga-Atkins, T., Saunders, S., & Hallam, S.B.S. (2026). Multimodal Learning with Generative AI