SAVVY: Student Attention Visualization for Video-based Learning Analysis

Synthesis: SAVVY is an interactive visual analytics system for video-based learning that integrates visual and auditory attention signals from multimodal brain data to support top-down exploration of student attention variation across instructional videos, connecting AI-based attention estimation to instructional-design practice.

Key Findings

SAVVY is an interactive visual analytics system for video-based learning that integrates visual and auditory attention signals from multimodal brain data to support top-down exploration of student attention variation across instructional videos.

• A novel attention modeling framework based on multimodal brain signals enables stable tracking of student attention in real-world environments, addressing the noise susceptibility of existing attention quantification algorithms.
• The system supports teachers in analyzing pilot cohorts' attention before releasing videos, reducing the guesswork of empirical revision by making attention patterns interpretable at scale.
• The work connects AI-based attention estimation to instructional design practice, giving teachers an evidence base for when and where videos lose student engagement — a contribution to Learning Analytics and multimodal learning analytics in education.

What this means for practice

• Instructors. Check pilot-cohort attention data before releasing an instructional video, so that revisions target the segments where attention actually drops rather than relying on subjective impressions of pacing and slides.
• Instructors. Read attention against the material: because the system links visual and auditory attention to information density and slide transitions, it can show whether a near-blank or low-density slide is worth keeping (in the reported case, the expert shortened time on a near-blank slide rather than removing content).
• Designers. Structure the tool for top-down exploration — course concept, then group, then individual trajectory — because a single aggregate heatmap or stacked-area view obscures the channel comparisons and level transitions that drive a revision decision.
• Designers. Reserve multimodal brain sensing (EEG plus fNIRS) for settings that need stable real-world tracking: the combined model outperformed single-modality variants in the reported evaluation, and the ablation confirms each modality contributes.

Limitations

• The attention model was trained on 31 healthy participants (16 male, 15 female; mean age 23.6) watching CNN and matrix-factorization videos, a young, well-rested, inexperienced-with-the-content sample that is not representative of learners generally.
• Information density is computed from the slides and narration as presented, so it reflects what the material delivers rather than what each learner actually processes — a point the study's own expert reviewers raised.
• The audiovisual decomposition does not measure top-down attention directly: it supplies surrogate indicators biased toward top-down attentional allocation, and the system identifies problems without generating specific optimization recommendations.
• Scalability is stated as a limitation: the visualization degrades as the number of concepts grows with video duration and the Individual Module becomes crowded with larger student samples, with path grouping and scrollbar interaction only partially alleviating this.

Citation

Zhou, S., Shen, M., Wen, X., Qiu, Z., Jiang, Y., Wu, X., Wu, F., Wang, Y., & Zhou, Z. (2026). SAVVY: Student attention visualization for video-based learning analysis.