Sensor and Gaze Analytics

Sensor and Gaze Analytics — the use of eye tracking, electroencephalography, electrodermal activity, heart-rate variability, and camera-derived posture to infer learner states such as attention, engagement, cognitive load, and affect. In AI in education these signals promise a continuous, non-verbal view of learning that quizzes and self-reports cannot supply. They also sit at the center of the field's hardest inference problem: a sensor records a body, not a mind, and every claim about what a learner thinks or feels is an interpretation layered on that recording (Kim and Han, 2026).

Questions to Consider

• A webcam can show that your eyes stayed inside a video player for a whole lecture. Does that mean you were paying attention?
• If a tutor could estimate your cognitive load in real time, should it act on that estimate without asking you first?
• A student's gaze may leave the screen because of a tic, a visual impairment, or a cultural norm. What does a "low attention" flag mean for that student?
• Should a university be allowed to collect brain and heart data for course improvement, and what would real consent require?
• A "75% engagement accuracy" figure sounds precise. Which layer of engagement did the sensor actually reach, and which did it miss?
• Proctoring now reads posture and head movement. Where is the line between detecting misconduct and judging a body?

Introduction

Sensor and gaze analytics describe a family of methods that turn physical measurements of a learner into educational inferences. The measurements include where the eyes rest and for how long, the electrical activity of the scalp, sweat-gland responses, beat-to-beat variation in heart rate, and the position and motion of the head and torso. The inferences include attention, engagement, cognitive load, confusion, stress, and, in assessment settings, suspicion.

The appeal is straightforward. Self-reports are retrospective and coarse, and quizzes sample learning at a few moments. Sensors run continuously and cost the learner little conscious effort. That combination has made sensing central to Learning Analytics and Learner Modeling and Adaptive Instruction, and it has produced a large literature in which the sensing is technically impressive and the interpretation is thin.

A systematic review of 23 studies pairing AI with Embodied Learning makes the imbalance concrete. Analysis and evaluation accounted for 40.0% of coded AI roles, and recognition and detection for 22.5%, while feedback and coaching took 15.0%. Sensing roles made up 62.5% of role tags against 35.0% for participatory ones (Kim and Han, 2026). The field is better at watching learners than at changing what happens next.

What each signal actually measures

Gaze and fixation. Eye trackers report where the eye points, how long it dwells, when it saccades, and how widely it disperses. Screen-based systems such as WebGazer estimate gaze from a webcam, while glasses track the eye directly. TutorLoop converts gaze into two quantities: attention as the percentage of gaze points inside the content area, and workload as gaze entropy, on the reasoning that dispersed gaze tracks cognitive load (TutorLoop, 2026). Ollie, a gaze-informed assistant for children, instead defines areas of interest in a picture and triggers narration when a child looks at one (Wu et al., 2026).

Electroencephalography. EEG records voltage fluctuations across the scalp at high temporal resolution. It is the only signal in this family that reaches cortical activity directly, which is why it attracts assessment research. Practical education systems use dry electrodes and few channels: E3Sense took four dry EEG channels at 125 Hz and analyzed two frontal ones (Anupkrishnan et al., 2026). Research-grade rigs take the opposite approach: Kosmyna et al. (2025) used 32-channel EEG to compare students writing with an LLM, a search engine, or no tool, and found the weakest neural connectivity in the LLM group (Kosmyna et al., 2025). That study is also the clearest warning about single-channel evidence, since its central claim rests on an optional crossover session that only 18 of 54 participants attended.

Electrodermal activity. EDA measures sweat-gland conductance, an index of sympathetic arousal. It responds to stress, effort, and emotional intensity, and it cannot distinguish among them. The same construct appears in wristbands and in forehead electrodes fixed a few centimeters apart by an eye-tracker frame (Anupkrishnan et al., 2026).

Heart rate and heart-rate variability. Heart rate tracks arousal, and variability between beats indexes autonomic recovery. In a year-long study of 458 students wearing Oura rings, emotional-exhaustion language tracked lower sleep efficiency and lower heart-rate variability (Affective Text and Wearable Sensing, 2026). Neither measure identifies a cause.

Camera-based posture and motion. A proctoring system can extract seventeen body landmarks per frame and derive posture deviation, head-movement trend, and upper-body motion (Arif et al., 2026). This measures movement. Calling it suspicious behavior is a separate, unsupported step.

These signals differ in sampling rate, noise, and instrumentation burden. E3Sense aligned twelve EEG, six eye, and three EDA windows into a shared 120-second representation with 166 dimensions (Anupkrishnan et al., 2026). EduGage ran seven device types at once and concluded that lightweight combinations carried most of the usable signal (Leng et al., 2026).

What the signals are used for

Engagement detection in video learning. E3Sense asked 30 university students to rate five segments per video, producing 450 ratings. A fused model reached 75.0% balanced within-one-level accuracy against a 63.0% mode baseline on held-out participants (Anupkrishnan et al., 2026). EduGage reported an MAE of 0.81 and 83.75% within-one accuracy across 16 participants and 715 probe windows (Leng et al., 2026). Both treat the estimate as approximate.

Attention estimation in classrooms. A multimodal transformer fused facial images with wearable signals on DIPSER, a naturalistic dataset of 57 subjects, and reached an MAE of 0.283. Its edge over a visual-only baseline was modest, and the authors conclude that facial features were the primary information source (Fragkiadakis et al., 2026).

Cognitive load and feedback timing. TutorLoop used gaze-derived attention and workload to choose among four feedback types with a reinforcement-learning agent. Across 187 participants the full system intervened less often than baselines yet improved attention and lowered workload (TutorLoop, 2026).

Proctoring and integrity. Pose-based proctoring reports near-ceiling performance on its own labeled scenario set: 99.56% accuracy and an AUC of 0.9957 (Arif et al., 2026). Wearable AI changes the same problem from the other side. Smart glasses remove the gaze drop and posture shift that invigilation relies on, a condition Corbin and colleagues call dual transparency (Corbin et al., 2026).

Affect and wellbeing. Physiological signals collected during exams predicted grade categories, with random forest reaching a mean of 0.96 across 30 replicates (Yamazaki and Rimal, 2026). The top predictors were electrodermal activity, accelerometer movement, and skin temperature. That is arousal and motion, not comprehension.

Assessment research. EEG has been tested as a direct measure of familiarity and of mental representation. Both lines are promising and both remain exploratory, which the next section takes up.

The inferential gap

Every use above depends on a chain: a physiological or behavioral event, a feature computed from it, a model trained on labels, and a claim about a mental state. Each link leaks.

The label is often the weakest link. E3Sense predicted learners' own segment ratings, and 63% of those ratings sat at the two engaged levels. A model can score well by learning the distribution rather than the state. When the researchers added learners' own definitions of engagement, accuracy rose from 64.6% to 71.5%, which suggests the labels and the signals were measuring partly different things (Anupkrishnan et al., 2026).

Validation can inflate the model. A benchmark of fifteen models for EEG-based familiarity prediction reached 0.9853 F1 under stratified cross-validation. Trial-independent Group K-Fold, which respects temporal structure, dropped the peak to 0.6038 F1 against a 25% chance level (Nanayakkara and Halloluwa, 2026). The first number describes leakage; the second describes learning.

A neural correlate is not a score. Reconstructed images from EEG correlated with test performance at R² = 0.79 across 55 participant-video observations. Correctly answered items also showed higher semantic similarity than incorrect ones. Yet one participant produced high-quality reconstructions and answered incorrectly, so reconstruction quality cannot be read as correctness (Su et al., 2026).

Attention is not learning. In TutorLoop, attention correlated with learning at r = 0.017, while workload correlated negatively at r = -0.168 (TutorLoop, 2026). A system that maximized the attention signal would have optimized something close to noise.

A physiological profile is not a diagnosis. Exam stress models predict a grade category from arousal and movement, and the authors insist the output belongs beside personal circumstances and mental-health context rather than in a high-stakes decision (Yamazaki and Rimal, 2026).

Finally, collection is not integration. Kim and Han found that most multimodal studies gathered streams in parallel, one aligned facial affect, gaze, and face detection after the fact, and none fused its inputs live while a learner used the system (Kim and Han, 2026). A page of synchronized channels can look like a model of the learner when it is only a recording.

Ethics of sensing learners

Consent is weakened by institutional power. A student who must attend class or sit an exam cannot freely refuse a camera or a wearable. Demographic attributes may not even be volunteered: DIPSER's gender, age, and ethnicity labels were inferred from imagery by computer-vision models rather than supplied by participants (Fragkiadakis et al., 2026). Inferring who someone is, in order to test whether a model treats them fairly, is itself a profiling act.

Biometric data is not ordinary analytics data. EEG, EDA, and heart-rate variability are health-adjacent and, in many jurisdictions, special-category data. They can reveal sleep, stress, and medical conditions beyond anything a course needs. A gaze-informed system for children drew a privacy objection from a parent even in a small pilot (Wu et al., 2026).

Function creep is the normal path. Wellbeing monitoring becomes risk flagging, and risk flagging becomes grading evidence. Corbin and colleagues show the assessment version of this drift: once physical exclusion fails, inspection moves from artifacts to bodies, and students must prove their glasses are corrective and their devices therapeutic (Corbin et al., 2026).

Fairness failures cluster on bodies that differ. Face-based attention estimation depends on skin tone, eyewear, and lighting. Gaze-based attention penalizes learners who avoid eye contact, including some autistic and neurodivergent students. Pose-based proctoring flags tics, stimming, and cultural movement norms. The proctoring study does not test whether flagged posture corresponds to misconduct, or whether the detector treats students with disabilities or cultural differences in gaze fairly (Arif et al., 2026). Regulation is not a fix here; it is a way of laundering the error into a rule.

Fairness claims need subject-level evidence. A gender-targeted regularizer cut the validation MAE gap from 0.02 to 0.005, then increased both the gap and worst-group error on held-out subjects (Fragkiadakis et al., 2026). Reporting a single split as proof of fairness is a design failure with an equity cost.

Privacy can be designed in. TutorLoop ran WebGazer client-side so raw gaze never left the browser (TutorLoop, 2026). That choice shows that some of the cost is architectural rather than inherent. The harder question is whether the inference should exist at all.

Common Misconceptions

"Eye tracking tells you where attention is." It tells you where the eyes point. Gaze can rest on a slide while the learner thinks about something else, and it can leave the screen for reasons unrelated to attention. TutorLoop's attention measure correlated with learning at r = 0.017, which is close to no relationship at all (TutorLoop, 2026).

"A physiological signal is a direct readout of a mental state." EDA indexes sympathetic arousal, and heart-rate variability indexes autonomic recovery. Neither distinguishes stress from effort from excitement. A model that predicts exam grades from EDA, movement, and skin temperature has learned the body's arousal signature, not the student's understanding (Yamazaki and Rimal, 2026).

"High classification accuracy means the system detects the state." Accuracy depends on the labels, the split, and the base rate. EEG familiarity prediction fell from 0.9853 to 0.6038 F1 when validation respected trial boundaries (Nanayakkara and Halloluwa, 2026). Proctoring's 99.56% describes a bespoke labeled scenario set, not live examinations (Arif et al., 2026).

"Engagement is one thing, so one score can represent it." The construct has three strands, and a sensor reaches mostly the first. E3Sense found learners defined it through cognitive, behavioral, and emotional indicators in different proportions, and those definitions changed prediction (Anupkrishnan et al., 2026). A single number hides which layer was measured.

"If the model is fair on the validation split, it is fair in the classroom." A regularizer that equalized validation error across gender groups made the held-out gap worse (Fragkiadakis et al., 2026). Fairness is a property of a deployment, not of a checkpoint.

Connected Concepts

• Learner Modeling and Adaptive Instruction
• Learning Analytics
• Multimodal AI
• Affective Computing
• Student Engagement
• Self-Regulated Learning
• Embodied Learning
• Educational Measurement
• Automated Assessment
• Assessment Validity
• Remote Proctoring
• Academic Integrity
• Privacy
• Bias Mitigation
• Neurodiversity
• Equity
• Accessibility
• Machine Learning
• Cognitive Psychology
• Intelligent Tutoring

Connected Articles

• Gaze-Informed Proactive AI Assistance for Children’s Picture Exploration — Gaze-triggered LLM narration held children on a narrated region 6.48 s versus 1.67 s, but the study measured attention rather than comprehension (Wu et al., 2026)
• E3Sense: Head-Confined Multimodal Sensing of Learner Engagement — Head-mounted EEG, eye tracking, and forehead EDA predicted self-rated engagement at 75.0% within-one accuracy against a 63.0% baseline (Anupkrishnan et al., 2026)
• EduGage: Methods and Dataset for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning — EduGage fused seven wearable and camera streams into segment-level engagement estimates (MAE 0.81) and argues for lighter instrumentation (Leng et al., 2026)
• Fairness-Aware Multimodal Transformer Modeling for Real-Time Student Attention Estimation — A multimodal attention model reached MAE 0.283, and its validation fairness gains reversed on held-out subjects (Fragkiadakis et al., 2026)
• On AI glasses and wearable AI in assessment — Wearable AI is dually transparent, so physical exclusion fails and enforcement drifts toward adjudicating students' bodies (Corbin et al., 2026)
• Automating Learner Assessment: Benchmarking Machine Learning and Deep Learning Models for EEG-Based Familiarity Prediction — Fifteen models for EEG familiarity prediction fell from 0.9853 to 0.6038 F1 under trial-independent validation (Nanayakkara and Halloluwa, 2026)
• Seeing what students learn: assessing cognitive representations and academic outcomes via EEG image reconstruction — EEG-reconstructed images tracked test performance at R² = 0.79, with one high-reconstruction participant answering incorrectly (Su et al., 2026)
• Leveraging Physiological Signals to Predict Exam Outcomes with Machine Learning — Random forest predicted exam grade categories from EDA, movement, and skin temperature, signals that index arousal rather than understanding (Yamazaki and Rimal, 2026)
• A formative study of brief affective text as a complement to wearable sensing for longitudinal student health monitoring — In 458 Oura-wearing students, emotional register rather than concern topic tracked sleep and heart-rate variability (2026)
• TutorLoop: Regulating Student Learning Behaviors via Sensor-in-the-Loop Generative Feedback — Webcam gaze drove reinforcement-learning feedback for 187 learners; attention correlated with learning at r = 0.017 while workload correlated negatively (2026)
• An automated online proctoring system with live suspicious activity detection using YOLOv11 pose estimation and multi-model classification — YOLOv11 pose features flagged suspicious behavior at 99.56% on a bespoke labeled set, without testing misconduct or fairness (Arif et al., 2026)
• Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task — EEG connectivity scaled down with external writing support across 54 participants, framing reliance as accumulated cognitive debt (Kosmyna et al., 2025)
• Artificial intelligence in embodied learning: A systematic literature review — Across 23 studies, sensing roles dominated at 62.5% of tags and no reviewed system fused multimodal streams live (Kim and Han, 2026)