Research Article
Making learning visible in the age of generative AI: reported assessment practices for cognitive engagement and self-regulated learning, and the development and external appraisal of a higher education assessment framework
Synthesis: Lasekan, Ubilla and Mathew (2026) asked what higher education educators publicly say they are doing about GenAI and assessment, and whether the reasoning they claim to want is actually made inspectable. They collected posts and comments from Reddit plus Hacker News, Stack Exchange, Lemmy and Mastodon covering December 2022 to October 2026, reducing 29,355 unique records to 5,091 that named at least one of 11 practices, then built a seven-component framework and appraised it against 40 external documents. Security dominated: detection and sanction appeared in 43.2% of records and supervised in-class tasks in 28.5%, while planning, monitoring and reflection vocabulary appeared together in only 2.2%. A masking check showed much of the metacognitive language was definitional, with reflection vocabulary falling from 94.2% to 5.5% once shared words were removed. The framework described all 40 documents (mean fit 3.9/5), but their emphasis on authentic design and student co-design forced an eighth component, and no practice was described as eliciting both engagement and a regulatory cycle.
Key Findings
- Security dominates reported practice. Detection, surveillance and sanction appeared in 43.2% of the 5,091 records and supervised in-class tasks in 28.5%, against 10.5% for oral verification and 6.4% for reflection.
- Emphasis is shifting toward process designs. Detection's share of practice-coded records fell from 56.4% in 2023 to 37.6% in 2026, while oral verification (6.6% to 13.9%) and process evidence (7.1% to 12.0%) roughly doubled.
- Implementation is reported more often than assumed. Of all records, 31.0% described at least one implemented practice and 35.4% only proposals, contradicting the earlier claim that proposals outnumbered implementations two to one.
- Connected regulation is almost never described. Planning, monitoring and reflection or revision vocabulary appeared together in just 2.2% of records, so no single practice was described as eliciting a connected self-regulatory cycle.
- Much mechanism language is definitional. Reflection words accompanied 94.2% of reflective-task records but 5.5% once shared practice words were masked; monitoring fell from 64.2% to 39.7% and revision from 80.0% to 46.0%.
- External documents invert the priorities. All 40 appraisal documents fitted the framework (mean 3.9/5), but authentic task design was central in 62% of implemented cases against 8% for secured checkpoints.
What the corpus can and cannot show
The evidence base is public argument, not classroom data. Records came from Reddit's Arctic Shift archive plus Hacker News, Stack Exchange, Lemmy and Mastodon, covering December 2022 to October 2026; of 29,355 unique records, 13,903 were eligible and 5,091 named at least one practice. Reddit supplied 75.2%, and 94.3% of those came from r/Professors, a predominantly North American community. The authors are candid that this identifies reported practice and stated constraint, not prevalence or effect: implementations are unverified, identities rest on flair and self-description, and discipline was identifiable for 20.4% of records. Coding used dictionaries validated against an independent model-based coder on a hold-out sample (κ = 0.49 for any practice; κ = 0.69 for implemented versus proposed), and two practices with unreliable estimates were reported but not interpreted.
Where the learning vocabulary thinned
Reasoning and explanation was the intended process named most often, from 25% of supervised-task records to 44% of critical-AI records. Beyond that the picture thins. Cognitive engagement and SRL terms co-occurred in 20.3% of records, but planning, monitoring and reflection or revision appeared together in only 2.2%, which the authors read as phase mentions rather than a connected regulatory cycle. A masking check removed the words that had triggered each record's practice code and re-detected mechanism terms: reflection vocabulary fell from 94.2% to 5.5% of reflective-task records, monitoring from 64.2% to 39.7%, and revision from 80.0% to 46.0%. Oral verification was the exception, holding reasoning at 42.5% down only to 35.8%. Learning-oriented records still described more monitoring (9.1 percentage points) and reasoning (6.2) than detection-only records after masking.
From seven components to two lanes
The seven-component framework groups the 11 practices by the evidence they are meant to yield, from dialogic and social verification (17.7% of records) to secured performance checkpoints (28.5%). Detection sits outside because it yields a probabilistic signal, not evidence of capability, and only 291 records (5.7%) combined a secured or dialogic checkpoint with a process or reflection layer. Frozen before appraisal, the framework was applied blind to 40 separately sampled documents and described all of them (mean fit 3.9/5). The omissions were more informative: authentic task design was central in 62% of implemented cases and guidance, secured checkpoints in 8% and 15%, and detection was absent from 24 documents. Gap analysis raised an eighth component, student co-design and transparency (12 documents), alongside instructor-side AI use (10) and multimodal format choice (9). What emerged is a two-lane architecture: a secured lane carrying bounded demonstrations of independent capability, and an open learning lane permitting AI while requiring explanation, traceable process, revision, and justified AI decisions.
What this means for practice
- Instructors. Pair every secured demonstration with a process requirement: detection and supervised-task records described monitoring and reasoning least often, so supervision answers attribution, not understanding.
- Instructors. Require students to justify and critique AI contributions wherever AI is permitted, because disclosure and process artifacts alone read as compliance activities in the reported practice.
- Learning designers. Name the capability claim first and the evidence that would support it second, then combine a reasoning demonstration with proportionate process evidence.
- Administrators. Budget for the constraint each practice carries: orals need scale and workload planning, supervised tasks need accommodation and logistics, and detector output should trigger dialogue rather than stand as proof.
Limitations
- The corpus is Reddit-dominated and archive-dependent: 94.3% of Reddit records came from r/Professors, a predominantly North American community, and platform composition shifted across years, so trends partly reflect sampling.
- Coding reliability was moderate at best — hold-out κ = 0.49 for any practice, 0.23 for feedback and revision cycles and −0.02 for critical AI-use tasks — and an audit found a practice in 11 of 40 sampled uncoded records (27.5%), implying roughly 1,400–3,800 missed practices.
- No human double-coding was undertaken; agreement was estimated against an independent model-based coder on 150 hold-out records, presented as a check on the dictionaries rather than a substitute for inter-rater reliability.
- Mechanism signals are partly lexical and record stated intentions, not measured cognition; no student learning, transfer or SRL process was measured.
- The 40 appraisal documents were purposively sampled, English-language, rated by a single blind reader, and five were segments of one compendium; because they refined the framework, they cannot also validate it.
Citation
Lasekan, O. A., Ubilla, L., & Mathew, B. S. (2026). Making learning visible in the age of generative AI: reported assessment practices for cognitive engagement and self-regulated learning, and the development and external appraisal of a higher education assessment framework. Frontiers in Psychology, 17, 2022639.