Research Article
AI-interaction literacy: reflections on how generative AI might be used to support self-regulated learning in higher education
Synthesis: AI-interaction literacy — Brunnström and Palmqvist (2026) run an eight-round demonstration in which they take the role of a naive student using a GenAI chatbot on a take-home examination question ("Do machines think?"), then analyze the interaction itself with the SOLO taxonomy rather than grading the AI's answer. The default output stayed at the quantitative, multistructural phase — "polished but pedagogically thin" text that a novice could submit without understanding — and reaching the qualitative phase required repeated meta-level interventions from the learner ("simplify", "this is overwhelming, can you condense it?"). Their central claim: productive GenAI use demanded the very self-regulatory skills the tool was expected to support, so the interaction itself became an object of AI Regulation in Education. The paper names this capacity AI-interaction literacy — the ability to steer, evaluate, and learn from iterative interaction with GenAI — and proposes that take-home assessments emphasize the qualitative phase while recall-based knowledge moves to in-class formats.
Overview
Take-home examinations are common in the humanities and social sciences and are argued to support more complex reasoning than time-constrained formats. If a chatbot can produce a plausible response, however, the submitted product becomes a weaker indicator of what the student has learned — an assessment validity problem rather than only an integrity problem. The authors situate their work within calls to move past rules about permitted and prohibited use: GenAI and assessment has been described as a wicked problem with no single stable solution, and proposals such as Black Box Assessment argue for process-oriented assessment that attends to the learning trajectory, not just the polished artifact.
The paper's frame is that GenAI may support each phase of self-regulated learning — goal setting, strategy selection, monitoring, and adjustment — but that the value of an AI response depends on the student's capacity to interpret, evaluate, and use it. Because GenAI feedback arrives interactively and without a teacher's immediate framing, it also raises the bar for feedback literacy.
The demonstration
Two authors shouldered the role of a naive student aiming to use GenAI as study support. They prompted Claude Opus 4.6 (Anthropic) on 22 April 2026 with the original exam question, word limit (1000–1200) and course literature restrictions; ChatGPT 5.4 Thinking and Pro produced comparable but more extensive responses (not reported). Each prompt–response exchange was then coded for the structural complexity of the AI's output with the SOLO taxonomy, moving from the quantitative phase (listing, organizing) toward the qualitative phase (relating, evaluating, generalizing).
| Step | Prompt intent | AI output | SOLO level | Observation |
|---|---|---|---|---|
| 1 | Original exam question with word limit and literature | Polished, essay-ready answer naming concepts (systems reply, robot reply) without explaining them | Multistructural | Ready-to-submit text, pedagogically thin; assumes prior knowledge |
| 2 | "Can you define clear learning objectives?" | Eight broad conceptual objectives | Multistructural | Overwhelming breadth; no obvious starting point for a novice |
| 3 | "Use you as a conversational partner… how can you assist me?" | List of learning modes (explanation, questioning, argumentation practice) | Multistructural / early relational | Productive shift from "what" to "how", but still list-based |
| 4 | Selected argumentation practice; stated a claim about the Turing Test | Sophisticated objection using Janlert's computation–thinking distinction | Relational | High-quality argumentation, inaccessible without simplification |
| 5 | "Present this argument in less complex language" | Simplified reformulation with everyday analogies | Relational | Simplification came only after a learner meta-intervention |
| 6 | "What do I need to do to learn enough to answer this objection?" | Detailed study roadmap (functionalism, intentionality, Chinese Room replies) | Relational | Useful but comprehensive; learner must manage the load |
| 7 | Asked GenAI to apply a self-regulated learning process | Very long structured plan based on SRL theory | Relational | Invoking SRL theory did not by itself produce approachable output |
| 8 | "This is overwhelming, can you condense it?" | Three-step loop (set one small goal; learn and practice; reflect) plus "small steps build confidence" | Relational / extended abstract | Most pedagogically effective response — after seven rounds of calibration |
Findings
- Default output is fluent but pedagogically thin. The first response met the assignment criteria and would likely pass, yet it functioned as "a collection of factoids dressed in fluent academic prose" — evidence that a fluent product indicates neither learning nor its absence.
- Stagnation came from excess, not scarcity. Repeated requests for more content produced more comprehensive lists pitched above the learner's existing schema, keeping the interaction in the quantitative phase.
- Progress required meta-level intervention aimed at the tool. Reaching a usable learning loop took eight rounds; the learner had to signal overload, request simplification, narrow scope, and redirect toward focused practice — what the authors call metacognitive labor directed at the GenAI interaction itself.
- The tool presupposed the competence it was meant to build. "Arriving at that point… required us to possess, in advance, the self-regulatory skills we hoped the tool would support." At moments they judged that simply reading the course book would have been cognitively simpler and pedagogically clearer.
- Simple refusal to persist is part of the skill. AI-interaction literacy includes recognizing when an explanation is too abstract and deciding when to disengage from the tool — a decision the paper treats as legitimate rather than as failure.
- Metacognitive laziness is the failure mode on the other side. Where delegation of planning, monitoring, and evaluation to the system produces metacognitive laziness, this demonstration shows the opposite risk: unmanaged use of a legitimate learning tool adds a second object of regulation on top of the disciplinary content.
The equity implication
The authors expected GenAI might serve as a personalized tutor for students without informal academic support networks and reach the opposite conclusion. Because unguided use imposes an interaction-management skill that is unevenly distributed, GenAI "may be most beneficial to already advantaged students" — those with strong study habits, high motivation, and confidence in directing an AI system — while students with weaker study skills or lower academic self-efficacy meet complexity and frustration. Rather than substituting for missing conversation partners, the tool introduces a new competence whose acquisition creates its own gap. The stated remedy is that productive GenAI use is a learned competence, not an intuitive one, and the responsibility for teaching it cannot rest with the student alone.
Propositions
- For students: engage the course literature and treat the system as a conversational partner for self-regulated learning rather than a generator of ready-made answers; productive use requires informed, iterative prompting, requests for simplification, and Reflection against one's own developing understanding.
- For teachers: develop an explicit strategy for teaching GenAI-supported self-regulated learning, with particular attention to students with lower academic self-efficacy, including how to formulate productive prompts, recognize when a response is too abstract or broad, and structure a session around small goals.
- For assessing outcomes: because GenAI is powerful at quantitative, factoid-type content, take-home assessments should emphasize the qualitative phase of the SOLO taxonomy, while quantitative recall-based knowledge is better assessed in class where GenAI is unavailable.
What this means for practice
- Instructors. Teach the interaction, not just the tool: run explicit sessions on formulating prompts, recognizing when an output is too abstract or broad, and structuring a session around one small goal, since reaching a usable learning loop took eight rounds of learner intervention. Treat this AI-interaction literacy as a classroom-teachable enactment of the applicational, evaluative and integrational competences in broader AI literacy frameworks rather than as a personal aptitude.
- Instructors. Give students permission to disengage, and attend to those with lower academic self-efficacy, who meet complexity and frustration where already-advantaged peers gain: the authors judged at points that reading the course book would have been cognitively simpler, and treating stopping as a legitimate move rather than a failure — and making interaction management something you teach rather than assume — is part of the competence.
- Assessment designers. Move recall-based, factoid knowledge to in-class assessment where GenAI is unavailable, and reserve take-home work for the qualitative phase — relating, evaluating and generalizing — with validity evidence drawn from revisions, decisions and reflection, since an integrity framing alone misses how demanding legitimate use has become.
- Researchers. Treat the GenAI interaction itself as an analyzable object: code successive exchanges against a structural framework such as SOLO rather than grading the final product.
Limitations
- The demonstration is a single reflective exercise run by the two authors, who role-played a naive student across eight prompt–response rounds on one take-home examination question.
- The analyzed outputs came from one primary model, Claude Opus 4.6, prompted on 22 April 2026; comparable ChatGPT 5.4 Thinking and Pro responses were observed but not reported.
- There were no student participants and no measurement of learning outcomes, so the argument rests on a documented interaction trajectory rather than on evidence that learners gain anything.
- The authors note that whether general, uninformed prompts could support the same SOLO progression remains untested, and that the emphasis on informed prompting may not stay the bottleneck as systems improve.
Citation
Brunnström, L., & Palmqvist, L. (2026). AI-interaction literacy: reflections on how generative AI might be used to support self-regulated learning in higher education. Assessment & Evaluation in Higher Education.