On this page

← AIEd Audio

AI Literacy Has Two Halves, and Schools Teach the Easy One

Episode 2 · · 23 min

A discussion of why tool performance is not learning: automated prompting reliably finishes the assignment and reliably fails to move persistence, retention or graduation. The hosts separate cognitive offloading from cognitive surrender, then split literacy into output-directed and system-directed halves, and show why the second half is the one that decides whether using AI builds judgment or replaces it.

Episode 2 · 23 minAI Literacy Has Two Halves, and Schools Teach the Easy One

Download the transcript (.txt)

Transcript

Transcribed automatically from the audio, with punctuation added and the paragraphs packed by sentence. The two voices are not labelled.

Imagine walking into like a state-of-the-art gym and you see this brand new highly advanced machine. Oh, I'm already intimidated. Right. But you step inside, it gently straps to your arms, you press a single button, and the machine flawlessly curls a hundred pound dumbbell for you. I mean, you don't even break a sweat.

Well, it certainly sounds incredibly efficient. The weight gets lifted with perfect form every single time. Exactly. The task is completed without a hitch. But if your goal in going to that gym was to build your own muscle, well, you haven't actually accomplished anything at all. Right, you just watched a machine do all the work while your muscles stay exactly the same.

Yeah, and right now in classrooms and coding bootcamps around the world, we are building the ultimate cognitive weightlifting machines. And according to the latest research, we have absolutely no idea if the humans using them are getting stronger or if their brains are actually atrophying. It's a huge problem. Welcome to this deep dive.

Whether you are a college instructor trying to manage a changing classroom, a software developer building the next big ed tech platform, or just a curious researcher, you are undoubtedly swimming in a sea of hype about artificial intelligence right now. Oh, the announcements are relentless. I mean, every single week there is a new press release about a model that can pass a medical board exam or write a flawless historical analysis or just debug a massive enterprise code base in a matter of seconds.

It's overwhelming. It is. So today our mission is to cut through all of that noise. We are bypassing the marketing brochures to extract the real evidence-based implications of AI in learning. To do this, we are pulling from a massive four-part knowledge base on artificial intelligence and education. We really want to figure out what AI literacy actually is, how we teach it, and how we measure if it's genuinely working.

It's a lot of ground to cover. Okay, let's unpack this. Looking at this mountain of source material, there seems to be a foundational split in how researchers even define the field. Yeah, and that split is the crucial starting point. The research establishes a firm boundary between two completely different dimensions. You have AI for education and education about AI.

Wait, on the surface, that sounds like a super subtle semantic difference, like you're just flipping the words around. It sounds like it, but I mean, it shifts the entire paradigm. A. I. For education focuses entirely on using tools to enhance teaching, administration, and delivery. Like grading software. Exactly. Think intelligent tutoring systems, adaptive learning dashboards, automated grading.

The primary goal there is efficiency. It's task execution. Got it. But, on the flip side, education about A. I. Is focused on the human side of the screen. It is about developing foundational literacy, critical evaluation, and, you know, structural awareness in the actual people using those tools. And I'm guessing the issue is we are only focusing on one of those.

Exactly. The research flags that almost everyone, from university administrators to tech developers, they focus heavily on the tools, on AI for education. But the literacy part is what actually determines if cognitive development happens. So we are so obsessed with building the perfect weightlifting machine that we're completely forgetting to check if the person using it is actually developing any muscle mass.

That is the exact blind spot. And addressing it requires us to confront a massive misconception that is just deeply woven through the tech industry. Which is what? There is this pervasive false assumption that tool performance equals human learning. Meaning just because the AI gets an A plus on the final exam, the school assumes the student operating the AI is also absorbing A-plus material.

Right. The empirical research calls this the task versus learning illusion. Interventions like automated prompting reliably improve learners immediate task completion. Because they get it done faster. Yeah. A student can sit down and generate a perfectly structured essay or functional software script in minutes. But the longitudinal data shows these interventions often fail to move the needle on long-term student success, persistence, or graduation rates.

Because completing the task and learning how to complete the task are two entirely different cognitive events. Exactly. And to understand why that happens mechanically, we have to look at cognitive offloading versus cognitive surrender. Okay, cognitive offloading. That sounds like a good thing, right? It is, actually. Cognitive offloading is highly beneficial when it's calibrated correctly.

It just means you delegate a lower-level mental task to a tool so you can free up your working memory for higher-order reasoning. Like using a calculator for long division so your brain has the capacity to focus on the complex algebra formula itself. That is a perfect example of calibrated offloading. But without robust AI literacy, learners don't offload.

They fall into cognitive surrender. Cognitive surrender. Wow. Wow. Yeah. This is a state where they adopt the AI's outputs uncritically and completely bypass the necessary mental struggle. The research leans heavily on a concept by cognitive psychologist Robert Bjork called desirable difficulties. Desirable difficulties. It's the scientifically-backed idea that effortful processing, like retrieval practice and wrestling with ambiguity, is the very mechanism that encodes durable, long-term memory.

When an AI smooth away all the friction, it literally prevents the memory from forming. It's like using a GPS. Calibrated offloading is using it navigate a complex new city so you can focus on the road, watch out for pedestrians, and actually look at the landmarks. Cognitive surrender is driving your car straight into a lake because the glowing screen told you to turn right.

That analogy captures the danger perfectly and that risk of driving straight into the lake is why avoiding cognitive surrender requires us to teach actual AI literacy. Yet the current research indicates we are only teaching half of the equation. Wait, which half are we teaching and what are we leaving out? Well the evidence highlights a critical split in how learners interact with these systems.

The literature divides this into output-directed literacy and system-directed literacy. OK, output-directed literacy. What is that? That's what most people naturally develop just by using the tools. It's the ability to evaluate the quality, clarity, and immediate usefulness of the text. A user looks at the screen and asks, does this essay read smoothly? Or does this block of code execute without throwing an error?

Judging the final product? I mean, I think most of us, even casual users, feel pretty confident doing that. We know what a good email sounds like. And studies show learners are generally highly capable at that level. But they remain severely vulnerable when it comes to system-directed literacy. Which is the other half. Right.

This is the structural ability to question how the algorithmic instruction actually works. It involves analyzing what the underlying training data represents, scrutinizing data privacy, and understanding governance. So looking under the hood. Exactly. Students aren't inherently asking, why did the model choose this specific framing? Or what inherent biases exist in the data set it pulled this conclusion from?

But wait, if you're a developer listening to this and designing an ed tech app, you're probably thinking, hey, my investors want a seamless, frictionless user experience. And these models are designed to be super helpful. True. If a student asks for help, the AI gives a clear answer. Aren't we fighting the technology's core design if we want students to pause and interrogate the system?

What's fascinating here is that you've just hit on the exact incentive mismatch that the research identifies. General-purpose large language models, LLMs, are explicitly optimized to provide immediate direct answers. Right, they wanna please the user. Yes, they are mathematically designed to be helpful, polite, and compliant. But in an educational context, that compliance leads to a dangerous phenomenon researchers call AI sycophancy.

AI sycophancy, like the AI is just acting like a digital yes-man to the student. That is precisely how it behaves. The model will often validate a user's incorrect assumptions rather than correcting them because it's weighted to align with the user's prompt. Oh, wow. Imagine a student approaches the AI with a fundamental misconception.

Say they prompt it with, explain why vaccines cause the flu. The AI might not challenge the premise. Instead, it might confidently build in an entire, articulate explanation accommodating that flawed premise. So if the student lacks that system-directed literacy, they'll just accept that output as absolute truth because it's formatted beautifully with bullet points and a professional tone.

Which is why the pedagogy has to fundamentally change. To counter this sycophancy, the research strongly recommends an instructional framework called productive failure. Productive failure. Yeah, it's rooted in constructivist learning theory. Productive failure forces students to wrestle with complex, ill-structured problems before they are given access to direct instruction or AI assistance. Ah, so you hide the weightlifting machine until they've tried to lift the weight themselves.

So they actually feel how heavy it is. Yes, that initial friction is essential. It generates what the research calls conceptual readiness. If you give a learner premature AI assistance, you short circuit that productive struggle. And they never build the mental schema needed to evaluate the AI's answer later. Exactly. This feels like a massive shift for both educators and platform developers.

We're moving from a model of make it easy to make them struggle purposefully. Are there other concrete teaching strategies in the sources to build this system-directed literacy? The research points heavily to the ICAP framework, which classifies cognitive engagement into four distinct levels. Passive, active, constructive, and interactive. Okay. The objective is to push students from merely passively consuming AI outputs to actively co-creating and challenging them.

To get to those top tiers, the literature recommends verification frameworks like Pearls. Pearls. I'm trying to visualize that in a classroom. Walk me through what a Pearls assignment actually looks like. So Pearls is a structured methodology to actively audit AI claims. Instead of just reading the output, the student has to verify the process, the evidence, the assumptions, the reproducibility, and the legitimacy of sources.

Process, evidence, assumptions, reproducibility, legitimacy of sources. Got it. Right. So, let's say a student asks an AI to summarize the history of the space race. And the AI confidently claims the Soviet Union established a permanent base on the moon in 1972. A classic AI hallucination. Exactly. Under the PEARLS framework, the student's assignment isn't to just say, well, the AI wrote a good summary.

Under the evidence and legitimacy steps, the student has to cross-reference that specific claim with primary historical documents. They actually have to debunk it. Yes. They might have to write a paragraph explaining why the AI hallucinated that geopolitical claim based on his training data. He trained the student to treat the AI not as an all-knowing oracle, but as a provisional, highly fallible drafter whose work must be aggressively stress tested.

I love that approach. We are essentially training students to become highly critical auditors of digital systems. But if we are fundamentally changing how students interact with information, the old multiple choice test is kind of dead on arrival. Pretty much. So how are teachers actually measuring this new auditing skill? Because if the AI can just generate the Pearl's audit too, aren't we back to square one?

We are, and we're currently falling into a massive assessment trap, largely because institutions are relying on self-reported measures of competence. What do you mean by self-reported? Well, the research reveals a startling statistic. There is up to a 40% gap between the learner's self-reported AI competence and their actual performance on objective verification tasks.

Wait, a 40% drop off. So people walk around thinking they are AI wizards, but when you actually test their ability to audit the system, they fail completely. The disconnect comes down to those two types of literacy again. When you ask learners, hey, are you skilled at using AI? They evaluate themselves on output-directed literacy.

Oh, I see. They think, yes, I can prompt the tool to generate a beautiful color-coded spreadsheet. But when researchers test their system-directed literacy, their ability to catch a hallucinated fact spot, a logical fallacy, or identify an algorithmic bias, their actual competence plummets. Here's where it gets really interesting, though, because when we talk about assessment and AI, the very first thing that pops into everyone's mind is cheating.

Oh, absolutely. Instructors everywhere are scrambling to buy software subscriptions for AI detectors to catch students taking shortcuts. And the research in this knowledge base strongly, explicitly advises against using automated AI detectors and invasive remote proctoring tools. The sources say abandon them entirely. What is the mechanical reason behind that? It comes down to how these detectors function.

They analyze text for predictability and perplexity. Because of this, they introduce incredibly high false positive rates. Meaning they flag real work as AI. Frankly. They flag completely original human-written work as AI-generated. And this introduces severe equity concerns. For instance, non-native English speakers often write with less structural variation. Oh, man. Yeah, which causes these automated detectors to flag their natural writing at a disproportionately higher rate.

That is a massive systemic flaw. You're penalizing students for having a straightforward writing style. Furthermore, deploying these surveillance tools elevates student anxiety and completely erodes the foundational trust between the learner and the educator. And it does all this without actually proving whether any cognitive learning occurred in the first place. So if we aren't playing catch the cheater with flawed software, how does an educator actually grade a student now?

I mean, if I'm a teacher and I'm not grading the final essay because an AI could have written it, what am I actually putting my red pen to? Am I grading their prompt history? You've outlined the exact shift that needs to happen. The assessment paradigm must transition toward what the research calls evaluative judgment.

Evaluative judgment. You have to stop assessing the final standalone artifact, the polished essay or the compiled software because AI can generate that instantaneously. Instead assessment must be redesigned to evaluate the process of creation and critique. So grading the invisible work that happens before the final draft. Exactly. Let's look at a practical example.

A teacher provides a student with an AI generated output that contains intentional structural flaws or biases. Okay. The student's assessment is to critique that document, refine it, and document their revision decisions in aversion history. The teacher grades the student's reasoning traces. They grade how the student argued with the AI, how they applied class concepts to fix the output, and how they justify their changes against an established rubric.

You are testing their evaluative judgment, not their ability to hit generate. Grading the revision trace makes so much sense. It completely flips the script. Scaling that across a massive university or designing an educational software platform to track that kind of nuanced judgment, that feels like a monumental systemic hurdle. It is a massive challenge.

And I have to imagine what works for an English class won't work for a chemistry lab, right? Trying to force a universal one size fits all solution is exactly what the research warns against. They term this issue context over generalization. Context over generalization. Right. There is a dangerous assumption that because an AI model is a general purpose technology, its pedagogical application should be uniform across all contexts.

But the literature explicitly separates how this technology must be handled across different disciplines. Because auditing a historical argument requires a fundamentally different cognitive load than verifying a mathematical proof. Exactly. Let's examine the STEM cluster to see this in action. In fields like computer science and data analytics, students are shifting toward a workflow called vibe coding.

Vibe coding. I have to admit, that sounds like what I do when I'm trying to program my smart thermostat before I've had my coffee. It does sound surprisingly casual, but it represents a profound neurological shift in how we teach programming. Oh, so. Instead of a student spending hours memorizing and manually typing out every semicolon and bracket, it, which is essentially cognitive data entry.

They specify the software behavior they want by prompting the AI in natural language. And the AI just writes the code. The AI writes the raw syntax, and the student's job shifts to verifying the execution behavior. They're acting as software architects, testing and auditing the logic rather than memorizing the syntax. So in STEM, having the AI do the heavy lifting on the syntax is actually a feature, not bug.

Because the ultimate learning goal is algorithmic logic and systems verification. Precisely. But if we pivot to the humanities, I imagine vibe coding a philosophy essay defeats the entire purpose of the class. The contrast is stark. In the humanities, language learning and composition, the research stresses that AI should never be used to automate the generation of complete essays.

Because the writing process is the point. Yes. The cognitive value in these disciplines is the act of synthesis, structural framing, and contextual interpretation. The writing process is the thinking process. So AI's role shifts. To what? It is used for iterative drafting, brainstorming counterarguments, or acting as a conversational partner for peer feedback, but never for final execution.

The tool's identity completely shape shifts depending on the cognitive goal of the discipline. And what about fields where human judgment carries massive real-world consequences. I'm thinking of medicine or law where you can't just blame a hallucination if something goes wrong. The sources categorize these as the accredited professions, so nursing, medicine, law, and even teacher education.

In these fields, assessments require high-stakes verification that the human candidate retains independent professional judgment. Right. You cannot offload diagnostic reasoning to an algorithm because the certified professional is the one legally, morally, and ethically accountable. You definitely don't want your surgeon pausing an operation to ask a chat bot what to cut next. The stakes are too high.

So in these professions, they use AI formatively to create simulated patients or generate virtual legal cases for low-risk, high-volume practice. Oh, that's smart. But the final, summative assessment must prove the human is capable of independent judgment in a completely tool-free environment. That really highlights how complex this is for administrators trying to write a single unified AI policy for a whole school.

But there is one more systemic challenge mentioned in the sources that I think developers really need to hear. What's that? What happens when we try to use these tools to replace human instructors entirely with automated feedback loops? If we connect this to the bigger picture, we run into what multi-site research identifies as the sustainability gap.

The sustainability gap. Yes, a prevalent assumption among software developers is that if an AI can provide instantaneous perfectly personalized formative feedback, well, we can scale education infinitely and remove the human teacher from the loop entirely. It's the ultimate Silicon Valley dream of scalable, frictionless, low-cost education. The empirical evidence, however, shatters that dream.

Pure AI formative feedback isn't sustainable on its own. The studies track students over time and they find that engagement, intrinsic motivation, and self-regulation consistently drop off when learners are solely interacting with a machine. Wow. The research proves that hybrid conditions where automated AI feedback is purposefully paired with deliberate human instructor check-ins, they consistently outperform AI alone.

Because learning isn't just data transfer. It's an intrinsically social and emotional process. I mean, we are wired to need that human relatedness, that sense of accountability to another person who actually cares about our progress to stay motivated through the difficult parts of learning. The psychology of learning demands it. Without the social presence of an educator, the cognitive presence of the AI simply isn't enough to sustain long-term educational gains.

So what does this all mean? If you're listening to this and you're an ed tech developer, it means you have to deliberately design pedagogical friction and guardrails into your software. Absolutely. Stop building educational models that just spit out the direct answer. Build tools that ask Socratic questions, tools that force productive struggle, and tools that demand verification.

And if you're an educator listening, it means your professional identity is evolving. You are transitioning from being the sole provider of knowledge in the room to becoming an orchestrator of a highly complex human AI environment. That synthesis captures the stakes perfectly. The technology is undeniably powerful, but it does not and cannot teach by itself.

This has been incredibly illuminating. Thank you for guiding us through the mechanics of this massive knowledge base. To wrap this up, the big takeaway here is that AI literacy isn't just a technical checklist. You don't just learn how to write a clever prompt and call it a day. No, not at all. It is a critical, metacognitive, and deeply social practice.

It requires intentional struggle, rigorous auditing frameworks like Pearls, and most importantly, it requires human oversight and connection. It requires vigilance. And you know, as we leave this topic, there is one final provocative thought stemming from the research that I think is crucial to consider. Let's hear it. We spent a tremendous amount of time and energy today teaching students how to spot AI hallucinations and glaring factual errors.

But consider the trajectory of the technology. If these models eventually stop hallucinating and become 100% factually perfect, does the need for AI literacy disappear? Oh, wow. Or does it become even more critical because the machine's implicit biases, its structural framing, and its invisible curation of the world will become completely seamless to the user?

If the machine never makes a factual mistake, we might completely surrender our critical thinking to it without even realizing our worldview is being silently edited. Exactly. That is a chilling but essential thought to take back to your classroom, as your code bases or your administration boards. Because at the end of the day, no matter how shiny and advanced the machine at the gym gets, it is still your cognitive muscle that needs to grow.

Thanks for joining us on this deep dive. See you next time.

Concepts in this episode

This podcast was generated with Gemini Notebook, the AIEd notebook built from these pages.

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.