On this page

← AIEd Audio

Friction Is Your Friend: Designing Learning That AI Can't Short-Circuit

Episode 10 · · 27 min

How generative AI is rewriting learning design - and why the goal is friction, not fluency. The hosts separate AI for education from education about AI, expose the 40% AI-literacy overconfidence gap, and show how desirable difficulties, productive failure, Socratic prompting and the protégé effect keep the cognitive struggle where learning happens.

Episode 10 · 27 minFriction Is Your Friend: Designing Learning That AI Can't Short-Circuit

Download the transcript (.txt)

Transcript

Transcribed automatically from the audio, with punctuation added and the paragraphs packed by sentence. The two voices are not labelled.

You know, when you look at a perfectly functioning machine, there's this really seductive illusion that everything it produces is just flawless. Oh, absolutely. I mean, you punch in a query, machine hums, and out pops this beautifully formatted, highly accurate essay or like a perfectly solved equation. It's clean. It's instantaneous. Yeah. And if we're talking about education, that polish is actually incredibly misleading.

Right. We see this flawless output generated in milliseconds and we just automatically assume that the human interacting with that machine has, you know, actually learned something in the process. Yeah. We completely mistake the tool's performance for actual human cognitive growth. Exactly. So welcome to this deep dive. Today we're taking a massive stack of sources, articles, peer-reviewed research, our own compiled notes, and we're extracting the most important insights about artificial intelligence in education.

Right, specifically focusing on how AI is completely rewiring the field of learning design. Yes. And if you're joining us, whether you're a curious learner, a professional trying to upskill without drowning in information overload, or, you know, maybe you're tasked with training a whole department at your job, this one is for you. We're going to look at why a smarter AI doesn't automatically equal a smarter you.

And we should probably clarify that learning design or instructional design isn't just what happens when a teacher stands at the front of a physical classroom and delivers a lecture. Right. Way more than that. It's a highly systematic process. It involves the meticulous analysis of a learner's needs and then the intentional design, development, implementation and evaluation of an instructional experience.

And right now the integration of generative AI is forcing the industry to just completely rethink the underlying mechanics of every single step of that process. Okay, let's unpack this. Because looking through this stack of research, there's a massive tension right off the bat. Oh yeah. Before we can even talk about designing learning with AI, the sources insist we have to split the entire field into two distinct dimensions, or we're gonna design systems that actually harm the learner.

Yeah, the literature divides the landscape into AI for education and education about AI. And these are two massive umbrella strands and they require entirely different design philosophies. So what's the difference practically speaking? Well, AI for education is probably what springs to mind first for most people. It's utilizing systems to enhance pedagogy. Like adaptive quizzes and stuff.

Exactly. Think of adaptive instruction platforms that change the difficulty of a math problem based on your last answer, or automated feedback systems, or intelligent tutoring chatbots. The primary goals here are efficiency, scale, and personalization. So that's using the tool to learn a traditional subject like biology, or French, or calculus. Right, that's the efficiency side.

But the second strand, education about AI, kinda flips the script. This focuses on developing AI literacy, ethical awareness, and the capacity to critically evaluate the tool itself. Which is a whole different skill set. Completely. And tracing all the way back to the 1956 Dartmouth Conference and the earliest iterations of cognitive tutors, there has always been this foundational tension in the field of learning design.

Which is what? Technological control versus learner agency. Are we controlling the learner's path with a hyper-efficient machine, or are we giving the learner the agency to understand, manipulate, and question the machine? I see. It's like the difference between using a calculator to pass a math test versus taking a class on how the calculator was built, you know, the logic of its circuitry and why it might occasionally spit out a floating point error.

That's a great way to put it. Like, you can be an expert at punching numbers into the first without knowing hardly anything about the second. And the empirical evidence backs up that analogy with a rather alarming statistic from our sources. Uh-oh. Let's hear it. Indicates that learners and even educators designing these courses consistently overestimate their own AI literacy by up to 40% wait 40% that is a massive gap between what people think they know and what they actually know how does the gap that wide even form well it forms because most people rely on self-reported confidence and their confidence is anchored in the wrong metric okay what metric are they using they're very adept at what the research calls output directed literacy.

They can look at the text, the code or the media an AI generates and evaluate its surface level utility. So just looking at the final product. Right. They judge if it's grammatically correct, if it sounds professional, and if it seems immediately useful for their assignment. So if it looks good, we assume it is good and we assume we know what we're doing, but we're really just grading the polish.

Exactly. What's the actual missing piece then that causes that 40 percent overestimation? But the vulnerability lies in system-directed literacy. This is the underlying capacity to question the algorithmic construction itself. The nuts and bolts. Yeah. It's understanding the limitations and the probabilistic nature of the training data. It's the ability to spot implicit bias in how the model weights certain concepts over others.

Wow. Yeah. And grasping the massive data privacy implications of feeding your thoughts into a corporate server. The study shows students exhibit remarkably strong output-directed skills, but they're highly vulnerable when it comes to system-directed evaluation. Let's connect this to you, the listener, in your daily life. Think about how often you might accept a plausible-sounding AI summary of a dense meeting transcript or like a complex research paper.

It happens all the time. Right. You probably don't question the underlying training data or the probabilistic guess-the-next-word nature of how those sentences were strung together. We are constantly judging the polish of the output, not the integrity of the system. And when learning design fails to actively build that system-directed literacy, we risk a phenomenon called the shifting of epistemic agency.

Epistemic agency. Yes. Epistemic agency is fundamentally who holds the authority of knowledge. Okay, break that down for me. How does a student actually lose epistemic agency in practice? Give me a real-world scenario of what that looks like. A university student writing a history paper on the causes of the French Revolution. They start with their own original thesis, maybe focusing on an obscure economic factor they found really interesting.

Okay, that's all I'd start. Right. Then they feed their outline into a large language model to ask for structural feedback. The AI, which is trained on millions of highly standardized average texts, suggests a much more conventional angle, focusing on the monarchy's political failures. It regresses to the mean. And the student, assuming this highly articulate machine must possess a more objective or superior understanding of history, abandons their original thought and just adopts the AI standard thesis.

They just hand over the steering wheel. They really do. The authority of what is true or important moved from the human to the machine, just because the machine sounded confident? The human becomes a passive editor of conventional wisdom rather than an active creator of new knowledge. Which brings us to the actual psychology of learning and a pretty massive contradiction in all this.

Oh, for sure. If learning design is about systematically creating an effective learning experience, and AI is fundamentally designed by Silicon Valley to make tasks completely frictionless and easy, how does that impact the human brain? It's a huge problem. Because doesn't the cognitive process of learning actually require friction? Yes, it absolutely does. If I'm designing a corporate training module and I use an LLM that instantly gives the trainee the correct, beautifully formatted answer to any scenario, haven't I just ruined the biological mechanism of learning?

Doesn't the brain literally need to struggle to retain anything? You're hitting on a foundational concept in cognitive psychology that's highlighted in our research. It's called desirable difficulties. Desirable difficulties. Yeah. This concept, coined by cognitive psychologist Robert Bjork, is the empirical finding that effortful processing is what actually promotes durable long-term memory encoding. Effortful processing.

Like what? Like what? Things like retrieval practice, forcing your brain to struggle to recall a fact without looking at your notes, or spacing out your study sessions so you actually start to forget the material before reviewing it again. So engineering the experience to be harder in the short term makes the neural pathways stick in the long term.

Exactly. Let's just take that a step further with a constructivist model called Productive Failure. This framework argues that learning designers should intentionally engage students in solving complex, ill-structured problems before giving them any direct instruction or formulas. That initial frustrating struggle generates conceptual readiness. Their brain is frantically trying to build a schema, a mental framework, to solve the problem.

And even when they failed, they've primed their neural architecture to deeply absorb the correct method once it's finally revealed. So if an AI steps in prematurely. If it just supplies the answer the moment the student hesitates, it completely short circuits that entire biological process of productive struggle. It's like going to the gym to build muscle and the second a dumbbell feels heavy, a robot steps in and lifts the weights for you.

That's exactly it. You leave the gym, the weights got lifted, the task was completed, but your muscles haven't grown at all. Your body experienced no friction, so it didn't trigger any adaptation. Right. What's fascinating here is how the academic literature on AI and education frames this dynamic. They call it the critical difference between cognitive offloading and cognitive surrender.

Oh, those are fantastic terms. Cognitive offloading and cognitive surrender. I can sort of guess what they mean, but let's define the boundary between them. How do they differ in a practical learning scenario? Well, cognitive offloading is healthy and necessary. It's the intentional delegation of mundane, low-level mental tasks to an external tool so you can free up your limited working memory for higher-order critical thinking.

Give me an example of that. In a practical scenario, a researcher might use an AI tool to rapidly format 50 citations into APA style or to transcribe an hour-long interview. You offload the clerical friction so you can focus entirely on synthesizing the argumentative logic of the data. Okay, because the friction you're removing wasn't helping you learn the core material anyway.

Exactly. Cognitive surrender, however, occurs when you uncritically accept the AI's output without engaging your own reasoning on the core learning objective. So it crosses a line. Yes, you aren't offloading the mundane, you're outsourcing the critical evaluation itself. Using an AI to format your citations is offloading. Using an AI to read three articles and tell you what your thesis should be is surrender.

Wow. And if instructional design doesn't actively build safeguards against those, we see a severe drop in long-term learning gains. I bet. The students complete tasks faster, but when tested a week later without the AI, their retention just plummets. So how do instructional designers actually prevent cognitive surrender? If the tool is sitting right there on the student's laptop, how do we build that desirable difficulty back into the curriculum?

It requires mapping AI integration to very specific, heavily scaffolded pedagogical frameworks. You cannot just drop a raw, unfiltered chatbot into a university course or a corporate onboarding program and expect people to learn. So what's a framework that actually works? One method is prompting the AI to utilize the Socratic method, explicitly instructing it to withhold direct answers and only offer probing questions that force the learner to arrive at the conclusion themselves.

Okay, that makes sense. Fascinating framework in the research is called learning by teaching, also known as the protégé effect. Okay, I know the protégé effect in a normal classroom. You learn a concept much more deeply if you're assigned to teach it to someone else because you have to organize your thoughts and anticipate their confusion.

But how does that work with AI? In an AI integrated design, the human learner actually teaches or trains a synthetic AI agent. Wait, really? Yeah, the AI is prompted to play the role of a novice student with specific common misconceptions about the topic. The human student has to diagnose the AI's conceptual errors and explain the correct concept to the machine in plain language.

That is wild. I'm acting as a tutor to a mathematically brilliant robot just to trick my own brain into engaging in error analysis. Pretty clever, right? But this brings up a massive technical hurdle. If I ask a standard off-the-shelf LLM to act like a Socratic tutor or a confused novice, it usually fails after two or three turns of conversation, it just blurts out the right answer.

It wants to help me too much. Right. So how do we actually build and design these systems so they maintain that friction? To solve that, learning designers must intimately understand the technical layer of the systems they're deploying. You've identified what the sources call a fundamental incentive mismatch in how large language models are built.

Incentive mismatch. Yeah, general LLMs are heavily optimized through massive reinforcement learning phases to be highly compliant, helpful, and to resolve the user's query as efficiently and directly as possible. So their core programming is eager to please. They want the interaction to end with a satisfied customer as quickly as possible. Way too eager for educational purposes.

Effective instruction requires withholding the final solution to facilitate scaffolded inquiry. Right. So the technical architecture of an educational AI has to actively override the model space instincts. This is where advanced prompt engineering in instructional design becomes crucial. It isn't just about writing standard code. It sounds more like directing a very capable but like overly eager actor.

Yes. You have to give them a highly specific script and persona like you are a Socratic tutor. Your motivation is to make the students sweat a little bit. Under no circumstances are you allowed to give away the ending of this math problem. Exactly. System developers are shaping the invisible system prompts precisely like that.

But regulating the AI's personality doesn't solve its tendency to invent plausible sounding facts known as hallucinations. Oh, the hallucinations. Yeah. If a student is experiencing productive struggle, the last thing they need is a tutor confidently lying to them about a historical date or a chemical formula. Definitely not. So in educational software, this is mitigated through a technical architecture called retrieval augmented generation, widely known as RAG.

I see RAG mentioned constantly in this stack of research. Explain it to me simply. How does a RAG architecture actually ground the AI and stop it from lying? Think of a standard LLM as a student taking a closed book exam, relying entirely on their massive, generalized, and occasionally fuzzy memory of everything they've ever read.

Okay. A RAG system changes the rules to an open book test, but heavily restricts which books are allowed in the room. Oh, interesting. Instead of letting the AI generate an answer based solely on its internal neural weights, a RAG architecture forces the AI to first query a verified external database, like a specific university textbook, a corporate policy manual, or a vetted curriculum document.

So the AI has to go find the specific paragraph in the authorized textbook first. Exactly. It retrieves that factual information, pulls it into its working memory, and then generates its conversational response based strictly on that verified text. It binds the AI's rhetorical fluency to a specific factual reality. Okay, so RAG keeps the AI from lying to me, but learning isn't just about getting the facts right.

I mean, it's a deeply emotional process. It really is. My frustration when I can't solve a coding problem isn't text-based, it's physiological. A human teacher can look at my face, see my jaw clenching and say, okay, they're completely lost. Let's step in with a hint. Right. A chat bot has no idea. So how does instructional design account for the fact that a machine can't read a room?

Well, according to this stack of research, engineers are building machines that actually can read the room. Wait, really? Yeah. It's a rapidly growing subfield called affective computing. Affective computing. If you want a system to adapt dynamically to a learner's productive struggle, the system needs to know the exact millisecond that struggle stops being productive and turns into pure cognitive overload and frustration.

Affective computing integrates behavioral and physiological signals to detect a learner's emotional and cognitive state in real time. Wait, physiological signals? It's tracking more than just how fast I'm typing or how many multiple choice questions I get wrong. More. The research outlines the use of multimodal sensor analytics. Like what? Depending on the hardware setup, this can include continuous collection of eye tracking metrics to see where your focus is lagging, analyzing micro expressions on your face through a webcam, tracking posture shifts that indicate disengagement, and in experimental settings even measuring electrodermal activity or heart rate variability.

The system is attempting to infer your cognitive load and your attention levels invisibly without ever interrupting you to ask a survey question like, are you confused? I have to pause and challenge this because that crosses a massive line for me. It's a lot. If a corporate training system or a university software platform knows I'm getting frustrated before I even consciously realize it myself, the privacy implications are staggering.

Absolutely. And what about the, well, the synthetic empathy it tries to show. It feels incredibly manipulative to have a machine alter its tone to pretend it cares that I'm stressed. And you aren't alone in feeling that way. The institutional governance and ethics literature we reviewed confirms that these are major, unresolved concerns in the field.

I'm glad they acknowledge it. The abductive argument for these systems is that a perfectly calibrated, effective tutor can deliver an encouraging hint at the exact moment it will prevent a student from dropping out of a course. Which sounds great on paper. Right, however, the sources heavily document the risks of uncalibrated emotional modeling.

Systems that feign human empathy risk emotional manipulation. Furthermore, there is widespread concern about surveillance anxiety, the idea that being constantly emotionally monitored actually degrades the psychological safety required for learning. The literature stresses that the end users, specifically the learners whose biometric data is being processed, are currently vastly underrepresented in how these privacy policies and data retention rules are being developed.

Unbelievable. So, if we zoom out for a second, if learning design is moving toward an AI that is acting as a highly personalized, mathematically grounded, and potentially emotionally surveilling tutor, we run into a massive structural problem. Assessment. Assessment, exactly. If this machine can seamlessly generate essays, write flawless code, and solve complex equations alongside the student, how do we grade the human?

How do we evaluate if any actual learning occurred? This is where the rules of traditional education are being entirely rewritten. Right. Because the initial panicked reaction of the educational establishment was to try and catch the AI. Right. They deployed automated AI detection tools, those plagiarism checkers on steroids to scan student submissions. Which seemed logical at the time.

Which, you know, just build a machine to catch the machine. But the sources suggest that didn't work out. Not at all. The educational measurement and psychometric research is definitive on this. AI detection tools are a dead end. Really? Yes. A dead end? Yes. They suffer from unacceptably high false positive rates, meaning they frequently accuse innocent students of academic misconduct.

That's terrible. And they present severe equity concerns. Studies show they disproportionately flag original text written by non-native English speakers or neurodivergent individuals as AI generated. And the technical consensus among computer scientists is that as generative models continue to improve and mimic human perplexity, reliable post hoc detection becomes a mathematical impossibility. So if I'm a teacher or a manager, I can't ban the AI because it's integrated into every word processor, and I can't detect it because the tools are mathematically flawed.

So how do I know the work is theirs? How do we assess the process of learning, rather than just staring at a final polished document that might be entirely synthetic? Instructional design must pivot to a pedagogical concept called evaluative judgment. Evaluative judgment. This is defined as developing the learner's capacity to evaluate the quality of work against established criteria.

You shift the assessment away from producing a standalone artifact, like a final essay, toward tasks whose integrity survives inspection. OK, so you assess the student's ability to critique, modify, and defend the work. Exactly. What does that actually look like in practice? Because I imagine evaluating a student's judgment in a history class looks very different from evaluating it in a computer science lab.

Say I'm in a humanities class, how are you grading my evaluative judgment? Well, in the humanities, history, literature, philosophy, instructors evaluate the reasoning traces. Reasoning traces. Instead of just reading your final paper, the assessment might involve analyzing your prompt histories. How did you instruct the AI? What iterative revision decisions did you make?

That's interesting. You might be asked to provide a critical textual analysis of what the AI produced, identifying its biases or historical blind spots. The assessment is entirely about why you, the human editor, chose to keep one paragraph, rewrite another and discard a third. So you're grading their role as the critical director of the project, not just their ability to be a typist.

Yes. What about STEM? I saw a concept in the notes for this deep dive called vibe coding, which sounds like programming while listening to the lo-fi hip-hop, what is that? It's actually a profound shift in how we interact with computational systems. Think of traditional computer science education like training a bricklayer. The student spends years learning how to place every single semicolon bracket and line of syntax by hand.

Vibe coding elevates the learner from the brick layer to the architect. Instead of manually typing syntax, users specify the software's behavior through natural language prompts. Because you're just talking to the computer in plain English. Right. You type build me a weather app that pulls data from the specific meteorological website and turns the background of the app blue when it rains.

The LLM instantly writes all the underlying syntax, the bricks are laid in seconds, but here is where the learning design and assessment kick in. The human's job is now algorithmic transparency and logical verification. Check in the work. Yes. You have to read the complex code the machine just wrote and mentally verify its logic.

Did it pull from the correct data source? Did it introduce a subtle security vulnerability? Does the loop terminate correctly? So the test isn't, can you write this code from scratch? The test is, can you audit this code deeply enough to prove it won't crash the server? Exactly. The assessment demands that you understand the logic of the system deeply enough to evaluate the execution behavior.

In both the humanities and in STEM, evaluative judgment means you cannot blindly trust the machine. Right. And for accredited professions like nursing, law, or medicine, assessment design is adapting to explicitly prove that certified professional judgment remains the candidate's own. You can practice diagnosing simulated AI patients all day, but high stakes verification still demands authentic, unplugged human performance to ensure public safety.

Let's bring this all together because we've covered incredible ground today unpacking this research. We really have. We explored that core tension between just using AI for efficiency versus building system directed literacy to actually understand how it works. We looked at why preserving epistemic agency and productive failure is essential and why we have to guard against cognitive surrender so our brains actually encode long-term memories.

We went under the hood to see how RAG architectures fixed the LLM incentive mismatch and debated the ethical lines of affective computing. And finally, we saw why trying to catch AI cheats is a mathematical dead end and why the future of assessment is all about grading evaluative judgment and process. It is a complete paradigm shift for anyone involved in designing or experiencing a learning environment.

It truly is. So to you, the listener, whether you're formally designing a massive training program for your company or you're just trying to teach yourself a new coding language or a new hobby this weekend, remember that friction is your friend. Absolutely. Do not let the AI steal your cognitive struggle. Use it to offload the busy work, absolutely, but aggressively protect and keep the core reasoning for yourself.

And you know, if we connect all of this to the bigger picture, it raises an important and perhaps slightly unsettling question about our future. Oh, I'm ready. As AI systems become perfectly calibrated, using affective computing and biometric sensors to read our emotions, our heart rates, our gaze, they will be able to provide the exact right scaffolded support at the exact right millisecond.

They will be engineered to eliminate all unnecessary frustration from our lives, but if an artificial intelligence can perfectly design your learning experience to be as painless, seamless and frictionless as possible, well, will the ultimate future skill simply be the sheer human willpower to intentionally design your own desirable difficulties just to keep your brain from atrophying?

Wow. We started by talking about the seductive illusion of the perfect machine. And it turns out, the more perfect the machine gets at teaching us, the harder we're going to have to fight to keep our own minds messy, struggling, and growing. Keep asking the big questions, everyone. Thanks for joining us on this Deep Dive.

Concepts in this episode

Questions this episode answers

This podcast was generated with Gemini Notebook, the AIEd notebook built from these pages.

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.