Pedagogical Steering of LLMs for Tutoring: Modeling Productive Failure

Synthesis: Puech, Macina, Chatain, Sachan and Kapur (2025) introduce Pedagogical Steering — the problem of getting LLM-based tutors to follow effective teaching strategies rather than behaving as answer-giving assistants — and present StratL, an algorithm that steers an LLM through a predefined multi-turn tutoring plan represented as a transition graph. In a case study, they engineer an LLM tutor for high school math that follows Productive Failure pedagogy, withholding solutions and eliciting multiple solution attempts. A field study with 17 Singaporean students showed StratL significantly increased fidelity to the PF strategy, though the PF tutor was perceived as less "helpful" than a baseline — the central tension between learning effectiveness and user satisfaction.

The Pedagogical Steering problem

One-to-one tutoring is among the most efficient teaching methods, but is expensive — the "Two-Sigma Problem" (Bloom, 1984). LLM-based conversational tutors aim to scale it affordably. However, current LLMs are instruction-tuned to be helpful assistants and maximize user satisfaction; a tutor's goal is to maximize learning, not satisfaction. These goals conflict: directly providing the solution maximizes satisfaction and is what LLMs prefer, but promoting active engagement (indirect hints, guiding questions) is more effective for learning (Freeman et al., 2014; Merrill et al., 1992). Moreover, LLM tutors are optimized/evaluated for single-turn requests, while tutoring is a multi-turn interaction using various pedagogical cues. Shifting LLMs from their user-serving goal to a pedagogically suitable one is the Pedagogical Steering problem.

StratL

StratL models a multi-turn tutoring strategy with LLMs. A multi-turn strategy is a succession of single-turn pedagogical goals called tutoring intents. StratL uses a transition graph to dynamically redefine the tutoring intents after every student utterance, prompting the LLM to follow these intents for the next turn. Built with learning scientists, it integrates a strategy based on Productive Failure (Kapur, 2008, 2024).

Field study results (17 high school students, Singapore)

• Strategy fidelity: On both test problems, the StratL-controlled LLM (V1) achieved a higher PF score than the baseline (V2), statistically significant for problem Consistency (p=.046).
• Eliciting multiple solutions: The baseline LLM failed to elicit multiple RSMs (solution attempts), while a StratL-tutored student produced on average 2.6 RSMs per session (p=.05) — succeeding in the PF process of making students explore as many solutions as possible.
• Perceived helpfulness trade-off: StratL had no significant effect on coherence or empathy. The baseline (no intents) was perceived as more helpful — because the PF strategy only gives help when strictly necessary, so students spend more time stuck and exploring suboptimal solutions. The authors note this can feel frustrating or be misunderstood by students, even though it is more effective for learning.

Relevance to the wiki

This paper is the key technical anchor connecting LLM-based tutoring to Productive Failure pedagogy. It demonstrates that LLMs must be steered to implement productive failure (withhold answers, elicit attempts, help only when necessary) — and that doing so trades perceived helpfulness for learning effectiveness. It connects to Socratic Method (indirect hints and guiding questions), Cognitive Offloading (resisting the urge to hand over solutions), Scaffolding (giving help only when strictly necessary), and the design of AI tutors. Co-authored by Manu Kapur, the originator of productive failure.

Connected Concepts

• Productive Failure
• LLM
• Intelligent Tutoring
• Socratic Method
• Cognitive Offloading
• Scaffolding
• Generative AI
• K 12
• Math Education
• Human In The Loop AI

Connected Articles

• Kim AI Productive Failure Adult 2026 — Designing AI Systems for Productive Failure
• Rhaimi Productivemath 2025 — ProductiveMath: AI to Support PF Problem Design
• Wang Safety Gap Productive Struggle 2026 — The Safety Gap: Restoring Productive Struggle
• Lukesova Clue Before Correction 2026 — Clue Before Correction: ChatGPT for Autonomous Learning

Citation

Puech, R., Macina, J., Chatain, J., Sachan, M., & Kapur, M. (2025). Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure. Findings of ACL 2025, 26291–26311.