ProductiveMath: A Generative-AI-Powered App to Support Productive Failure Teaching

Synthesis: Rhaimi et al. (2025) present ProductiveMath, a generative-AI-powered tool that helps teachers design high-quality Productive Failure problems — a known challenge in PF Pedagogies and Teaching Strategies. Across three studies, they define a rubric for high-quality PF problems, use GPT-4o to generate and evaluate them alongside human raters (finding strong AI-human correlation), and report that seven math teachers rated AI-generated algebra problems as high quality with positive usability perceptions and adoption intent.

The challenge it addresses

Productive Failure engages students in Problem Solving before instruction, but designing effective PF problems is challenging (Kapur, 2015). ProductiveMath addresses this by using generative AI to support teachers in creating PF problems — lowering the design burden that otherwise makes PF difficult to implement.

Three studies

• Study 1: A literature review defined high-quality PF problems and produced a rubric; GPT-4o generated and evaluated 30 problems alongside human raters.
• Study 2: Replicated with 60 additional problems, showing strong correlations between AI and human ratings — confirming GPT-4o's capability to produce high-quality PF problems.
• Study 3: Seven math teachers evaluated human- and AI-generated problems via surveys and interviews. They rated AI-generated algebra problems as high-quality, reported positive perceptions of ProductiveMath's usability, and expressed intentions to use it. Teacher feedback included adjusting problem difficulty, simplifying text, enhancing visuals, and providing additional PF support.

Relevance to the knowledge base

This paper connects Productive Failure to teacher support and Generative AI: AI can lower the barrier to implementing PF by Scaffolding the creation of high-quality PF problems. It sits alongside the other productive-failure articles as evidence that AI can support (here, help design) productive-failure pedagogy, complementing the articles that focus on AI preserving struggle during learning.

What this means for practice

• Instructors. Generate candidate PF problems with GPT-4o and then edit them before class: the seven teachers rated AI-generated problems as high quality (M = 17.19, SD = 3.07), close to human-generated problems (M = 17.43, SD = 3.49), but flagged vague information, excessive length, and demands on advanced literacy.
• Instructors. Check curriculum alignment yourself before assigning a generated problem — teachers rated AI-generated problems lower on Curriculum Alignment (M = 2.57, SD = 1.43) than human-generated problems (M = 3.29, SD = 0.49).
• Instructors. Apply the fixes teachers asked for most when adapting generated text: visual enhancements (100% of interviewees), better formatting (71%), simplified text (57%), and reading-level adjustments (42%), with visuals placed at the top and text segmented and bolded.
• Instructors. Screen generated problems against the five rubric dimensions — prior knowledge activation, affective engagement, sweet spot calibration, open-endedness, and multiple representations, each rated 1–5 with a maximum total of 25 — since AI problems scored well on Sweet Spot Calibration and Affective Engagement but varied elsewhere.
• Instructors. Use ProductiveMath problems as warm-ups or small-group activities, the uses the interviewed teachers identified as the best fit for the tool.

Limitations

• Study 3 rests on 7 middle-school math teachers (Grade 6 = 2, Grade 7 = 3, Grade 8 = 2) recruited through district partnerships; the evidence is teacher survey ratings and interview perceptions, not student learning.
• The AI-generated corpus was algebra-only and model-specific: 30 problems in Study 1 and 60 in Study 2, all produced by GPT-4o.
• Ratings used the authors' own five-item rubric (maximum total 25) rather than an external quality or learning measure, so the studies do not establish that productive-failure outcomes improve for students.
• ProductiveMath has no student-facing features yet — the authors state that future work will add them — so the tool's classroom effects on productive failure remain untested.

Citation

Rhaimi, S., Ercan, D., Gao, R., Esmaeiligoujar, S., Babaee, M., Li, H., Zhang, S., Lee, S., Closser, A., & Botelho, A. (2025). ProductiveMath: A Generative-AI-Powered App to Support Productive Failure Teaching. University of Florida. DOI: 10.1007/978-3-031-99264-3_43.