Creating an AI-powered platform for generating modelling problems: A case study on direct variation in secondary school

Synthesis: This study reports the design, development, and evaluation of an AI-powered platform that generates mathematical modeling problems and accompanying pedagogical recommendations for secondary school mathematics. The work addresses two practical gaps: teachers' shortage of suitable modeling resources and the tendency of existing Generative AI tools to produce conventional word problems or routine exercises rather than application-oriented tasks that build modeling competencies. The platform is grounded in the ADDIE (Analysis, Design, Development, Implementation, Evaluation) instructional-design model and combines seven established design principles for mathematical modeling instruction with retrieval-augmented generation over expert-crafted exemplar tasks. The topic of direct variation served as an accessible case for an in-depth mixed-methods case study (Mixed-Methods Research).

The platform gives teachers a unified workspace: a central panel to specify generation requests (question type, free-text theme, and output language such as English, Chinese, or bilingual), a history panel for reusing prior resources, and an embedded conversational assistant with "chat" and "generate" modes. Text generation uses GPT-4.1 for curriculum-aligned problems and teacher guides, while DALL-E 3 produces contextually appropriate illustrations. A focal teaching intervention with 49 secondary students (Grades 10–12, Hong Kong) and an evaluation study with 36 in-service teachers generated both quantitative and qualitative evidence. The authors frame the platform as a human–AI co-design partner that reduces teachers' preparation workload while still requiring professional judgment to refine cognitive demand and classroom use of AI-generated content.

Key Findings

• Significant short-term learning gains. Students (n = 49) scored significantly higher on the post-test (median = 15.00) than the pre-test (median = 7.00), confirmed by a Wilcoxon signed-rank test (Z = 6.10, p < 0.001, large effect size r = 0.62). School-mathematics scores rose from a median of 5.00 to 8.00 (Z = 5.58, p < 0.001, r = 0.56), and modeling scores from 1.00 to 8.00 (Z = 5.95, p < 0.001, r = 0.60).
• Modeling-specific competencies improved sharply. Over 90% of students provided correct solutions for identifying assumptions in both linear and exponential contexts on the post-test (e.g., "the thickness of each book is the same"), and roughly 70–86% correctly identified relevant factors — though over a quarter of students still failed to reach correct solutions on some items.
• Engagement dimensions operated differently. Behavioral engagement significantly and positively predicted post-test scores (B = 1.279, p = 0.029, R² = 0.098), and emotional engagement positively predicted learning gains (B = 1.359, p = 0.037, R² = 0.090). In contrast, cognitive engagement showed no significant correlation with either post-test scores or learning gains.
• Classroom discourse remained instructor-dominated. Analysis of 151 speech turns across five lecture episodes (using a mathematics-discourse-in-instruction framework) showed the AI-generated problems structured an "I do, we do, you do" progression that supported naming, legitimation, examples, and tasks. However, opportunities for extended, student-initiated contributions remained limited, echoing patterns in Confucian-heritage mathematics classrooms.
• Teachers rated resources positively but noted limitations. All five evaluation dimensions exceeded a mean of 4.0 on a 5-point scale, with over 80% of teachers rating each as "good" or "very good." Qualitative analysis (inter-coder agreement 91.4%) surfaced strengths such as authentic everyday-life scenarios, clear structure, and stepwise scaffolding, alongside concerns about oversimplified assumptions, limited variety and difficulty, and occasional issues with data realism and AI-generated images.
• Data realism was a recurrent weakness. Both experts and teachers flagged that some numerical values were adjusted for computational convenience rather than realism — for example, a generated context referenced a petrol price of HKD 18 per liter when the market rate was around HKD 30 at the time, prompting recommendations to introduce randomness into data preparation.

Implications for Practice

• Use AI as a co-design partner, not a replacement. The platform should be treated as a Human AI Collaboration partner that drafts candidate materials, while teachers retain responsibility for refining contextual realism, verifying correctness, and calibrating difficulty to students' ability levels.
• Ground generation in design principles and exemplars. Embedding seven modeling design principles and expert-crafted exemplar tasks into the retrieval-augmented generation base produced curriculum-aligned, stepwise-scaffolded classroom materials — a design strategy teachers can demand from similar tools.
• Scaffold and support modeling discourse. Because classroom talk stayed teacher-led, future teacher guides should include suggested academically productive talk moves (e.g., pressing for reasoning and challenging ideas) to create more space for students to articulate, compare, and justify modeling assumptions.
• Raise cognitive demand deliberately. Only about half of students reported intense cognitive immersion (CE_4, 51.0%), suggesting that scaffolding designed for Accessibility can undershoot cognitive engagement. Presenting alternative AI-generated datasets (with noise or outliers) and tasking students to justify data-handling choices could deepen engagement.
• Add platform features for iteration and trust. Recommended enhancements include teacher-editing interfaces, validation checklists (e.g., data-realism checks comparing generated values with plausible ranges from market prices and public statistics), in-platform reporting to flag hallucinations, and options to adjust task complexity.

Connected Concepts

• Math Education
• Generative AI
• Learning Design
• Curriculum Design
• Intelligent Tutoring
• Scaffolding
• Prompt Engineering
• Human AI Collaboration
• Teaching
• Student Engagement
• Problem Solving
• RAG (Retrieval-Augmented Generation)

Connected Articles

• A didactical-driven teacher assistant for a dimensional modeling course — Both papers build AI teacher assistants that ground instructional output in domain-specific design frameworks to support lesson preparation.
• Inquiry-Based Learning in STEM Education: The Impact of Generative AI-Based Chatbots on Primary School Students' Problem Posing Ability in Science — Both examine generative-AI tools that pose mathematics problems, here extended from primary problem-posing to secondary mathematical modeling.
• Assessing AI-TPACK readiness in mathematics teacher education: The role of self-efficacy and teaching beliefs — Both situate AI integration in secondary mathematics teaching and consider the teacher competencies and beliefs required to use AI tools effectively.

Citation

Creating an AI-powered platform for generating modelling problems: A case study on direct variation in secondary school mathematics — Lo, C. K., Huang, X., Cheung, H. W., Yee, T. L., Bai, S., Chen, G., & Tlili, A. (2026). Computers and Education: Artificial Intelligence, 11, 100640.