TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors

Authors: Isabel Molnar, Peiyu Li, Si Chen, Sugana Chawla, James Lang, Ronald Metoyer, Ting Hua, Nitesh V. Chawla Year: 2026 Venue: arXiv (cs.AI)
Year: 2026
Venue: arXiv (cs.AI)

Summary

Higher education instructors often lack timely and pedagogically grounded support. Universities operate teaching and learning centers that offer workshops and consultations, but these resources face key limitations: support may not be available at the moment of need, feedback is often generic rather than tailored to an instructor's background, and some educators hesitate to seek direct help for fear of appearing unskilled. General-purpose LLMs like ChatGPT are widely accessible, but their responses are typically generic and rarely apply evidence-based principles in a pedagogically scaffolded way. TeachingCoach is a pedagogically grounded chatbot that simulates the role of a teaching expert and delivers conversational guidance on instructional practice through problem identification, diagnosis, and strategy development.

The system is built on a data-centric pipeline. First, pedagogical rules are extracted from educational resources, with experts encoding 36 core instructional rules as structured system prompts that anchor conversations in evidence-based practices. Second, GPT-4o generates realistic and diverse training data: teacher profiles specifying years of experience and teaching subject, teaching challenges such as managing classroom attention, and multi-turn conversations between a teacher and a simulated expert that progress over 20–30 turns through clarifying the challenge, exploring strategies, planning next steps, and reflecting on practice. Human experts review the generated conversations, removing those that are inconsistent, repetitive, or pedagogically unsound; after filtering, the dataset contains 406,183 training, 4,156 validation, and 4,143 test examples. Finally, a LLaMA-2-13B-Chat model is fine-tuned with full parameter updates; at each turn the assistant first predicts an instructional step — Step 1: Identify the Problem, Step 2: Explore Reasons, Step 3: Develop Strategies — and then generates the response conditioned on that step, aligning dialogue progression with a structured pedagogical scaffold.

Key Contributions

Outperforms a GPT-4o baseline on expert-rated dialogue quality: across 200 conversations rated on a 3-point scale over four dimensions (clarity, respectful tone, encouragement of reflection and reasoning, acknowledgment of user input), TeachingCoach consistently scored higher (e.g., 2.62 vs. 2.01 on clarity).

A user study with higher education instructors highlights trade-offs between conversational depth and interaction efficiency: although the baseline model was preferred overall (21 vs. 13), participants more often attributed learning to the fine-tuned model (18 vs. 13), and agreement between preference and learning attribution was low (39%).

Step labels generated by the model can be logged for analytics and personalization while optionally being hidden from the visible transcript, enabling supervision of the instructional process.

The demo system provides onboarding (asking users about experience, current courses, and AI attitudes), the chat interface, and a dashboard for scheduling consultations with live experts, managing collected resources, and storing user data.

Study Design & Method

Expert evaluations compared TeachingCoach with a GPT-4o baseline in a zero-shot setting, highlighting the impact of explicit pedagogical supervision and step-aware training independent of model scale or proprietary data. The user study with higher education instructors examined both overall preference and perceived learning, finding that participants consistently distinguished the models on conversational engagement, efficiency of interaction, and breadth of suggestions: the fine-tuned model was described as resembling dialogue with a human pedagogy expert through reflective questions and follow-up prompts, while the baseline was valued for generating responses quickly when immediate guidance was needed. The work positions TeachingCoach against prior systems that primarily support students, arguing that instructor-facing support grounded in pedagogical practice has been underserved.

Connected Concepts

Teacher AI Competency

Connected Articles

AI TPACK Teacher Multi Agent Workflow — Modeling AI-TPACK in Practice: Insights from Teachers'' Multi-Agent Workflow Design

AI Pedagogical Orientation — Faculty Orientations Shape Adoption of AI in Research and Teaching

Learner Centered Feedback AI — Enhancing learner-centered feedback with AI: teachers'' practices and perceptions

Teacher AI Adoption Confidence — AI Adoption Among Teachers: Insights on Concerns, Support, Confidence, and Attitudes

AI Generated Slides Student Perception — AI-Generated Slides: Are They Good? Can Students Tell?

Didactical Teacher Assistant Dimensional Modeling — A didactical-driven teacher assistant for a dimensional modeling course

Citation

Isabel Molnar et al. (2026). TeachingCoach: A Fine-Tuned Scaffolding Chatbot for Instructional Guidance to Instructors. arXiv:2603.18189. cs.AI.