Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education

Synthesis

KITE (Knowledge-Informed Tutoring Engine) introduces a Intelligent Tutoring architecture that grounds its responses in course materials through a multimodal RAG pipeline. Unlike generic LLM tutors that may drift from curriculum content, KITE retrieves relevant material — lecture slides, problem sets, code examples — before generating Socratic hints, guiding questions, and progressive scaffolds tailored to the student's current error.

The system's intent-aware Socratic response strategy builds on Socratic AI Dialogue principles but operationalizes them through retrieval: rather than relying on the model's parametric knowledge of how to tutor, KITE anchors every response in authoritative course content. This design addresses a key gap identified in AI Tutor Effectiveness Review — that pedagogical quality alone is insufficient without content fidelity.

KITE was evaluated through three complementary lenses. RAGAs-based metrics confirmed response grounding and quality. Expert pedagogical review assessed the educational appropriateness of its Socratic scaffolding. Most innovatively, a simulated student pipeline — where a weaker LLM acts as a student across two-turn dialogues — measured whether KITE's feedback actually improved follow-up accuracy on procedural and algorithm tracing questions. Results were positive: simulated students produced more accurate responses after receiving KITE feedback, particularly on tracing tasks where step-by-step reasoning is critical.

This work contributes to the growing literature on LLM-based student modeling by demonstrating that retrieval-augmented tutoring can maintain both pedagogical and content fidelity. It also connects to Programming ITS research by targeting algorithmic reasoning specifically, and to Personalized Learning through its adaptive response strategies. The simulated student evaluation approach offers a replicable, low-cost methodology that could accelerate tutor development cycles — a theme also explored in Eduagentbench Agent Teaching Benchmark for broader tutor evaluation.

Connected Concepts

Connected Articles

Citation

Jain, M., Bhatt, T., Pitts, G., Pandya, A., Brusilovsky, P., Norouzi, N., Hellas, A., Leinonen, J., & Akram, B. (2026). Retrieval-Augmented Tutoring for Algorithm Tracing and Problem-Solving in AI Education. arXiv:2605.12988. BEA 2026.