Advancing diagram-based reasoning in AI tutoring systems: a structural approach for STEM education

Synthesis: Presents StructRAG, a pattern-aware framework that improves how AI tutoring systems interpret complex engineering diagrams (circuit schematics, network topologies, block flowcharts) in STEM. Current AI systems fail on diagrammatic questions because they cannot reliably extract spatial relationships and structural connectivity from noisy real-world diagrams (curved lines, overlapping elements, occluded labels). StructRAG shifts from surface-level visual recognition to pattern-level structural reasoning: convert the diagram to a graph, retrieve structurally similar canonical patterns, and prompt an LLM to validate plausibility and completeness — enabling detection of missing connections and structure-aware feedback.

Architecture (four components)

1. Diagram-to-Graph Conversion — OCR (Tesseract, multi-scale 1.0/1.5/2.0, majority voting) + classical CV (Hough Line Transform, contour detection, Zhang–Suen skeletonisation for curved paths). Edge confidence Cij = 0.45·scont + 0.25·sprox + 0.20·salign + 0.10·snode; edges ≥ 0.65 accepted, 0.40–0.65 routed to an uncertain-edge set U for pattern-level validation, < 0.40 discarded as noise.
2. Structural Pattern Retrieval — abstracts the recognized graph into topology patterns (star, ring, chain/bus, bridge/mesh, tree, hybrid, cross-layer) and retrieves similar templates from a curated library using graph-edit-distance (GED) matching — a retrieval strategy that outperforms node-level statistics and example-based retrieval.
3. Pattern-Aware Prompt Construction — feeds the LLM the graph G, uncertain edges U, retrieved templates, and candidate corrections ΔE.
4. LLM-Guided Structural Reasoning and Correction — GPT-4 jointly interprets G, U, T*, and ΔE to decide which uncertain edges to add and which structures to correct, with output ensembling.

Results (1,650 STEM diagram-based questions)

• RQ1 (accuracy): macro-average question-level accuracy 89.3% (95% CI [86.8, 91.9]) — beating OCR+CV only (70.8%), GPT-4 graph-only (81.9%), direct-image GPT-4o (84.8%), and StructRAG-without-LLM (82.7%); significant after Holm–Bonferroni correction (p < 0.01).
• RQ2 (missing-connection recovery): micro-averaged edge-level F1 = 93.0% (CI [91.8, 94.2]) — best in every topology category; largest gains over OCR+CV on irregular (26.4 pts), bridge/mesh (23.1), and ring (21.6) topologies.
• RQ3 (retrieval strategy): pattern-level retrieval beats node-level statistics and example-based retrieval.
• Best on star (94.5%) and chain/bus (93.1%); hardest on bridge/mesh (85.8%) and unclassified/irregular (81.4%) diagrams — yet still best-in-class there.
• Component ablation (Table 10): full pipeline 89.3/93.0 vs pattern-aware GPT-4 without ensembling 87.8/91.4 and template retrieval + GED without LLM 82.7/87.6 — every component contributes.

Interpretation

• Question-level accuracy is always lower than edge-level F1 because Acc demands a perfect whole-diagram match, while F1 captures partial correctness — relevant for education, where identifying most missing connections already yields useful structural Feedback.
• Grounded in cognitive apprenticeship: instructors guide students through visual Problem Solving by highlighting analogies, missing links, and logical dependencies; StructRAG's template retrieval mirrors that by making structural relationships explicit and auditable (an intermediate graph representation, rather than end-to-end visual matching). This reduces hallucination risk and strengthens student trust relative to end-to-end Multimodal AI parsing.
• Interviews with 12 STEM educators were positive: actionable structural feedback, adaptability across diagram types, and help explaining missing/ambiguous connections to students struggling with abstract topologies.
• Limitations: dataset withheld (university teaching materials; schema/code/prompts released), and no learner-outcome study yet — future work should test effects on learning, retention, transfer, Motivation, and cognitive load in live tutoring contexts.

What this means for practice

• Designers. Make diagram feedback structural rather than binary: because edge-level F1 stays high (93.0%) where whole-diagram accuracy fails (89.3%), a system can name the specific missing or misrecognized connections even when the overall diagram is imperfect — the gains over OCR+CV are largest on irregular (26.4 points), bridge/mesh (23.1), and ring (21.6) topologies.
• Designers. Retrieve canonical topology patterns by graph-edit-distance matching against a template library instead of relying on raw image parsing, node-level statistics, or example-based retrieval, and keep the intermediate graph representation explicit so recognition errors remain auditable.
• Designers. Keep outputs human-gradable: the corrected JSON graph and its short justification are inspectable by instructors and students, supporting explainable Automated Assessment of diagrammatic submissions rather than opaque auto-grading.
• Instructors. Use the corrections as hints: identifying most of the missing connections already yields useful structure-aware feedback for students who struggle with abstract topologies, which was the main appeal reported by the 12 STEM educators interviewed.

Limitations

• Evaluation is benchmark-only: 1,650 STEM diagram-based questions scored with GPT-4 (gpt-4-0613) and GPT-4o baselines, with no learner-outcome study, so effects on learning, retention, transfer, motivation, and cognitive load are untested — future live-tutoring work is called for.
• Performance is uneven across topologies: accuracy falls to 85.8% on bridge/mesh and 81.4% on unclassified/irregular diagrams, and question-level accuracy requires an exact whole-diagram node-and-edge match, so partial failures are penalized in the main metric.
• The evaluation dataset is withheld (it comes from university teaching materials; only schema, code, and prompts are released), so the reported numbers cannot be reproduced on the same items.
• Ground truth was annotated by 12 STEM educators under a partially overlapping two-stage protocol, with each sample first checked independently by two educators and disagreements flagged as inconsistent, so the benchmark labels embed rater judgment.

Citation

Sun, Y., Liao, Y., & Ma, X. (2026). Advancing diagram-based reasoning in AI tutoring systems: A structural approach for STEM education. Smart Learning Environments. Advance online publication