Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning

Synthesis: This paper introduces an evidence-grounded multimodal pipeline that constructs provenance-rich knowledge graphs from lecture videos by integrating speech transcripts, slide OCR, and vision-language model analysis. Processing three neural-network lectures, the pipeline extracted 172 canonical concepts and 282 typed relationships with 90.38% endpoint coverage, achieving perfect retrieval accuracy on a preliminary test. The approach addresses a key challenge in educational AI: converting rich multimodal lecture content into structured, queryable knowledge representations without losing the evidential provenance that makes them trustworthy.

Pipeline Architecture

The multimodal pipeline processes lecture videos through several stages:

1. Transcription: Speech-to-text conversion of lecture audio

2. Semantic Anchor Selection: Identification of key concept-bearing segments

3. OCR Extraction: Text extraction from slide content and diagrams

4. Vision-Language Analysis: Concept and relationship extraction with evidential grounding

5. Validation and Canonicalization: Cross-referencing mentions against multiple evidence sources

6. Knowledge Graph Construction: Typed relationships with provenance tracking

Key Metrics

Stage	Volume
Frames processed	3,118
Transcript segments	756
Semantic anchors	559
Concept mentions retained	1,022
Relationship mentions	312 (corrected from paper)
Canonical concepts	172
Typed relationships	282
Endpoint coverage	90.38%
Retrieval top-1 accuracy	100%
Retrieval top-5 recall	100%

Significance for Educational AI

The auditable construction method is the key contribution — unlike black-box knowledge extraction, every concept and relationship in the graph is traceable to specific evidence (transcript segment, OCR text, or visual element). This provenance tracking is critical for educational applications where:

Accuracy matters: Incorrect concept relationships can mislead student models

Adaptive Learning systems need trustworthy prerequisite relationships

Lecture review tools benefit from structured, queryable knowledge representations

Multi-lecture reasoning requires connecting concepts across different instructional sessions

Connected Concepts

Connected Articles

Proprl Prerequisite Relation Learning — ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs

Xie Hillm Cd 2026 — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis

Skill Acquisition Without Temporal Info — Estimating Learners' Skill Acquisition Without Temporal Information

Pattern Kc Programming Recommendation — Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components

LLM Item Difficulty Prediction — Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction

Leveraging Complex Systems Leading For Transformative Change — Leveraging complex systems: Leading for transformative change

Citation

Al Farib, S., Meem, M. A., Islam, S. R., & Raihan, M. T. (2026). Evidence-Grounded Multimodal Knowledge Graph Construction for Multi-Lecture Educational Reasoning. arXiv:2608.03161v1.