PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning

Synthesis: Most learning path recommendation infers the next step from item-level interaction logs, which means two learners with identical correctness records get identical advice even when their goals differ. PersonaPath reframes the task as Knowledge-Centric planning: given an explicit learner persona, a mastery state and a stated target unit, decide which textbook, unit and concept to study next so that prerequisite structure is respected and the goal is actually reached. The benchmark pairs 2,000 synthetic personas with a hierarchical knowledge graph of 347 textbooks, 1,751 units and 4,092 concepts across 77 subjects, and the results are sobering — the best model reaches a 29.5% final pass rate in Basic Education and no model exceeds 44.7% on adaptivity, even though several score above 90% on structural validity. The ablations locate the failure precisely: mastery information and step-by-step feedback are what models cannot do without, and the prevailing pattern is a planner that produces a defensible curriculum and ignores the learner it was given.

From exercise-centric recommendation to knowledge-centric planning

The paper's opening example is the argument for the whole enterprise. Two students attempt the same three vector exercises and produce identical records — both solve the vector-coordinate item, both miss vector addition and dot product. A recommender conditioned on that record would assign them the same drills. But Student A's target is straight-line equations and Student B's is the concept of vector, and the paths they need diverge immediately. Distinguishing them requires three things jointly: the learner's goal, the learner's current knowledge state, and a curriculum-scale prerequisite structure connecting the two. Existing public benchmarks supply interaction logs (ASSISTments 2009 and 2012, Junyi Academy, OLI Engineering Statics, synthetic Deep Knowledge Tracing data) and support exercise-level prediction, but none combines explicit personas, long-term goals and prerequisites.

The reframing from Exercise-Centric (EC) to Knowledge-Centric (KC) is therefore a change in the unit of the decision: not which exercise to practice next but which knowledge to study next, and why. Where prior work in this space — collaborative filtering, ant colony optimization, immune and genetic algorithms, and more recently deep Reinforcement Learning policy learning — optimizes over items, PersonaPath asks a planner to operate over a structured knowledge space while a simulated learner's state evolves in response to each choice.

The benchmark: personas, goals and a hierarchical knowledge graph

The knowledge space is a three-level hierarchy formalized as a directed graph: textbooks (347), the units inside them (1,751), and atomic concepts (4,092), with 411 prerequisite edges between textbook nodes from which unit- and concept-level prerequisites are induced. Basic-education textbooks come from the People's Education Press and higher-education resources from the Smart Education of China platform; units and concepts were extracted by DeepSeek-V3 in a hierarchical pass (tables of contents first, then atomic concepts within each unit) and then manually corrected by domain experts. Structure grows with educational stage, from 930 concepts at primary level to 1,601 at university, and basic-education subjects are wired as prerequisites for related higher-education majors (chemistry and biology before clinical medicine), which is what makes long-horizon cross-stage planning testable at all.

Quality control is reported with unusual specificity: graph analysis verified strict DAG compliance with zero circular dependencies across the 347 textbook nodes and 411 edges; expert verification on a validation subset yielded Cohen's Kappa of 0.93 ("almost perfect"); and after correction the verified prerequisite relations reached 99.5% precision. Two experienced educators then rated 100 randomly sampled personas on a five-point scale, giving means of 4.68 for curriculum consistency, 4.37 for progression plausibility, 4.33 for persona coherence and 4.11 for mastery plausibility, with ICC(2,2) of 0.85, 0.82, 0.79 and 0.74 respectively — mastery plausibility being both the lowest-rated and least reliably rated dimension, which is worth remembering when reading the benchmark's adaptivity results.

Personas follow the synthetic-student tradition: each is defined by an academic goal (for example, a target major) plus a structured mastery state, with learners stratified into Low-, Average- and High-performing archetypes in an approximate 1:3:1 ratio that mirrors tertile bins of the normal ability distribution used in item response theory. A current grade level splits the curriculum into learned, learning and unlearned regions and fixes both the planning horizon and the starting state. The released cohort is 2,000 personas — 1,000 basic and 1,000 higher education — with 620 and 609 average-performing learners respectively. Mastery then evolves through an IRT-based simulator: a logistic performance model whose discrimination parameter and difficulty-utility coefficients were tuned so an average learner needs roughly 3–5 interactions per unit, with a Zone-of-Proximal-Development challenge bonus for succeeding above current proficiency, and a sensitivity analysis (1,000 Monte Carlo runs per configuration) confirming that the Low > Average > High ordering of steps to mastery is preserved across all 25 tested parameter configurations.

Evaluation design: validity, adaptivity, efficiency

Planning is scored on three dimensions that must all hold for a path to pass. Validity covers prerequisite violation (a concept appearing before its antecedents) and hallucination (an entity outside the knowledge space). Adaptivity covers persona alignment and difficulty adaptability, measured by Cog-Gap — the mean absolute difference between unit mastery and the difficulty of each selected concept along the path, capturing both under- and over-challenging choices. Efficiency covers goal completion within a step budget and progress continuity (the average proficiency increment over a five-step sliding window). A path earns a Final Pass only as the product of all three, so a structurally impeccable path that ignores the learner scores zero.

The main protocol is a closed loop: the model selects a concept, the simulator updates the corresponding unit's mastery, and the updated state is supplied for the next decision. Ten Open Source models from 1B to 30B+ parameters were evaluated under both zero-shot and chain-of-thought prompting, on Huawei Ascend hardware with vLLM at bfloat16, temperature 0.1 and a 32,768-token generation budget.

Key findings

1. Even the strongest model falls far short of the task: DeepSeek-V3.1 reaches a Final Pass Rate of 29.5% in Basic Education and 14.6% in Higher Education.
2. The bottleneck is adaptivity, not structure. DeepSeek-V3.1 reaches 90.9% on Validity in Basic Education but only 44.3% on Adaptivity, and no model exceeds 44.7% on Adaptivity in either split (best: Qwen3-30B-A3B with CoT, Basic Education).
3. Performance halves in Higher Education across nearly every metric: DeepSeek-V3.1 drops from 29.5% to 14.6% Final Pass, Qwen3-30B-A3B (CoT) from 27.7% to 10.8%, and Qwen3-4B-2507 (CoT) from 20.8% to 5.5%.
4. Chain-of-thought prompting changes results unevenly and sometimes negatively. Qwen3-30B-A3B rises from 10.0% to 27.7% Final Pass and from 21.9% to 44.7% Adaptivity in Basic Education, while Llama-3.1-8B's Validity declines from 19.2% to 14.6% and its Final Pass from 1.3% to 0.8%; Llama-3.2-1B and Llama-3.1-8B never exceed 1.3% Final Pass across either stage.
5. Retrieval augmentation is a validity intervention only. Adding top-3 retrieved knowledge chunks to Qwen3-4B-2507 (zero-shot, Basic Education) lifts Validity from 39.7% to 73.8% (+34.1 points) and Adaptivity from 12.5% to 19.3%, but Efficiency falls from 25.8% to 18.4% and Final Pass moves from 3.2% to 2.9% — supplying the right knowledge does not make the resulting sequence better for the learner.
6. Domain specialization is not a cure: InnoSpark-7B, an education-tuned model, reaches 41.3% Validity (better than same-sized baselines) while lagging them on Adaptivity, a "specialist" profile skewed toward curriculum-aligned output at the cost of learner-conditioned reasoning.
7. Scaling is not monotonic: Qwen3-4B-2507 (CoT) slightly exceeds Qwen3-30B-A3B (CoT) on Efficiency (60.2% vs. 59.4%), while model size generally does help Final Pass (Pangu-7B 15.0% vs. Pangu-1B 1.1% in Basic Education).

Ablations: what the models are actually missing

Three controlled experiments isolate the inputs. Removing the mastery field from the persona (keeping the goal and curriculum intact) leaves Validity almost unchanged — Qwen3-4B-2507 (CoT) even gains 0.2 points — while Adaptivity falls for every model tested, by up to 26.1 points for Qwen3-30B-A3B (CoT). The authors' reading is cognitive load: complex mastery constraints force deviation from canonical teaching sequences, and without them models revert to the standard ordering that dominates pretraining data. Fine-grained learner modeling is what personalization needs; generic logical correctness does not require it.

Injecting contextual noise (irrelevant textbook titles in the candidate pool) degrades everything, most of all Validity — Qwen3-30B-A3B (CoT) loses 54.9 points and Qwen3-4B-2507 (CoT) loses 43.2 — alongside Efficiency losses up to 35.7 points. Since the ablations hold language and curriculum fixed, the drop is about selecting the relevant part of the knowledge structure, not about domain knowledge.

Replacing step-by-step interaction with one-shot generation produces the most revealing trade-off. Generating the entire path in a single turn raises Validity sharply (+30.8 points for Qwen3-4B-2507 (CoT)) while Adaptivity falls by up to 28.8 points and Efficiency by up to 33.8 points, because the sequence is fixed before any mastery feedback arrives. Structural validity is obtainable from the curriculum alone; adaptation requires the loop.

The three hand-constructed error cases in the appendix make the failure modes concrete for a basic-education learner targeting "Four Operations": jumping to the target textbook while three units of the prerequisite textbook remain unmastered (validity), repeatedly selecting the easiest available concept at difficulty 0.1 against a mastery of 0.75 for a Cog-Gap of 0.65 (adaptivity), and repeatedly selecting the hardest concept at difficulty 0.4 for a mastery gain of +0.0085 per round, so that by round 20 the learner is still in the same grade-3 textbook (efficiency).

Limitations and what it means in practice

The authors state the central limitation plainly: learner states and mastery updates are synthetic, so expert evaluation establishes the pedagogical plausibility and internal coherence of 100 sampled personas, not the relationship between benchmark scores and real learner outcomes. Closing that gap requires longitudinal traces and classroom studies. The benchmark is also grounded in Chinese textbooks and curricula, and cross-language or cross-curriculum transfer would require replacing the source corpus, re-extracting the hierarchy and re-validating prerequisites with domain experts — the pipeline itself, including the simulator and the three evaluation dimensions, is presented as language-agnostic but needing curriculum-specific calibration.

The practice implication is a designed-in warning rather than a capability claim. Because models produce hallucinated entities and prerequisite violations with measurable frequency, the paper argues that generated paths should be reviewed by a teacher before classroom use, and that educational applications should retain human oversight of both content selection and learning goals. For institutions building adaptive systems, the operational lesson is that a planner evaluated only on curriculum conformance will look far better than it is: validity is cheap, adaptivity is the hard part, and the mastery signal — the thing the system claims to personalize on — is exactly the input whose removal changes almost nothing on the metric that looks best.

Connected Concepts

• Recommender Systems and Learning Paths — The decision layer this benchmark evaluates
• Personalized Learning — Goal-directed personalization as the open problem
• Knowledge Graph — The textbook–unit–concept hierarchy and its prerequisites
• Adaptive Learning — Adaptation to learner state as the bottleneck
• Learner Modeling and Adaptive Instruction — Personas, archetypes and mastery vectors
• Curriculum Design — Prerequisite ordering as an evaluated constraint
• Mastery Learning — Unit-level mastery thresholds and step budgets
• Item Response Theory — The simulator's ability–difficulty model
• Cognitive Diagnosis — Learner-state estimation feeding a planner
• Knowledge Tracing — The state-estimation tradition the benchmark supplies as input
• Hallucination Risk — Fictitious textbooks and concepts as a validity failure
• Human-in-the-Loop — Teacher review of generated paths
• Higher Education — The higher-education split is markedly harder
• K-12 — Basic education as the benchmark's largest split

Connected Articles

• Bayesian cognitive diagnosis optimizes personalized learning paths via mediation of cognitive load and Hidden Markov Model state transitions — Bayesian DINA diagnosis and shortest remediation paths, mediated by cognitive load
• ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs — Prerequisite relation learning with an irreversibility constraint
• Learning paths and patterns in learning design: Insights from 500+ courses — Markov and pattern-mining analysis of 29,064 designed activities across 554 courses
• Fair and explainable educational recommendations with a hybrid Graph-GRU framework — Heterogeneous knowledge graph plus GRU recommender with fairness and counterfactual explainability
• Detecting Knowledge Gaps from Conversational AI Interactions Using Curriculum Prerequisite Graphs — Mapping AI TA questions to a prerequisite graph to find curriculum-level gaps
• Automated Recommendation of Programming Learning Content Using Pattern-based Knowledge Components — Recommending programming practice by pattern-based knowledge-component similarity
• EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners — A long-horizon benchmark for pedagogical LLM agents with simulated learners
• The personalization paradox: how AI-driven adaptive learning environments are associated with college students' — How perceived personalization can reduce self-regulated learning
• PersonaVLM: Long-Term Personalized Multimodal LLMs — Long-term learner personalization with multimodal profiles
• Rethinking Higher Education: From Fixed Curricula to Learnity Graphs — Learnity graphs as a lifelong-learning alternative to fixed curricula

Citation

Liu, Y., Liu, Z., Zhang, T., Cheng, Z., Guo, Y., Chen, K., Zhang, M., Wang, Y., & Wang, H. (2026). PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning. AACL-IJCNLP 2026 Main Conference.