PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning

Synthesis: Most learning path recommendation infers the next step from item-level interaction logs, which means two learners with identical correctness records get identical advice even when their goals differ. PersonaPath reframes the task as Knowledge-Centric planning: given an explicit learner persona, a mastery state and a stated target unit, decide which textbook, unit and concept to study next so that prerequisite structure is respected and the goal is actually reached. The benchmark pairs 2,000 synthetic personas with a hierarchical knowledge graph of 347 textbooks, 1,751 units and 4,092 concepts across 77 subjects, and the results are sobering — the best model reaches a 29.5% final pass rate in Basic Education and no model exceeds 44.7% on adaptivity, even though several score above 90% on structural validity. The ablations locate the failure precisely: mastery information and step-by-step feedback are what models cannot do without, and the prevailing pattern is a planner that produces a defensible curriculum and ignores the learner it was given.

From exercise-centric recommendation to knowledge-centric planning

The paper's opening example is the argument for the whole enterprise. Two students attempt the same three vector exercises and produce identical records — both solve the vector-coordinate item, both miss vector addition and dot product. A recommender conditioned on that record would assign them the same drills. But Student A's target is straight-line equations and Student B's is the concept of vector, and the paths they need diverge immediately. Distinguishing them requires three things jointly: the learner's goal, the learner's current knowledge state, and a curriculum-scale prerequisite structure connecting the two. Existing public benchmarks supply interaction logs (ASSISTments 2009 and 2012, Junyi Academy, OLI Engineering Statics, synthetic Deep Knowledge Tracing data) and support exercise-level prediction, but none combines explicit personas, long-term goals and prerequisites.

The reframing from Exercise-Centric (EC) to Knowledge-Centric (KC) is therefore a change in the unit of the decision: not which exercise to practice next but which knowledge to study next, and why. Where prior work in this space — collaborative filtering, ant colony optimization, immune and genetic algorithms, and more recently deep Reinforcement Learning policy learning — optimizes over items, PersonaPath asks a planner to operate over a structured knowledge space while a simulated learner's state evolves in response to each choice.

The benchmark: personas, goals and a hierarchical knowledge graph

The knowledge space is a three-level hierarchy formalized as a directed graph: textbooks (347), the units inside them (1,751), and atomic concepts (4,092), with 411 prerequisite edges between textbook nodes from which unit- and concept-level prerequisites are induced. Basic-education textbooks come from the People's Education Press and higher-education resources from the Smart Education of China platform; units and concepts were extracted by DeepSeek-V3 in a hierarchical pass (tables of contents first, then atomic concepts within each unit) and then manually corrected by domain experts. Structure grows with educational stage, from 930 concepts at primary level to 1,601 at university, and basic-education subjects are wired as prerequisites for related higher-education majors (chemistry and biology before clinical medicine), which is what makes long-horizon cross-stage planning testable at all.

Quality control is reported with unusual specificity: graph analysis verified strict DAG compliance with zero circular dependencies across the 347 textbook nodes and 411 edges; expert verification on a validation subset yielded Cohen's Kappa of 0.93 ("almost perfect"); and after correction the verified prerequisite relations reached 99.5% precision. Two experienced educators then rated 100 randomly sampled personas on a five-point scale, giving means of 4.68 for curriculum consistency, 4.37 for progression plausibility, 4.33 for persona coherence and 4.11 for mastery plausibility, with ICC(2,2) of 0.85, 0.82, 0.79 and 0.74 respectively — mastery plausibility being both the lowest-rated and least reliably rated dimension, which is worth remembering when reading the benchmark's adaptivity results.

Personas follow the synthetic-student tradition: each is defined by an academic goal (for example, a target major) plus a structured mastery state, with learners stratified into Low-, Average- and High-performing archetypes in an approximate 1:3:1 ratio that mirrors tertile bins of the normal ability distribution used in item response theory. A current grade level splits the curriculum into learned, learning and unlearned regions and fixes both the planning horizon and the starting state. The released cohort is 2,000 personas — 1,000 basic and 1,000 higher education — with 620 and 609 average-performing learners respectively. Mastery then evolves through an IRT-based simulator: a logistic performance model whose discrimination parameter and difficulty-utility coefficients were tuned so an average learner needs roughly 3–5 interactions per unit, with a Zone-of-Proximal-Development challenge bonus for succeeding above current proficiency, and a sensitivity analysis (1,000 Monte Carlo runs per configuration) confirming that the Low > Average > High ordering of steps to mastery is preserved across all 25 tested parameter configurations.

Evaluation design: validity, adaptivity, efficiency

Planning is scored on three dimensions that must all hold for a path to pass. Validity covers prerequisite violation (a concept appearing before its antecedents) and hallucination (an entity outside the knowledge space). Adaptivity covers persona alignment and difficulty adaptability, measured by Cog-Gap — the mean absolute difference between unit mastery and the difficulty of each selected concept along the path, capturing both under- and over-challenging choices. Efficiency covers goal completion within a step budget and progress continuity (the average proficiency increment over a five-step sliding window). A path earns a Final Pass only as the product of all three, so a structurally impeccable path that ignores the learner scores zero.

The main protocol is a closed loop: the model selects a concept, the simulator updates the corresponding unit's mastery, and the updated state is supplied for the next decision. Ten Open Source models from 1B to 30B+ parameters were evaluated under both zero-shot and chain-of-thought prompting, on Huawei Ascend hardware with vLLM at bfloat16, temperature 0.1 and a 32,768-token generation budget.

Key findings

1. Even the strongest model falls far short of the task: DeepSeek-V3.1 reaches a Final Pass Rate of 29.5% in Basic Education and 14.6% in Higher Education.
2. The bottleneck is adaptivity, not structure. DeepSeek-V3.1 reaches 90.9% on Validity in Basic Education but only 44.3% on Adaptivity, and no model exceeds 44.7% on Adaptivity in either split (best: Qwen3-30B-A3B with CoT, Basic Education).
3. Performance halves in Higher Education across nearly every metric: DeepSeek-V3.1 drops from 29.5% to 14.6% Final Pass, Qwen3-30B-A3B (CoT) from 27.7% to 10.8%, and Qwen3-4B-2507 (CoT) from 20.8% to 5.5%.
4. Chain-of-thought prompting changes results unevenly and sometimes negatively. Qwen3-30B-A3B rises from 10.0% to 27.7% Final Pass and from 21.9% to 44.7% Adaptivity in Basic Education, while Llama-3.1-8B's Validity declines from 19.2% to 14.6% and its Final Pass from 1.3% to 0.8%; Llama-3.2-1B and Llama-3.1-8B never exceed 1.3% Final Pass across either stage.
5. Retrieval augmentation is a validity intervention only. Adding top-3 retrieved knowledge chunks to Qwen3-4B-2507 (zero-shot, Basic Education) lifts Validity from 39.7% to 73.8% (+34.1 points) and Adaptivity from 12.5% to 19.3%, but Efficiency falls from 25.8% to 18.4% and Final Pass moves from 3.2% to 2.9% — supplying the right knowledge does not make the resulting sequence better for the learner.
6. Domain specialization is not a cure: InnoSpark-7B, an education-tuned model, reaches 41.3% Validity (better than same-sized baselines) while lagging them on Adaptivity, a "specialist" profile skewed toward curriculum-aligned output at the cost of learner-conditioned reasoning.
7. Scaling is not monotonic: Qwen3-4B-2507 (CoT) slightly exceeds Qwen3-30B-A3B (CoT) on Efficiency (60.2% vs. 59.4%), while model size generally does help Final Pass (Pangu-7B 15.0% vs. Pangu-1B 1.1% in Basic Education).

Ablations: what the models are actually missing

Three controlled experiments isolate the inputs. Removing the mastery field from the persona (keeping the goal and curriculum intact) leaves Validity almost unchanged — Qwen3-4B-2507 (CoT) even gains 0.2 points — while Adaptivity falls for every model tested, by up to 26.1 points for Qwen3-30B-A3B (CoT). The authors' reading is cognitive load: complex mastery constraints force deviation from canonical teaching sequences, and without them models revert to the standard ordering that dominates pretraining data. Fine-grained learner modeling is what personalization needs; generic logical correctness does not require it.

Injecting contextual noise (irrelevant textbook titles in the candidate pool) degrades everything, most of all Validity — Qwen3-30B-A3B (CoT) loses 54.9 points and Qwen3-4B-2507 (CoT) loses 43.2 — alongside Efficiency losses up to 35.7 points. Since the ablations hold language and curriculum fixed, the drop is about selecting the relevant part of the knowledge structure, not about domain knowledge.

Replacing step-by-step interaction with one-shot generation produces the most revealing trade-off. Generating the entire path in a single turn raises Validity sharply (+30.8 points for Qwen3-4B-2507 (CoT)) while Adaptivity falls by up to 28.8 points and Efficiency by up to 33.8 points, because the sequence is fixed before any mastery feedback arrives. Structural validity is obtainable from the curriculum alone; adaptation requires the loop.

The three hand-constructed error cases in the appendix make the failure modes concrete for a basic-education learner targeting "Four Operations": jumping to the target textbook while three units of the prerequisite textbook remain unmastered (validity), repeatedly selecting the easiest available concept at difficulty 0.1 against a mastery of 0.75 for a Cog-Gap of 0.65 (adaptivity), and repeatedly selecting the hardest concept at difficulty 0.4 for a mastery gain of +0.0085 per round, so that by round 20 the learner is still in the same grade-3 textbook (efficiency).

What this means for practice

• Designers. Run the planner closed-loop and update mastery after every selected concept. Generating a whole path in one turn raised validity by up to 30.8 points while adaptivity fell by up to 28.8 points, because the sequence is fixed before any mastery feedback arrives.
• Designers. Pass the mastery state explicitly and verify that it changes the output. Ablating the field left validity nearly unchanged while adaptivity dropped by up to 26.1 points, so a planner can look curriculum-aligned while effectively ignoring the learner it claims to personalize for.
• Designers. Keep a teacher reviewing generated sequences before they reach students. The strongest model reached only a 29.5% final pass rate in basic education, and prerequisite violations and hallucinated entities appeared often enough to be scored as separate validity failures.
• Researchers. Report validity, adaptivity and efficiency separately instead of one pass rate. No model exceeded 44.7% adaptivity even when validity ran above 90%, and the multiplicative final-pass product hides which dimension failed.
• Researchers. Add retrieval as a validity intervention, not a personalization fix. Top-3 retrieved knowledge chunks lifted validity from 39.7% to 73.8% for Qwen3-4B-2507 while efficiency fell from 25.8% to 18.4% and the final pass rate barely moved (3.2% to 2.9%).

Limitations

• Learner states and mastery updates are synthetic. Mastery evolves through an IRT-based simulator tuned so an average learner needs roughly 3–5 interactions per unit, so benchmark scores describe planner behavior against a modeled learner rather than outcomes for real students.
• The least reliable persona dimension is the one adaptivity depends on. Two educators rated 100 randomly sampled personas, and mastery plausibility drew both the lowest mean (4.11) and the lowest agreement (ICC(2,2) = 0.74) of the four rated dimensions, so expert judgment establishes pedagogical plausibility and internal coherence rather than a link to real learner outcomes.
• The knowledge graph is built from Chinese curricula — People's Education Press textbooks plus the Smart Education of China platform — with units and concepts extracted by DeepSeek-V3 and then corrected by domain experts; reuse in another language or system requires replacing the corpus, re-extracting the hierarchy and re-validating prerequisites.
• Model rankings were produced under a single simulator configuration, and the authors note that testing rankings under alternative configurations would require rerunning every planner.

Citation

Liu, Y., Liu, Z., Zhang, T., Cheng, Z., Guo, Y., Chen, K., Zhang, M., Wang, Y., & Wang, H. (2026). PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning. AACL-IJCNLP 2026 Main Conference.