EVOL: Simulator-Guided Evolutionary Expert Synthesis for Deployment-Free Learning Path Recommendation

Synthesis: EVOL addresses a structural obstacle in learning path recommendation (LPR): the policy must commit to a whole L-step sequence with reward only at the final step, and no human tutor has recorded an optimal path for each learner. The authors transplant a sim-to-real recipe from robotics, using a deep Knowledge Tracing model as a knowledge evolution simulator (KES) that both synthesizes per-learner expert demonstrations through evolutionary search and trains a deployment-free policy. An asymmetric actor-critic lets the actor plan blindly from the initial mastery vector while the critic sees privileged evolving mastery during training. Across three datasets (ASSIST15, Junyi, EdNet; 39–189 concepts) and path lengths L ∈ {5, 10, 20}, EVOL surpasses 8 baselines spanning heuristic, sequential, Reinforcement Learning, graph-enhanced RL and LLM-enhanced methods. Ablations show the gain comes from evolutionary expert quality rather than the imitation objective: behavioral cloning, AWR and DAPG finish within 0.01 EP of each other, while replacing evolutionary experts with random paths collapses performance.

Key Findings

1. EVOL leads in every dataset–length cell. Across ASSIST15, Junyi and EdNet at path lengths L ∈ {5, 10, 20}, at least one EVOL variant achieves the best result, with EVOL-BC reaching +0.614 EP on ASSIST15 L = 10.
2. Evolutionary experts, not the imitation objective, drive the gain. Behavioral cloning, AWR and DAPG land within 0.01 EP of each other across all nine ⟨dataset, L⟩ cells, so the downstream learner is interchangeable and expert quality is the operative variable.
3. Greedy and random experts fall far short. Under identical candidate sets and budgets, greedy 1-step lookahead reaches only +0.155 EP and random-path experts +0.354, against +0.614 for evolutionary search on ASSIST15 L = 10.
4. Sequencing carries most of the advantage. Uniformly shuffling recommended paths drops EVOL-BC from +0.627 to +0.330 EP on ASSIST15 L = 10, a SeqDrop of +0.297 that isolates order-sensitive structure.
5. Coverage and ordering both improve. EVOL-BC covers 0.876 of target concepts against 0.713 for PPO-vanilla, and its per-step lift reaches +0.022, matching the strongest variants.
6. The advantage survives simulator shift. Re-executed on four alternative DKT instances (seeds 7, 100, 200, 333), EVOL-BC holds a +0.341 cross-instance mean EP against +0.250 for PPO-vanilla.
7. Sparse reward, not architecture, is the bottleneck. PPO-vanilla, sharing EVOL's asymmetric critic and reward, plateaus near +0.51 and crosses EVOL's early level only around episode 6,000, while EVOL climbs to +0.62.

The deployment-free constraint and the sim-to-real analogy

Learning path recommendation means choosing and ordering L concepts from a curriculum of M so a learner's mastery of target concepts improves. The paper's first obstacle is that the deployed system cannot ask a student to practice an intermediate concept just to observe the resulting mastery, so the policy must commit to an entire L-step path from the initial mastery estimate h0 alone. The combinatorial space M^L · L! grows super-exponentially for M = 189 and L = 20, and reward arrives only at step L, making randomly initialized reinforcement learning extremely sample-inefficient. The second obstacle is that expert demonstrations—the usual cure for sparse-reward learning—do not exist in education, because student logs record what learners did, not what they should have done. The authors observe that a simulator plays the role of a physics engine, the learner the "robot", and a learning path the "trajectory", which lets evolutionary search synthesize the per-learner expert paths that human tutors cannot supply.

How EVOL works: evolutionary experts, cloning, and asymmetric PPO

EVOL runs in three training stages, all consulting the KES, while the shipped actor is simulator-free. Stage 1 evolves a population of 60 candidate paths over 50 generations inside the KES, using tournament selection, ordered crossover and point mutation to maximize Effectiveness Percentage (EP), the normalized mastery improvement on target concepts; the single best path per learner becomes an expert trajectory. Stage 2 distills these into a feed-forward actor by behavioral cloning conditioned only on h0, the target mask and the step index—never on simulator-updated intermediate mastery. Stage 2.5 warm-starts the critic on expert returns, and Stage 3 refines the actor with PPO under an asymmetric architecture: the actor consumes [h0 ∥ g ∥ t/L] while the critic consumes the privileged [ht ∥ g ∥ t/L]. This asymmetry gives the critic accurate advantage estimates during training while the deployed actor plans an entire path in L forward passes without querying the simulator. The underlying learner model is a deep knowledge tracing network whose validation AUCs were 0.70 on ASSIST15, 0.74 on Junyi and 0.76 on EdNet.

What the ablations and robustness tests show

The ablations form a clean causal story. Behavioral cloning alone is not enough—PPO-vanilla, trained from random weights, reaches +0.543 EP and beats imitation on random-path experts (+0.354)—and random experts actively bias the policy, so any source of demonstrations will not do. Evolutionary search reaches +0.614, a +0.459 EP edge over greedy one-step lookahead, which isolates the value of joint sequence optimization. The authors also probe pedagogical coherence: EVOL's advantage shrinks to near parity with PPO-vanilla once paths are shuffled, implying that it chiefly transfers order-sensitive structure, and its recommended paths cover more target concepts. Two stress tests—four alternative DKT instances and three stochastic response models—show EVOL remains strongest, weakening the worry that it merely exploits the deterministic response threshold r = 1[hc > 0.5]. Generation of the experts took 8.4 minutes on Junyi (8,000 learners), 31 minutes on ASSIST15 (5,827 learners) and 41 minutes on EdNet (19,880 learners), and the datasets span mathematics tutoring (ASSIST15, Junyi) and English Language Learning (EdNet).

What this means for practice

• Instructors. Treat the recommended sequence, not just the concept set, as the pedagogical object: EVOL-BC's +0.614 EP on ASSIST15 L = 10 versus +0.543 for the same architecture without expert pretraining shows that ordering decisions carry measurable mastery gains.
• Developers. Separate privileged search from deployment: synthesize demonstrations offline inside a simulator, then ship a policy that consumes only deployable inputs (h0, targets, step index), so no runtime simulator query is needed.
• Researchers. Score deployment-free performance explicitly; methods that query the simulator at inference measure a counterfactual oracle rather than a servable system, and the reported gaps here characterize the h0-only constraint.
• Learning analytics designers. Plan for simulator sensitivity: fixed graph-derived candidate structures degraded most across DKT instances, so validation across simulator instances is a practical readiness check.

Limitations

• Expert synthesis optimizes EP inside a DKT-based simulator, so the demonstrations inherit what the DKT fails to represent—most notably, two learners with identical histories are assigned identical dynamics regardless of differing aptitude.
• The study reports no prospective validation with live learners; the authors call for validation with human-designed curriculum constraints as an important next step.
• Baselines originally designed for inference-time simulator querying are evaluated under the stricter h0-only constraint, so the gaps compare deployment-free performance rather than the original methods under their own assumptions.
• Evaluation rests on simulated mastery (DKT) rather than measured real-world learning gains.
• Robustness is tested only within the DKT family, across four alternative instances (seeds 7, 100, 200, 333); other simulator families are untested.
• Results come from offline datasets (ASSIST15, Junyi, EdNet) with three random seeds {42, 123, 7}, not from a deployed tutoring system.

Citation

Bang, G., Kim, D., & Min, M. (2026). EVOL: Simulator-Guided Evolutionary Expert Synthesis for Deployment-Free Learning Path Recommendation. CIKM '26.