RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning

Synthesis: Long-term personalized tutoring asks a language model to turn a learner's accumulated history into instruction, and most systems do this with memory and retrieval while leaving the pedagogical rationale implicit. RegulaTutor makes that rationale explicit: a staged harness in which learner-facing teaching policies are derived from, and stay linked to, the historical evidence that supports them, guided by self-regulated learning research and a cognitive, affective, metacognitive and motivational (CAMM) framework. Across six backbones evaluated on a 1,000-case human-annotated tutoring Benchmark, adding the harness raised historical-evidence accuracy by a mean of 25.7 points and learner diagnosis by 21.8 points on local Qwen checkpoints, while the effect on the judged quality of teaching was small and uneven. The gains concentrate on weaker backbones and can reverse on stronger ones.

Key Findings

1. The harness lifted evidence acquisition sharply on weaker models. Qwen3.5-4B scored 20.0 on historical evidence alone and 56.2 with RegulaTutor (a 36.2-point gain), with a paired 95% interval excluding zero.
2. Diagnosis improved as much as evidence. The same backbone moved from 29.7 to 50.8 on four-class learner diagnosis, and the mean gain across four local Qwen checkpoints was 21.8 points.
3. Teaching quality moved only slightly. The mean gain on the 1–5 teaching rubric across the local checkpoints was 0.016, so better evidence handling did not automatically translate into better instruction.
4. Stronger backbones gained less, and two lost ground on evidence. DeepSeek-v4-pro gained 10.9 points on evidence, Gemini-3.7-flash 18.2, but GPT-5.6-sol fell 4.87 points and Kimi-k3 was flat (−1.23).
5. Diagnosis could regress even where evidence improved. Gemini-3.7-flash gained 18.2 points on evidence while its diagnosis fell 7.9 points, showing the stages do not move together.
6. Refinement was driven by failure routing. PairRoute distinguished harness, model and mixed failures, revised the harness first, and only then built reviewed supervision for the model failures that survived under the selected harness.

What the harness does

RegulaTutor organizes tutoring into stages that connect three things: the historical evidence drawn from a learner's record, an interpretation of that learner, and the learner-facing instruction that follows. Teaching policies retain links to the evidence supporting them and are updated as new observations arrive, so a later turn can show why the system chose a particular strategy. The design is grounded in self-regulated learning and the CAMM framework, and its refinement procedure, PairRoute, uses intermediate execution records to tell apart failures caused by the harness from failures caused by the model. That separation matters because it decides whether the fix is an interface change or more supervised training.

What the benchmark measured

Evaluation used LongTutor's 1,000-case human-annotated split, which supplies 199-record learner histories and scores three things separately: historical evidence acquisition (information extraction, multi-session reasoning and hallucination control), four-class learner diagnosis, and teaching quality averaged over history utilization, strategy alignment, coherence and appropriateness. Each backbone was compared with and without the harness, with paired case-level bootstrap intervals over the same 1,000 cases. The harness improved evidence and diagnosis almost everywhere on the local models but the teaching rubric barely moved, which the authors report without claiming that training on better evidence alone produces better teaching. The mechanisms studied here — evidence-linked policies, staged diagnosis, failure routing — are the same shape as the tutoring and student modeling machinery other systems use, but kept auditable by the explicit evidence links.

Why the result is uneven

The pattern across backbones is the paper's real contribution to design practice. Gains are largest where the model is weakest: a small local model that scores 20.0 on evidence has room to be organized by a harness, while a strong API model that already scores 54.9 can be constrained by one. Gemini's divergence between evidence and diagnosis shows that the stages are not a single capability ladder — a system can retrieve the right history and still misread the learner. Read together, the results say a tutoring harness is not a free improvement: its value depends on the backbone's prior competence, and its weakest link is the translation from correct evidence into correct instruction.

What this means for practice

• Software developers. Do not assume a tutoring harness helps every model: benchmark it against the specific backbone, because the gains reversed on the strongest one tested here.
• Software developers. Track evidence acquisition and learner diagnosis as separate metrics; a system improved evidence by 18.2 points while its diagnosis fell 7.9 points.
• Researchers. Use failure routing that distinguishes harness faults from model faults before spending supervision, since that distinction decides whether the fix is an interface change or training data.
• Educational technology developers. Keep teaching policies linked to the evidence behind them so a strategy's basis can be inspected after the fact — the auditable link is what separates this design from black-box retrieval.

Limitations

• All evaluation uses synthetic learner histories from one benchmark (LongTutor's 1,000 cases with 199-record histories), so gains measure benchmark performance rather than learning in a real course.
• Teaching quality is judged by an LLM evaluator scoring four rubric dimensions on a 1–5 scale, not by learners or instructors, and the intervals receive no multiplicity correction across the many reported contrasts.
• The backbone set includes API models whose weights and versions change outside the authors' control, so the ranking among them is a snapshot and may not reproduce.
• The harness was refined on data drawn from the same benchmark family used for evaluation, which risks optimizing the harness to the test split even though the authors keep discovery, probe and development sets separate.

Citation

Li, Y., Han, B., Sheng, R., Sun, Y., & Chen, Z. (2026). RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning. arXiv:2610.13399 (cs.AI).