Beyond ID Embeddings: Process-Grounded Language Modeling for Cognitive Diagnosis

Synthesis: PLCD replaces the discrete ID embeddings at the heart of cognitive diagnosis with language-derived structures: LLM-generated concept schemas and exercise process graphs act as cognitive priors, while response records calibrate a learner-specific posterior through target-conditioned retrieval. Across Junyi, XES3G5M, and MOOC, this process-grounded model reaches 80.81%, 83.51%, and 87.16% accuracy respectively, outperforming strong ID-based and knowledge-enhanced baselines under a significance-tested protocol. The largest gains appear precisely where ID-centric Learner Modeling and Adaptive Instruction fails — new concepts, new exercises, and a masked Q-matrix — and teacher ratings of the generated process graphs support their educational plausibility. The work argues that structured language, not side information bolted onto ID vectors, can organize diagnostic representation itself.

Key Findings

1. PLCD attains 80.81% ACC / 84.52% AUC on Junyi, 83.51% ACC / 85.37% AUC on XES3G5M, and 87.16% ACC / 88.33% AUC on MOOC, beating every baseline including DMC-CDM's 83.81% ACC on MOOC under a two-sided paired t-test over ten runs (p < 0.05).
2. Removing structured evidence causes the largest ablation collapse — Junyi ACC falls from 80.81% to 76.52% and MOOC from 87.16% to 78.95% — showing raw exercise text cannot expose prerequisite relations or process-level operations.
3. In cold-start evaluation on XES3G5M, PLCD exceeds KCD by 4.60 ACC points for new concepts (75.58% vs 70.98%) and 4.00 points for new exercises (73.09% vs 69.09%).
4. With the entire Q-matrix masked, PLCD still reaches 80.46% ACC and 82.37% AUC, 4.52 ACC points above KCD's 75.94%, though below the Oracle upper bound of 83.51% ACC.
5. Memory strategy matters: Semantic+Q retrieval (PLCD) scores 83.51% ACC versus 79.96% for a static learner summary, 78.72% for random retrieval, and 80.21% when retrieved correctness labels are shuffled.
6. The process-prior DA-MoE raises gate-prior cosine similarity from 0.41 to 0.68 and cuts KL divergence from 0.88 to 0.43 over an ungrounded MoE, with expert entropy rising from 1.38 to 1.67 — evidence against expert collapse.
7. Exercise-level guess–slip calibration mainly improves probability quality: XES3G5M ECE drops from 0.071 to 0.037 and Brier score from 0.165 to 0.145, with smaller accuracy movement.
8. Five teachers rated 120 stratified exercises (40 per dataset); Gemini 3 Pro graphs scored 4.44 overall, highest among five generators, with concept relevance 4.58 ± 0.31 and misconception plausibility the weakest dimension at 4.23 ± 0.48.

From ID Embeddings to Structured Cognitive Evidence

The paper's starting point is a representational mismatch: traditional Cognitive Diagnosis models encode students, exercises, and knowledge concepts as discrete ID embeddings and model learning as numerical adjustment between those indices. The authors argue this contradicts cognitive psychology, where knowledge is organized around meaning, relations, and conceptual similarity, and leaves the compositional semantics of exercise text, concept descriptions, and prerequisite relations inaccessible. When a new exercise or concept appears, an ID-based CDM has no route to its textual content until enough responses accumulate — the cold-start failure illustrated in the paper's opening figure.

PLCD's first stage converts educational text into structured cognitive evidence rather than free-form semantic profiles. For each concept c, an Large Language Models (LLMs) populates a typed concept schema $G_c$ with definition, prerequisites, subskills, common misconceptions, and related cognitive operations. For each exercise e, the model builds a cognitive process graph $G_e = \langle C_e, O_e, U_e, S_e, D_e \rangle$: weighted required concepts, weighted cognitive operations, textual cues that trigger those operations, solution steps, and plausible error types. Crucially, exercise evidence moves beyond coarse concept membership to capture the reasoning operations each item demands and the error-prone steps in its solution — the material a Knowledge Graph over concepts alone would not supply. The paper is explicit that PLCD is not response-free: language provides a zero-shot cognitive prior and response records calibrate the learner-specific posterior. IDs survive only as retrieval indices during training and are never learned as embeddings, optimized as parameters, or used as input features.

Language-to-Cognition Mapping and Calibration

The second stage maps language-derived evidence into diagnostic states through a domain-adaptive mixture-of-experts mapper (DA-MoE) and process-level supervised contrastive learning. Because educational data is cognitively heterogeneous — geometric reasoning, algebraic manipulation, and proportional reasoning require different mental operations — PLCD binds each of its 8 experts to a predefined process type drawn from the exercise graphs (semantic-to-symbolic translation, relational mapping, arithmetic computation, algebraic manipulation, spatial reasoning, proportional reasoning, conceptual recall, procedural execution). The gating network combines the input embedding with the exercise's process prior and activates only the top 2 experts, and a KL anchor regularizes the gate toward that prior so activated experts remain traceable to explicit reasoning operations. A load-balancing term prevents the gate from collapsing onto a few experts.

Diagnosis proceeds through concept-specific knowledge states and exercise difficulty factors rather than a single global profile. Target-conditioned semantic memory preserves a per-student history of cognitive process graphs with correctness labels and time gaps, and retrieves prior exercises weighted by graph-level semantic similarity plus Q-matrix concept overlap. This design directly targets the limitation the authors attribute to existing language-enhanced CDMs: compressing a learner's whole history into one fixed profile obscures which prior tasks are cognitively relevant to the current target. The contrastive objective is process-level rather than exercise-level, so a wrong answer generates negative evidence only for processes that the target exercise requires and that the student has historically failed — an exercise may demand multiple concepts, and an incorrect response does not imply every required process failed. Finally, an Item Response Theory-inspired head estimates exercise-level guessing and slipping effects, so the final probability preserves diagnostic structure while correcting for item-level response noise. This psychometric layer is part of what makes the model a psychometrically aware predictor rather than a pure accuracy maximizer.

Evidence Across Sparsity, Cold Start, and Grounding

Experiments span three mathematics platform datasets: Junyi (10,000 students, 835 exercises, 835 concepts, 324,631 records), XES3G5M (3,000 students, 4,314 exercises, 677 concepts, 792,720 records), and MOOC (3,000 students, 2,726 exercises, 983 concepts, 437,563 records), split 8:1:1 with ten runs and an average of 32.46 to 264.24 records per student. As response records sparsify from 0.9 to 0.1, PLCD's advantage over RCD and KCD widens — consistent with the claim that reusable cognitive priors substitute for unavailable interaction history. The memory-mechanism study isolates why: label-shuffled retrieval keeps the same retrieved exercises as full PLCD but drops to 80.21% ACC, demonstrating that gains come from the student's actual correctness patterns on cognitively similar tasks, not exercise similarity alone. Retrieval diagnostics agree, with PLCD achieving the highest Q-Jaccard@5 of 0.412 and a residual response correlation@5 of 0.168 that the shuffled variant cannot match.

The grounding analysis shows the DA-MoE gate increasingly matching the LLM-derived process prior as each component is added (cosine 0.41 → 0.68 → 0.72; KL 0.88 → 0.43 → 0.38), and t-SNE projection of mapped exercise representations yields a continuous manifold in which geometry and fraction problems cluster separately while basic arithmetic transitions smoothly into fractions through division and multiplicative-thinking boundary regions. A case study on a held-out student shows PLCD producing sharper mastery estimates than ID-based reports confined to a generic 0.4–0.6 band, distinguishing weaknesses such as semantic translation errors from procedural failures — a lens on Misconceptions about AI and Problem Solving processes. Independent teacher validation of the LLM-generated graphs (five teachers, 120 exercises, anonymized generators) rates them educationally plausible, with the lowest score on misconception plausibility (4.23 ± 0.48) — the authors' own caveat that error types are harder to infer from exercise text alone. The stated limitations are equally direct: true mastery states are unobservable, so results support a predictive and cognitively grounded framework rather than causal claims about learning mechanisms.

What this means for practice

• Instructors. Read diagnostic output as model-based evidence to be checked rather than an automated verdict: the authors state results are conditioned on available response records and exercise content and should be used with human oversight, not for high-stakes decisions.
• Instructors. Interrogate the process behind a wrong answer rather than the concept label alone, and verify error types before acting on them — the model separates translation errors from procedural failures, but teacher raters scored misconception plausibility lowest of the generated graph attributes (4.23 ± 0.48).
• Designers. Base retrieval on the learner's own correctness on cognitively similar items instead of one static profile: semantic-plus-Q retrieval reached 83.51% ACC on XES3G5M against 79.96% for a static learner summary and 78.72% for random retrieval.
• Designers. Target cold start explicitly, where ID-based models have no history to draw on: PLCD exceeded KCD by 4.60 ACC points for new concepts (75.58% vs 70.98%) and 4.00 points for new exercises (73.09% vs 69.09%), and still reached 80.46% ACC with the whole Q-matrix masked.
• Researchers. Model structured process evidence rather than raw exercise text: the ablation that removes it drops Junyi ACC from 80.81% to 76.52% and MOOC ACC from 87.16% to 78.95%.

Limitations

• True learner mastery is unobservable, so the study evaluates diagnosis indirectly through response prediction and calibration; the authors state the results support a predictive, cognitively grounded framework and not causal evidence about learning mechanisms.
• The model relies on available response records to calibrate learner-specific states, and the quality of its structured cognitive evidence depends on the clarity of the exercise text and concept descriptions.
• Teacher validation used five paid teachers rating 120 stratified exercises (40 per dataset), and misconception plausibility scored weakest at 4.23 ± 0.48 — the authors' own caveat that error types are harder to infer from exercise text alone.
• Evidence comes from three public mathematics datasets (Junyi, XES3G5M, MOOC) rather than live classrooms, and even with the Q-matrix masked the model reaches 80.46% ACC, below the Oracle upper bound of 83.51%.

Connected Concepts

• Cognitive Diagnosis
• Learner Modeling and Adaptive Instruction
• Item Response Theory
• Psychometrically Aware AI
• Educational NLP
• Knowledge Graph
• Large Language Models (LLMs)
• Knowledge Tracing
• Personalized Learning
• Misconceptions about AI
• Math Education
• Prior Knowledge
• Problem Solving

Connected Articles

• Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work — Benchmarking Large Language Models for Diagnosing Students' Cognitive Skills from Handwritten Math Work
• HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis — HiLLM-CD: LLM-Enhanced Hierarchical Cognitive Diagnosis
• Bayesian cognitive diagnosis optimizes personalized learning paths via mediation of cognitive load and Hidden Markov Model state transitions — Bayesian cognitive diagnosis optimizes personalized learning paths via mediation of cognitive load and Hidden Markov Model state transitions
• Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach — Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach
• Privacy-Preserving Heterogeneous Multi-LLM Federated Inference for Cognitive Diagnosis — Privacy-Preserving Heterogeneous Multi-LLM Federated Inference for Cognitive Diagnosis
• ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs — ProPRL: Property-Aware Prerequisite Relation Learning in Educational Knowledge Graphs
• LearnOpt: Recovering the Latent Cognitive Structure of Standardized Examinations via Knowledge Graphs and Constrained Optimization — LearnOpt: Recovering the Latent Cognitive Structure of Standardized Examinations via Knowledge Graphs and Constrained Optimization
• Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling — Neural-Symbolic Knowledge Tracing
• Interpretable Knowledge Tracing — Interpretable Knowledge Tracing

Citation

Minghang Liu, Yuanzhuo Wang, Qiang Qiu, Huawei Shen, and Xueqi Cheng (2026). Beyond ID Embeddings: Process-Grounded Language Modeling for Cognitive Diagnosis. arXiv preprint.