ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models

Synthesis: Jiang et al. (2026) introduce ELBench, the first benchmark to evaluate education-facing LLMs on all four required dimensions — General Capability, Safety and Trustworthiness, Basic Education, and High-Level Cultivation — under a common protocol, combining curated public sources with newly synthesized safety and cultivation data. Testing nine models, they find module-level profiles are more informative than a single aggregate: the top six models are statistically indistinguishable overall yet differ substantially by module leader, and safety is anti-correlated with practical teaching (r = −0.83). The two education-specialized models lead neither education module, and all models share a systematic blind spot on High-Level Cultivation's structured-judgment task. The work connects to Benchmark, AI Ed Evaluation, and Pedagogical Safety frameworks.

An Integrated Profile, Not a Single Score

A usable education-facing model must be accurate, safe under sensitive prompts, instructionally useful, and aligned with pedagogical goals at the same time. Existing benchmarks evaluate these requirements largely in isolation; ELBench is the first to assess all four as an integrated profile under a common protocol.

Key Findings

1. Module-level profiles beat a single aggregate. The top six models are statistically indistinguishable on overall score (83.1–83.7 with overlapping 95% bootstrap confidence intervals), yet their module leaders differ substantially — the score spread among them is 19.5 points on Safety, 9.7 on Basic Education, and 7.0 on General Capability.
2. Chinese-developed models lead the safety module, the most discriminative in the suite; the advantage is largest on region-specific normative content and narrows, but does not vanish, on universal-harm content (a difference-in-differences of 28.9 points between the two subsets).
3. Safety and practical teaching trade off. Basic Education is strongly anti-correlated with Safety across the nine models (r = −0.83), a result stable under leave-one-model-out recomputation, suggesting refusal training reduces the openness that practical teaching rewards.
4. Education-specialized models lead neither education module, and on High-Level Cultivation all models share a systematic blind spot — on the structured-judgment task they converge on the same non-reference option, favoring pedagogical style over fit to the stated goal, so the module is uniformly low and does not separate the field.

The Four Modules

ELBench operationalizes the four requirements as four modules, each built by one of two strategies chosen by whether suitable public material exists: curation from established sources or synthesis through a human-in-the-loop pipeline. General Capability aggregates knowledge, reasoning, mathematics, and instruction-following items sampled from MMLU-Pro, C-Eval, IFEval, and a MATH-500 subset, together with competition mathematics from AIME (2024–2026). Safety and Trustworthiness has five families — refusal, safe guidance, and benign answering (newly synthesized, 250 items each), plus teaching safety (150 multi-select items) and adversarial safety (100 jailbreak-style prompts) curated from EduGuardBench. Basic Education covers knowledge-point explanation, contextualized question generation, interdisciplinary lesson planning, and guided Problem Solving tutoring, drawn from the ELMES education-scenario collection; the tutoring task is multi-turn, with the transcript scored for instructional quality rather than final correctness. High-Level Cultivation, synthesized in full, has a structured educational-judgment task and an open-ended educational-generation task measuring whether the model's pedagogical judgments match a domain expert's.

Construction: Curated Plus Synthesized

The self-built portions are produced by a four-stage human-in-the-loop pipeline: expert seed authoring grounded in a taxonomy, Large Language Models (LLMs)-based expansion with multiple generators to mitigate single-model bias, automated pre-screening for formatting errors, near-duplicates, and rule violations, and iterative expert review checking realism, reference-answer correctness, option distinctness, and request plausibility. Synthesis is necessary where suitable public material is absent — refusal items must pair a to-be-declined request with a category label, and cultivation situations need a classroom scenario, distinguishable options, and a defensible preferred choice. Curated General Capability items pass through a parallel pipeline with balanced sampling across subjects, item types, and difficulty, plus reference verification and train-set contamination checks (motivating contamination-resistant formats such as MMLU-Pro).

Evaluation Protocol

Closed-form tasks are scored by reference matching or deterministic checks; the multi-select teaching-safety items award full credit for an exact option-set match, partial credit for a safe subset, and none for any answer admitting an unsafe option. Open-ended tasks (instructional quality, safe redirection, educational generation) are scored by rubric-based LLM judging, using a judge (Qwen3.6) selected for highest agreement with human annotation (mean κ = 0.83) with a majority vote over nine calls and randomized presentation order to control position bias. Modules are reported as normalized 0–100 scores; the overall score is the unweighted mean of the four, reported separately because averaging discards deployment-relevant trade-offs.

Results

The aggregate General Capability score is dominated by competition mathematics: on AIME, per-year success ranges from above 90% for the strongest models to the thirties and fifties for others, so two models can differ by roughly ten points almost entirely on AIME — the education-specialized InnoSpark-235B scores near the top on curated knowledge but in the teens on AIME, localizing its deficit to multi-step reasoning. On refusal, the U.S.-developed systems decline 39.5–50.4% of requests that should be refused while three of the four Chinese-developed general models decline 94.7–99.0%, and the safety-specialized variant 99.7%; the group gap is much larger on region-specific than universal-harm content, suggesting the difference reflects where the two groups concentrate safety effort rather than a uniform ability difference. High-Level Cultivation is every model's lowest-scoring module and correlates with General Capability (r = 0.69), yet the strongest general models do not pull ahead — scaling general ability alone does not resolve it.

Discussion: The Return on Education-Specific Specialization

The two education-specialized models, among the strongest education-oriented systems available, were caught within months by general models through ordinary version updates alone. Domain specialization has paid off most durably where the target carries a verifiable reward signal — competition mathematics, code, or clinical diagnosis — but high-level educational judgment has no agreed definition of the right response and no reward model to optimize against, so general pre-training and present-day education post-training converge on a similar style-over-fit tendency. Where much of the domain's instructional content is already in the pre-training corpus, a stronger general base may absorb most of what specialization was meant to add. The paper also points to a limit of the evaluation paradigm: static single-turn question answering reaches a construct-validity ceiling for measuring teaching, which is interactive, adaptive, and longitudinal, pointing toward simulated-student dialogue, adaptive student personas, and outcome-grounded scoring of learning gains.

What this means for practice

• Software developers. Select an education-facing model by module profile rather than a single leaderboard rank, because module strengths trade off and an aggregate score hides which model is actually safe, teachable, or capable for a given context.
• Software developers. Weigh safety against practical teaching deliberately: with Safety and Basic Education anti-correlated (r = -0.83), no single education score serves a deployment that needs both, so organizations must weight requirements by use case.
• Administrators. Read region-specific refusal behavior against your own deployment context, because a higher refusal rate on normative content measures conformance to one jurisdiction's specification rather than a universal notion of safety.
• Researchers. Do not assume education-specific post-training buys a durable edge: the two education-specialized models were caught by general models through ordinary version updates, so invest where the target carries a verifiable reward signal.
• Researchers. Move evaluation beyond static, single-turn items toward interactive, learner-in-the-loop protocols if the goal is to measure teaching rather than answer accuracy.

Limitations

• The model set is a sample of nine representative systems, so group-level findings (including the Chinese-developed safety advantage) describe this set rather than the population of available models.
• The self-built safety and cultivation items are expert-verified but synthetic, and the benchmark is text-only, so it cannot assess multimodal inputs or authentic classroom interaction.
• Open-ended tasks (instructional quality, safe redirection, educational generation) are scored by rubric-based LLM judging, which is calibrated (mean κ = 0.83) but remains imperfect.
• Module sizes are uneven by design, so module scores are not directly comparable, and the Safety module measures behavior against one education-oriented specification that includes region-specific content, so its result is interpretable only within that deployment context.

Citation

Jiang, Y., Zhu, X., Tan, F., Zhang, Z., Huang, K., Yu, Y., Fei, Z., Luo, Y., Li, K., Hao, H., Zhai, G., & Zhou, A. (2026). ELBench: A multi-dimensional benchmark for education-facing large language models.