--- source_url: https://arxiv.org/abs/2601.21375 ingested: 2026-05-08 sha256: 6431e0ab38e9967b465dfcec4033d2e43a1befb527f0ac7b6a78dd7a18a0934b --- # TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models **Authors:** Li, Zheng, Song, Siyao, Ma, Jingyuan, Li, Rui, Zeng, Ying, Li, Minghao, Sui, Zhifang **Published:** 2026/01/29 **Categories:** (extracted from API if available) **arXiv:** https://arxiv.org/abs/2601.21375 **PDF:** https://arxiv.org/pdf/2601.21375 **HTML:** https://arxiv.org/html/2601.21375 ## Abstract Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving knowledge-centered teaching underexplored. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via student performance improvement after multi-turn instruction. By restricting teacher agents to structured knowledge points and example problems, the framework avoids information leakage and enables reuse of existing benchmarks. We instantiate the framework on Gaokao data across multiple subjects. Experiments reveal substantial variation in teaching effectiveness across models and domains: some models perform well in mathematics, while teaching remains challenging in physics and chemistry. We also find that incorporating example problems does not necessarily improve teaching, as models often shift toward example-specific error correction. Overall, our results highlight teaching ability as a distinct and measurable dimension of LLM behavior. ## Full Text (Extracted from HTML version) Large language models (LLMs) show promise as teaching assistants, yet their teaching capability remains insufficiently evaluated. Existing benchmarks mainly focus on problem-solving or problem-level guidance, leaving knowledge-centered teaching underexplored. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via student performance improvement after multi-turn instruction. By restricting teacher agents to structured knowledge points and example problems, the framework avoids information leakage and enables reuse of existing benchmarks. We instantiate the framework on Gaokao data across multiple subjects. Experiments reveal substantial variation in teaching effectiveness across models and domains: some models perform well in mathematics, while teaching remains challenging in physics and chemistry. We also find that incorporating example problems does not necessarily improve teaching, as models often shift toward example-specific error correction. Overall, our results highlight teaching ability as a distinct and measurable dimension of LLM behavior. TeachBench: A Syllabus-Grounded Framework for Evaluating Teaching Ability in Large Language Models Zheng Li 1,2 , Siyao Song 2,3 , Jingyuan Ma 1,2 , Rui Li 1,2 , Ying Zeng 2 , Minghao Li 2 † † thanks: Corresponding authors. , Zhifang Sui 1 † † footnotemark: 1 State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University 2 ByteDance BandAI , 3 Institute of Automation, Chinese Academy of Sciences lizheng2001@pku.edu.cn; songsiyao2024@ia.ac.cn; minghao.li@bytedance.com Large language models (LLMs) have demonstrated strong performance across a wide range of reasoning and problem-solving tasks. These advances have made LLMs promising candidates for AI-powered teaching assistants and automated educational systems. Recent work has begun to explore the use of LLMs in educational settings, such as interactive tutoring and learning support Nye et al. ( 2023 ); Liu et al. ( 2024b ); Patil et al. ( 2024 ) . However, despite increasing interest in these applications, systematic benchmarks for evaluating the teaching capability of LLMs remain largely underdeveloped. Several recent benchmarks have begun to explore educational scenarios. For example, datasets such as EducationQ Shi et al. ( 2025 ) and Teach2Eval Zhou et al. ( 2025 ) evaluate whether an LLM can guide a student toward solving a specific target problem. While valuable, these benchmarks primarily focus on problem-level guidance rather than teaching ability in a broader sense. In particular, whether a model can effectively act as a teacher—helping students learn underlying knowledge points and acquire problem-solving strategies—remains insufficiently evaluated. Moreover, directly teaching target questions introduces the risk of information leakage, which can invalidate evaluation results. In this work, we propose a teaching-centered evaluation framework that explicitly measures the instructional effectiveness of LLMs acting as teacher agents. Our framework is built around structured knowledge points derived from a syllabus, rather than target questions themselves. A teacher agent conducts multi-turn instruction based on these knowledge points, while a fixed-capability student agent serves as a proxy for a human learner. Teaching effectiveness is assessed by the student’s performance improvement after instruction, enabling controlled and fair comparison across different teacher agents. To support this evaluation, we construct a knowledge structure tree from exam syllabi and annotate each question with fine-grained knowledge paths. We further generate example problems of varying difficulty for each knowledge point, allowing us to study different teaching strategies under a unified protocol. Using data from the Chinese National College Entrance Examination (Gaokao) across multiple subjects, we conduct extensive experiments to evaluate teaching effectiveness under different interaction settings. Our experimental results reveal several key findings. First, among the evaluated models, Qwen3-235B-A22B-Instruct demonstrates strong teaching effectiveness, achieving a 7.63-point improvement in the mathematics domain. Second, teaching effectiveness varies substantially across domains: models perform better in subjects where knowledge points can be directly applied (e.g., mathematics, history, and politics), while teaching remains more challenging in domains requiring deeper integration of knowledge with complex problem contexts (e.g., physics and chemistry). Finally, we find that incorporating example problems does not necessarily enhance teaching effectiveness. Instead, current LLMs tend to shift from syllabus-grounded instruction to example-based error correction, which fragments the teaching process and ultimately weakens overall instructional quality. Together, these results highlight both the potential and limitations of current LLMs as teachers. More importantly, they show that teaching ability is a distinct and measurable dimension of LLM behavior, separate from raw problem-solving performance. We hope our framework provides a foundation for future research on LLM pedagogy and the development of models that can teach more effectively. We propose a syllabus-grounded evaluation framework that measures LLM teaching capability via post-instruction student performance, while preventing information leakage by restricting teachers to structured knowledge points and enabling reuse of existing benchmarks. We construct a knowledge-structured benchmark based on Gaokao syllabi and questions, including fine-grained knowledge annotation and controlled Student–Teacher interactions across multiple subjects. Through extensive experiments, we provide a systematic analysis of LLM teaching behaviors, revealing both effective teaching patterns and key limitations, particularly in example-based instruction. Mainstream LLM evaluation has predominantly focused on answer correctness under static test settings, measuring knowledge coverage and reasoning skill rather than instructional competence. Representative suites include MMLU for broad academic/professional knowledge Hendrycks et al. ( 2020 ) , HELM for standardized multi-scenario and multi-metric reporting Liang et al. ( 2022 ) , and math-focused benchmarks such as GSM8K and MathBench for multi-step quantitative reasoning and hierarchical proficiency assessment Cobbe et al. ( 2021 ); Liu et al. ( 2024a ) . While these benchmarks are invaluable for characterizing “solver” capability, they do not operationalize whether a model can act as a teacher who improves a learner’s mastery. TechBench addresses this gap by shifting the evaluation target from solving to teaching: the model is assessed by its ability to organize instruction around a syllabus-derived knowledge structure and to induce measurable learning gains under controlled instructional inputs. To increase ecological validity, several benchmarks evaluate LLMs using standardized exams and Chinese-context tests. AGIEval adopts a human-exam-centric design spanning multiple official exams Zhong et al. ( 2024 ) ; C-Eval and CMMLU provide large-scale Chinese multi-discipline multiple-choice suites across difficulty levels Huang et al. ( 2023 ); Li et al. ( 2024 ) ; and GAOKAO-Bench directly evaluates LLM performance on Gaokao questions, including subjective formats with grading considerations Zhang et al. ( 2023 ) . Despite their realism, these efforts remain largely solution-centric : they quantify how well a model answers exam questions, but not how well it can teach the underlying knowledge without being shown the test item. TechBench differs by explicitly instantiating a Gaokao-like teaching scenario: the teacher agent is not given the target question and must teach solely from knowledge points plus curated example problems, and success is defined by post-instruction improvement on held-out tests rather than raw exam scoring alone. Recent work has begun to formalize pedagogical behaviors through tutoring dialogues and rubric-based evaluation. MathDial provides teacher–student tutoring dialogues grounded in multi-step math problems with pedagogical properties Macina et al. ( 2023 ) . The BEA shared tasks benchmark the generation and assessment of teacher responses in educational dialogues, highlighting both progress and the limitations of generic automatic metrics for educational contexts Tack et al. ( 2023 ); Kochmar et al. ( 2025 ) . Beyond shared tasks, TutorBench and MathTutorBench evaluate tutoring skills such as adaptive explanations, feedback, and hinting, often via rubric-based or learned scoring Srinivasa et al. ( 2025 ); Macina et al. ( 2025 ); Shi et al. ( 2025 ); Zhou et al. ( 2025 ); Lelièvre et al. ( 2025 ); Liu et al. ( 2024b ); Maurya et al. ( 2025 ) . TeachBench complements these lines by targeting a different notion of pedagogical success: rather than primarily scoring per-turn response quality, it evaluates end-to-end teaching effectiveness under a knowledge-tree curriculum and leakage-controlled inputs. To evaluate the teaching capability of LLMs when acting as genuine teachers, we design a workflow that leverages a knowledge syllabus and corresponding exam questions to assess an LLM’s ability to teach specific knowledge points. As illustrated in Figure 1 , our approach first extracts the syllabus into a structured knowledge tree using a semi-automatic pipeline. Based on this knowledge tree, an LLM-based Tagger is applied to annotate the knowledge points involved in each question. In parallel, an LLM-based Question Generator produces example problems of varying difficulty levels for each knowledge point. Finally, we employ an LLM with moderate capability as a student agent and conduct multi-turn interactions between the student agent and the teacher agent under evaluation. The teaching ability of the teacher agent is assessed through the improvement in the student’s performance over the course of the interaction. To facilitate downstream processing, we convert the syllabus of exam topics into a structured tree representation. We first use Gemini-3 Google DeepMind ( 2024 ) to process the syllabus and extract an initial knowledge structure tree. We then employ GPT-5 OpenAI ( 2024 ) to review the tree and supplement any missing knowledge points. Finally, the resulting knowledge structure tree is manually inspected to ensure its accuracy and completeness, such that it fully covers the intended scope of the syllabus. Each question may involve multiple knowledge points. Given a question and a set of candidate knowledge nodes at the current level of the knowledge hierarchy, we prompt an LLM to identify all knowledge nodes that are relevant to the question. For each selected node, the tagging process is then recursively applied to its child nodes. We adopt a depth-first traversal strategy over the knowledge hierarchy. All paths expanded from the selected nodes down to leaf nodes are collected, and each root-to-leaf path is treated as a knowledge tag associated with the question. The prompt template used for question tagging is shown in Appendix A .