Educational Technology Developers

Educational Technology Developers — the people and organizations that build educational technology: product designers, software developers, learning engineers, learning-analytics designers, and the edtech companies, university labs and Open Source projects they work in. In AI in education this is the role that turns a model capability into something a teacher or learner can actually use. It carries decisions no later stage can undo: what evidence a design claim rests on, how far a analytics pipeline or a tutoring system is grounded in the institution's own licensed material, and whether teachers and learners are included in design. It also fixes which instructional design assumptions are baked into the defaults, and what happens to the product after the funding stops. Across the knowledge base's system reports and deployment studies, the recurring lesson is that the deployment context, not the model, is usually the binding constraint.

Questions to Consider

If the meta-analyses claiming that "AI improves learning" rest on invalid methodology, as the audit in Presumed Effective: The Manufacturing of an Evidence Base for AI-in-Education Through Flawed Meta-Analysis found, what evidence is a product roadmap actually allowed to build on?
Should an algorithm's explanations be written in the teacher's curricular language even when that costs more design effort than exposing feature importances — and who pays for that effort?
When a tool is co-designed with students, whose verdict decides: measured learning gains, or the 96% who said they wanted it kept?
Is on-premise, open-licensed deployment a technical choice or a governance one — and should transparency requirements become a condition of purchase?
What does a developer owe an institution when the grant ends: a maintained product, a forkable repository, or a candid statement that the system was never a validated intervention?

Introduction

The developer sits one level below the platform. Edtech Platform describes the deployed system and the stakeholder it becomes once it is in a school or university; this page is about the people who decide what that system does. The distinction matters because platform-level findings — low take-up, equity skew, procurement friction — are usually consequences of design choices made earlier, by someone who never met the learners.

The role is also distinct from its neighbours. Learning Design and Curriculum Design design a course for a known cohort; a technology developer designs a product that many courses, taught by people they have never met, will use — which is why defaults, configurability and documentation carry pedagogical weight. Educational Development supports the teaching staff of an institution from inside it; developers sit outside or alongside, supplying the tools those staff are then asked to adopt. And Design-Based Research is the evidence standard such developers are increasingly asked to meet: iterative, contextual, and reported with its own limitations.

Who builds educational AI

Research labs building public infrastructure. OATutor was built at UC Berkeley as the first fully open-source adaptive tutoring system on ITS principles: an MIT-licensed codebase with a Creative Commons algebra library, Bayesian Knowledge Tracing mastery estimation, A/B testing infrastructure and LTI support. Its reason for existing is a design decision — proprietary platforms had confined Adaptive Learning research to closed systems — and its authoring route is another: 16 creators produced a College Algebra course in six months after 2.27 hours of training.

Model builders. LearnLM: Improving Gemini for Learning reframes improving a model for learning as pedagogical instruction following: behavior is set per application through system instructions rather than one fixed definition of Pedagogies and Teaching Strategies, and expert reviewers preferred it by +31% over GPT-4o and +13% over base Gemini. The practical point is that pedagogy is too context-dependent to define globally; the useful capability is adherence to the instructions a developer writes, measured by conversation-level scenarios rather than single-turn benchmarks.

Architects of knowledge models. Ontology-Based Layered Hybrid AI-Driven Knowledge Model for Personalized E-Learning argues Personalized Learning needs more than a static ontology, proposing systems of mapped ontologies plus rules and analytics in place of the classic four-model ITS architecture, and a reuse framework of eight metadata classes that aims to cut the cost of each new build.

Infrastructure and measurement engineers. Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains describes a modular, domain-agnostic pipeline for learner interaction data validated across three educational AI assistants, where methods built for one domain extended to another — reusable Learning Analytics infrastructure rather than a one-course dashboard. StanBKT: Rethinking Parameter Estimation in Bayesian Knowledge Tracing shows the complementary case: a Bayesian reimplementation produced identical prediction to the established point-estimate tool (AUC 0.711), differing only in cost and in credible intervals that make a condition comparison interpretable.

Builders inside institutions. From Surface Learning to Deep Understanding: A Grounded AI Tutoring System for Moodle embeds Large Language Models (LLMs) tutoring in an existing LMS rather than shipping a standalone tool, lowering the adoption threshold the ITS literature names as a reason systems fail in practice. SAVVY: Student Attention Visualization for Video-based Learning Analysis turns Multimodal AI attention signals into an interface teachers can read before releasing a video. Instructional Agents automates ADDIE's first three phases with role-specialized agents, and its ablation is a design lesson. The single-agent baseline scored worst, Full Co-Pilot beat Autonomous by 0.5–0.9 points, and no quality difference between backends made the cheapest the default.

What a design claim can rest on

The evidence base is weaker than it looks. Presumed Effective: The Manufacturing of an Evidence Base for AI-in-Education Through Flawed Meta-Analysis audited 14 meta-analyses claiming that AI improves education and found none justified its claims: all but two defined the treatment as a tool rather than a pedagogical intervention, 61% of 59 vetted primary studies had validity problems, heterogeneity was high wherever reported, moderator analyses were underpowered, and publication bias was never validly assessed. A meta-analysis that had been retracted still carried authority for 60% of the later papers sampled. For a developer, "AI improves learning" is a product-category claim, not a design input.

Report uncertainty and full cost, not just accuracy. For StanBKT, Bayesian inference buys nothing in prediction and everything in being able to say which effects were credible. Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources reports retrieval ablations, quantization-aware fine-tuning, VRAM, energy per query and hallucination measured against retrieved open resources — with the authors' own caveat that the system is not a validated tutor. That is the evaluation discipline that makes a deployment claim checkable.

Co-design with teachers and learners

Explanations must speak the teacher's language. The Impact of Explainable AI on Teachers' Trust and Acceptance of AI EdTech Recommendations: The Power of Domain-specific Explanations ran a within-subject experiment with 41 chemistry teachers on an ML recommendation tool: understandability, Trust and acceptance correlated positively, and domain-driven explanations in curricular language produced significantly higher understandability, learned trust and acceptance than feature-importance explanations. Trust was also dynamic — several teachers said only classroom experience would settle it — and acceptance depended on pedagogical alignment and workload reduction.

Credibility cues are not evidence of correctness. Adding citations and authority signals to a multi-agent debate raised abandonment of a correct judgment for the AI's position from 15.0% to 31.8%, so provenance has to be checkable at the point of use (Tuo et al., 2026).

Co-design at institutional scale. AIDA at the Open University was built through six design-based studies over 18 months with 498 students and 20 staff. About 20% were initially skeptical; after hands-on use 96% wanted it kept, and an exploratory RCT found twice the usage time but no significant differences on learning process data. The enabling factors were organizational — senior sponsorship, cross-unit collaboration, data-informed iteration — with gaps in systems-thinking capacity.

Procurement, openness and after the funding stops

Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources supplies the inputs a procurement decision needs: a 12 GB VRAM hardware floor, an accuracy ceiling for a 7B-class model, per-query energy, and an ordering of choices — retrieval first (without it the model scored 52.3%, below a TF-IDF baseline), then fine-tuning, then quantization-aware compression. Open licensing is the precondition for serving a corpus locally. Reclaiming Epistemic Agency: A Critical Framework for Human-Generative AI Co-Agency in Education frames the same decision as governance: transparency requirements turn purchasing into epistemological governance, contestability and provenance become conditions, and districts with the least capacity face the highest bar. What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education adds that vendor governance appeared in only 29% of 30 audited policy packages, which specified what AI may do far more readily than what evidence of learning remained. Generative AI (GenAI) as a mindtool that supports generative learning (GL) poses the design question — does the product encourage learning with the tool or offload cognitive work — and What Makes Words Hard? Sakura at BEA 2026 Shared Task on Vocabulary Difficulty Prediction shows the trade in miniature: the top-scoring black-box model (r > 0.91) was less explainable than the interpretable one (r > 0.77).

Common Misconceptions

"The meta-analyses support the product claim." They do not, and the reason is specific rather than a matter of degree. An audit of 14 meta-analyses claiming AI improves education found none justified its claims — all but two failed on methodology — and a retracted meta-analysis was still cited as authoritative by 60% of sampled later papers. The retraction propagated faster than the correction, which means the literature a roadmap is likely to cite contains claims the field has already withdrawn.
"Adding citations and provenance makes a system more trustworthy." In a multi-agent debate setting, adding citations and authority signals to the AI's position raised abandonment of a correct judgment from 15.0% to 31.8%. Provenance was not neutral information: it made the wrong answer more persuasive. Credibility cues need to be evidence of correctness, not decoration around a claim.
"Co-design with users validates the design." Co-design answers a different question than efficacy does. When a tool is co-designed with students and 96% say they want it kept, that is a finding about acceptability. Whether learning improved is a separate measurement, and the two can point in opposite directions without either being wrong.
"Agents are the next capability tier." The Instructional Agents ablation is the useful counterexample: the single-agent baseline scored worst and the Full Co-Pilot configuration beat the Autonomous one. More autonomy was not the better design; the co-pilot arrangement was. Architecture choices have to be evaluated rather than assumed to improve with capability.

What this means for practice

For developers and designers

Ground a design claim in a study you have read, not an effect size you have quoted. With 14 audited meta-analyses failing on method and a withdrawn one still propagating, citation count is not a validity signal.
Test autonomy as a variable rather than shipping it as an upgrade. Run the ablation: single-agent, co-pilot, autonomous. The Instructional Agents result showed the co-pilot configuration winning, so the default should be the one that measured best, not the one that sounds most capable.
Separate acceptability from efficacy in your own evaluation plan. A high willingness-to-keep number is a real finding and it is not evidence of learning, and conflating them in a report is how a weak product looks strong.
Treat credibility cues as a manipulation with a measured effect. Adding citations moved abandonment of a correct answer from 15.0% to 31.8%, so surfacing provenance changes user judgment — design for that rather than assuming it informs.

For instructors evaluating a tool

Ask how the tool was evaluated on the task you intend to use it for, and whether the evaluation was co-pilot or autonomous. The two are not interchangeable.
Ask when the evaluation happened. Capability claims tied to an unstated model version date quickly and describe a system you will not be running.

Connected Concepts

Connected Articles

Connected Resources

Playlab
A nonprofit creator platform where educators build, share and evaluate their own AI apps, grounded in their curriculum, run on the model of their choice, with teacher visibility built in.

Metadata

Intended audience	Instructional Designers, Software Developers, Learning Analytics Designers, Institutions, Educational Technology Developers
Technologies and techniques	Learning Analytics, Edtech Platform, Open Source
Research methods and evaluation	Design-Based Research
Foundations of AI in education	Educational Development, Learning Design
Page kind	Evaluation

Source History Edit Chat