ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education

Synthesis: ELEVATE (Efficient LLM Education with Virtual Avatar Teaching Engine) is a framework for building GenAI-driven 3D avatar tutors that run entirely on local, consumer-grade hardware rather than cloud APIs. Its three-stratum design separates student-facing avatar interaction, a local GenAI execution core, and a teacher-facing governance layer, so pedagogical control and privacy stay inside the school. A deployed prototype — a History of Computer Science tutor running Hermes-3B and Coqui TTS on commodity GPUs — reached near real-time spoken interaction while exposing the concrete limits of local-first deployment.

Overview

Most educational GenAI is delivered as a cloud-based, text-only chatbot: a centralized service with limited pedagogical control, weak transparency over knowledge sources, and non-trivial privacy and regulatory exposure, since student data is transmitted off-site in ways that complicate compliance with frameworks such as the EU GDPR. That paradigm also presumes continuous connectivity and recurring API costs, creating structural barriers for under-resourced institutions and reinforcing existing digital divides. At the same time, educational interaction with an LLM stands to benefit from multimodal cues and embodied presence, which text-only tutoring cannot supply.

ELEVATE targets these barriers directly. Its premise is that GenAI tutoring in education is a sociotechnical problem rather than a purely technical one, and that a single fixed architecture tends to privilege well-resourced contexts. The framework therefore treats deployability under real constraints — consumer hardware, local execution, modular components — as the entry condition for responsible adoption, while preserving educator-centered AI Governance through an explicit separation between student interaction, teacher control, and execution infrastructure.

Study Design & Method

ELEVATE is a framework plus a working reference prototype deployed in a real-world educational curriculum. It is organized as a closed-loop three-stratum system: (i) a student-facing virtual avatar interaction layer, (ii) a local GenAI execution and multimodal synthesis core, and (iii) a teacher-facing governance layer. Student interactions produced in the first stratum are processed by the engine under teacher-defined constraints, and the resulting multimodal responses are embodied by the avatar.

• Client / interaction layer. A real-time 3D rendering subsystem built on game-engine libraries handles avatar selection, animation control, lip-sync, synchronized speech, facial animation, and visual cues, with a unified interface offering a dialogue panel, speech controls, and optional camera/microphone input. It is a deliberately thin, latency-sensitive client that performs only orchestration, rendering, and playback. In the reference implementation it was built in Unity (v6.0) with C#/.NET; the authors note that fully open-source alternatives such as Godot are an optional deployment path rather than a framework requirement.
• Local engine core. A client–server decomposition in which the server performs automatic speech recognition, LLM inference, and text-to-speech synthesis using a modular, streaming-based orchestration strategy that generates incrementally and reduces perceived latency. Responses are constrained to teacher-approved materials, and incremental delivery supports conversational turn-taking while the avatar displays intermediate states.
• Teacher governance layer. A lightweight web-based parameters editor (Python 3.13) where teachers set system prompt, avatar configuration, and knowledge base. Controls cover high-level instructional parameters, content boundaries, refusal and grounding policies, and persona configuration — voice, speaking style, supportive-versus-neutral stance, degree of scaffolding, and communication pacing — so pedagogical framing is persistent and does not depend on how an individual student prompts the system.
• Models and domain adaptation. The LLM is Hermes-3B run locally via llama.cpp in its quantized Q8 form, selected after hardware-constrained screening of small-to-mid models for 8–16 GB VRAM GPUs; larger models triggered VRAM saturation and unstable runtime behavior. To inject domain knowledge without retrieval infrastructure, the authors performed a one-time QLoRA fine-tune of the 3B model, merged the adapters into the base checkpoint, and converted it for offline inference — a RAG-free configuration that shifts cost to a one-time training phase. TTS used a Coqui TTS engine chosen for local-first modularity and easy integration on commodity hardware.
• Apparatus. To reflect schools with many consumer machines, the server pipeline was partitioned across two local machines: one with an AMD Ryzen 9 9900x CPU, 64 GB DDR4 RAM, and an NVIDIA RTX 5060 Ti (16 GB VRAM) for LLM generation and orchestration; another with an Intel Core i7 6700K, 32 GB DDR4 RAM, and an NVIDIA GeForce RTX 3060 (12 GB VRAM) for TTS. Both models stayed fully resident in memory. The Unity client ran separately on a consumer Nokia G42 Android smartphone.
• Task and materials. The evaluation used the "History of Computer Science" use case, targeting short, curiosity-driven interactions (definitions, historical context, conceptual clarifications) because historical claims are easily verified. The system prompt — intentionally short and explicit, mimicking what a non-prompt-engineering teacher would write — fixed an academic persona, forbade invented facts and speculation, and required responses of at least five sentences so that experiments would reliably trigger multi-sentence outputs.
• Instruments and measures. Twenty representative student-style prompts were run to stress the end-to-end pipeline. Measures were server-side: per-sentence LLM generation time and TTS synthesis time, token counts, end-to-end latency (summed generation over N spoken units), time-to-first-talk (first unit's LLM plus TTS time), and perceived latency, defined to capture the masking effect of incremental delivery — if the avatar is still speaking while the next unit is generated, the user perceives zero or negligible waiting. Client-side latency was not measured, as it was negligible in practice.

Key Findings

• The overall generation times fell within acceptable bounds for interactive educational use, supporting the feasibility of running an embodied, streaming GenAI tutor on consumer-grade devices.
• Time-to-first-audio — the initial wait before the virtual teacher begins speaking — was approximately five seconds on average (approximately 5–6 seconds), with modest variability across prompts. The authors judged this compatible with conversational and instructional interaction patterns in classroom and individual study settings.
• Perceived latency was concentrated almost entirely in the first generation step. Once speech playback began, subsequent generation and synthesis steps were largely masked by the ongoing audio, so perceived latency effectively dropped to zero for later paragraphs.
• LLM inference contributed only a minor fraction of total latency, confirming that the selected 3B-parameter model executed on the stronger GPU achieves fast, stable inference even under streaming generation, with a weak and nearly linear dependence on token count.
• TTS synthesis dominated the pipeline. This was expected: it ran on the less powerful GPU with a library carrying no underlying optimizations, a deliberate choice prioritizing modularity and resource reuse over peak synthesis performance. TTS also showed a steeper growth trend and markedly higher dispersion attributable to utterance length and phonetic complexity.
• GPU technologies from different generations coexisted within the same deployment without integration bottlenecks, supporting incremental hardware reuse rather than synchronized full-system replacement.
• Stated limitations (technical discussion). End-to-end responsiveness is bounded by the heaviest stage on the least capable compute resource, so the baseline configuration may introduce delays on longer explanations — predictable and mitigable by provisioning a more capable on-prem server or dedicating accelerators to TTS. The in-memory conversation session lacks persistence, so continuity is lost after client restarts; adding persistence would introduce governance requirements around retention, consent, and access control. Even with multiple local servers, the queue system can become a bottleneck when serving many students concurrently, risking "noisy neighbor" effects without a queueing policy and admission-control limits.
• Stated limitations (evaluation scope). No controlled user study was reported: the authors state that evaluating learning outcomes and socio-technical effects in real school settings remains future work. Embodiment can also create barriers for some learners, such as sensory sensitivities or discomfort with anthropomorphic agents, making its configuration a strong requirement.

What this means for practice

• Designers. Profile the whole pipeline on the least capable node before promising interactivity: TTS synthesis, not the 3B LLM, dominated end-to-end latency because it ran on the older 12 GB GPU, and streaming masked later generation so perceived latency dropped to near zero once speech began after an average 5-6 s time-to-first-audio. Do not treat the first generation step as the only cost to optimize.
• Designers. Make the teacher governance layer the control surface: expose system prompt, knowledge base, refusal and grounding policy, persona, voice, supportive-versus-neutral stance, scaffolding degree and pacing in a lightweight editor, so pedagogical framing stays persistent instead of depending on how a student prompts.
• Designers. Plan for the operational and accessibility load local-first deployment creates: schools take on updates, safety configuration, device management and policy alignment, and every configuration should keep captions, alternative inputs, reading-level controls, adjustable pace and switchable text-only or voice-only modes available. Default to aggregate and minimal logging, restricting raw transcript or audio retention to clearly justified cases, and measure the workload of configuration cycles and refusal follow-ups rather than assuming they will be sustained.
• Instructors. Frame an embodied spoken tutor as a pedagogical instrument inside lesson plans (warm-up, guided practice, revision stations) rather than a substitute for instruction, and configure it for coaching that elicits reasoning and partial attempts instead of delivering solutions. Because body and voice amplify authority, have the avatar signal uncertainty and invite verification against notes or textbooks, and prefer explicit classroom modes chosen by the teacher over silently inferred ones.
• Researchers. Evaluate these systems as sociotechnical interventions with controlled classroom studies, not with system benchmarks: whether always-available embodied tutors reduce productive struggle, shift help-seeking away from teachers and peers, and change ownership of learning outcomes is unresolved, while hallucination, over-reliance and deskilling, uneven benefits across student groups, and accountability drifting into surveillance through excessive logging remain structurally open risks. If on-demand tutoring is available, shift assessment toward process — oral defenses, in-class problem solving, stepwise derivations, reflective journaling — and assess students on documenting how assistance contributed to their work.

Limitations

• No controlled user study was run: the evaluation pushed 20 representative student-style prompts through the end-to-end pipeline and measured only server-side generation and synthesis times, token counts, latency and perceived latency, with client-side latency unmeasured because it was negligible in practice. The authors state that evaluating learning outcomes and socio-technical effects in real school settings remains future work.
• Everything rests on one prototype and one domain: a single "History of Computer Science" use case with a one-time QLoRA fine-tune of Hermes-3B in Q8 and Coqui TTS, chosen after hardware screening showed larger models saturating 8-16 GB VRAM, so latency and answer quality are not established for other subjects or knowledge bases.
• The deployment spans two local machines with GPUs from different generations (RTX 5060 Ti 16 GB and RTX 3060 12 GB), and the authors note that end-to-end responsiveness is bounded by the heaviest stage on the weakest resource: longer explanations may delay under the baseline configuration, and the queue can become a bottleneck with concurrent students, risking "noisy neighbor" effects without a queueing policy and admission control.
• The conversation session is held in memory with no persistence, so continuity is lost after client restarts, and the authors note that adding persistence introduces governance requirements around retention, consent and access control. Embodiment itself is also a barrier for some learners, such as those with sensory sensitivities or discomfort with anthropomorphic agents, which makes avatar configuration a strong requirement rather than a preference.

Citation

Lorenzo Stacchio, Michele Giordano, Daniele Berardini, Primo Zingaretti, Emanuele Frontoni (2026). ELEVATE: Designing Human-Centered GenAI Virtual Tutors for Scalable and Inclusive Education. submitted 17 Jun 2026