Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources

Synthesis: Shen et al. (2026) show that an Large Language Models (LLMs) knowledge-base assistant for introductory computer science can be built entirely from openly licensed content and run on consumer-grade hardware — an NVIDIA RTX 3060 with 12 GB VRAM — as an alternative to cloud-dependent AI tutors. Their evaluation protocol is unusually complete for this literature: it reports retrieval ablations, quantization-aware fine-tuning, VRAM and energy per query, and hallucination measured against retrieved OER chunks rather than asserted. The strongest results (DeepSeek-MoE at 78.6% overall accuracy, 1.8 mWh per query) come with the authors' own caveat that the system is a knowledge-base assistant, not a validated pedagogical intervention.

Overview

The paper targets a specific institutional situation: schools and departments that want AI-assisted technical support but cannot depend on cloud APIs because of cost, latency, intermittent internet, or Privacy and data-governance rules. The authors combine two ingredients that are each independently unremarkable — Open Source models that can be hosted locally, and open educational resources that can be indexed and adapted without publisher permission — and ask how far the combination can go when the hardware budget is one consumer GPU.

A second motivation is Sustainability: if every student query carries a cloud inference cost in money and energy, an on-premise deployment changes the accounting for both. The paper therefore reports per-query energy alongside accuracy, and treats accuracy alone as an insufficient evaluation.

Study Design & Method

• Corpus. Structured knowledge extraction from 82 open-licensed Markdown documents, indexed for retrieval in a retrieval-augmented generation pipeline.
• Models. Qwen-7B and DeepSeek-MoE, each evaluated in a local configuration without retrieval, with retrieval but no fine-tuning, and with NF4 4-bit quantization-aware fine-tuning.
• Baselines. A TF-IDF retrieval baseline and the local LLM with no retrieval, to separate the contribution of retrieval from the contribution of the language model.
• Metrics. Overall and multi-hop answer accuracy against reference answers; hallucination operationalized through a two-stage natural language inference procedure against the retrieved OER chunks (κ = 0.76 agreement with expert judgment); VRAM footprint and energy in mWh per query; and a local perplexity diagnostic.
• Evaluation conditions. Multiple ablations (metadata-enriched embeddings, curriculum-specific attention masking, dynamic batching, KV caching) isolate which design decisions drive accuracy versus efficiency.

Key Findings

1. On-premise inference is feasible on one consumer GPU. All inference runs on institution-owned hardware without transmitting student queries to external servers, removing cloud latency, recurring cost, and the data-governance exposure of sending student questions to a vendor — and keeping the assistant available where internet access is unreliable.
2. Retrieval, not the model, is the first-order design decision. The local LLM with no retrieval reached only 52.3% accuracy, below the TF-IDF baseline of 55.4%; adding RAG (Retrieval-Augmented Generation) without any fine-tuning lifted performance to 66.6%. A generic small model grounded in nothing is worse than a classical retrieval baseline.
3. Fine-tuning adds value beyond retrieval. Quantization-aware fine-tuned Qwen-7B reached 69.8% (+3.2 pp, p = 0.031) and DeepSeek-MoE reached 78.6% (+12.0 pp, p < 0.001), the latter including 82.3% on multi-hop questions — the same open-licensed corpus supports substantially better performance once the model is adapted to it.
4. Quantization-aware fine-tuning contains the cost of compression. The full-precision-to-4-bit accuracy gap fell from 4.2 pp to 1.7 pp for Qwen-7B and from 2.9 pp to 1.2 pp for DeepSeek-MoE, while VRAM dropped 38.7% and 37.9% respectively. Quantizing without fine-tuning costs more accuracy than quantizing with it.
5. Energy per query becomes reportable. The most efficient configuration (fine-tuned DeepSeek-MoE) consumed 1.8 mWh per query, a 43.8% reduction against the Qwen-7B FP16 baseline at 3.2 mWh per query. The authors convert this into a classroom figure: a class of 30 students submitting 10 queries each consumes roughly 0.54 Wh for a session.
6. Compression raises hallucination, fine-tuning pulls it back. Quantization increased hallucination for Qwen-7B (8.6% → 12.3%) and DeepSeek-MoE (7.2% → 10.4%), but the fine-tuned models recovered much of it (Qwen-7B 9.8%; DeepSeek-MoE 8.1%). Crucially, this is measured by NLI entailment against the retrieved OER chunks rather than by self-report, and the agreement with expert judgment (κ = 0.76) is reported so readers can judge the instrument.
7. The ablation assigns credit precisely. Removing metadata-enriched embeddings caused the largest accuracy loss (69.8% → 65.3%), while curriculum-specific attention masking and dynamic batching mainly improved energy efficiency, and KV caching mainly reduced latency.

What this means for practice

• Designers. Ground the model in your own licensed corpus before tuning anything else: the local LLM with no retrieval scored 52.3%, below the TF-IDF baseline (55.4%), while adding RAG (Retrieval-Augmented Generation) alone lifted it to 66.6%.
• Designers. Quantize with fine-tuning rather than after it — quantization-aware training cut the FP16-to-4-bit gap to 1.7 pp (Qwen-7B) and 1.2 pp (DeepSeek-MoE) while reducing VRAM 38.7% and 37.9%, and pulled hallucination back to 8.1% for DeepSeek-MoE where plain quantization had raised it to 10.4%.
• Designers. Spend effort on retrieval quality before caching tricks: removing metadata-enriched embeddings caused the largest accuracy swing (69.8% → 65.3%), while KV caching mainly reduced latency and dynamic batching and curriculum-specific attention masking mainly improved energy efficiency.
• Administrators. Plan around one consumer GPU as the hardware floor (NVIDIA RTX 3060, 12 GB VRAM) and budget energy per query — the most efficient configuration used 1.8 mWh per query, about 0.54 Wh for 30 students submitting 10 queries each — when comparing on-premise deployment against cloud APIs.
• Administrators. Insist on an openly licensed corpus, since open licensing is what allows documents to be indexed, adapted for instruction tuning, and served locally without negotiating rights.

Limitations

• The authors characterize the system as a knowledge-base assistant, not a validated tutor: no controlled learner study was run, so no learning-gain claim can be made.
• Evaluation was performed by the investigators (blinded to model identity), not by independent external judges.
• Quality diagnostics are partial: perplexity was a local 500-token probe rather than a corpus-level measure, and hallucination was scored by a two-stage NLI procedure against retrieved OER chunks (κ = 0.76 agreement with expert judgment).
• The corpus is 82 English-medium open-licensed Markdown documents benchmarked on a single consumer GPU, which limits generalization to other languages and resource settings.

Citation

Shen, X., Feng, L., Hua, S., Liu, D., Xie, Z., & Liu, B. (2026). Towards sustainable AI knowledge-base assistants in computer science education: on-premise deployment and optimization with open educational resources. Frontiers in Psychology, 17, 1843444. https://doi.org/10.3389/fpsyg.2026.1843444