A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant

Synthesis: This IAAI-27 paper presents a Prompt Engineering framework for real-time, micro-level personalization of the Jill Watson Large Language Models (LLMs)/RAG-based AI teaching assistant. By conditioning responses on six learner dimensions (metacognitive Self-Assessment, abstraction, verbosity, perception, processing, understanding) and classifying each query's cognitive demand with Bloom's Taxonomy, the framework yields 96 distinct learner profiles and adapts the form of responses (abstraction, structure, verbosity, framing) without retraining the model. NLP analyses of 2,910 responses and a human study with five evaluators found measurable, perceptible response differences across personalization conditions, with abstraction and processing preferences most consistently aligned with their intended effects. It offers a scalable, modular route to hybrid adaptability-and-adaptivity in AI tutoring.

From content adaptation to response-form adaptation

The aspiration to tailor learning to individual needs is decades old, anchored by Bloom's 2 Sigma finding that one-to-one tutoring can produce ~two-standard-deviation gains [Editorial Note: this has been refuted]. Earlier adaptive systems followed a diagnose–prescribe cycle that adapted content or sequencing through learner modeling. This work shifts personalization toward response-form adaptation: answers remain grounded in a shared retrieved knowledge base (via retrieval-augmented generation), while their abstraction, structure, verbosity, and cognitive framing are modified through prompt conditioning.

The framework combines learner-driven adaptability (students explicitly select their preferences) with system-driven adaptivity (the system infers query cognitive demand). Learner preferences derive from the Felder-Silverman learning model and are treated as user-selected preferences rather than fixed psychometric classifications; the six dimensions control granularity, length, communication orientation, engagement style, and organizational structure. The combination produces 96 distinct learner profiles.

Personalizing the Jill Watson teaching assistant

The framework operates on Jill Watson, a general-purpose LLM/RAG teaching assistant deployed across multiple institutions. Each student question is automatically classified by a fine-tuned BERT-based Bloom's Taxonomy classifier (0.92 test accuracy), and learner attributes are encoded into a structured prompt that conditions generation while preserving grounding in course-specific retrieved content. An example prompt declares: "I have a beginner level of knowledge in this topic. The Bloom's Taxonomy category of my question is Synthesis. Please provide a technical and concise response, using a sensory communication style..." — a modular design that lets preferences update at runtime with no model retraining or domain-specific authoring.

Evaluation results

The framework was evaluated in two complementary ways:

• NLP experiments: 30 real student questions from a Spring 2023 Knowledge-Based AI course, 96 personalized plus one baseline prompt configuration each, yielding 2,910 GPT-4.1 responses. Responses showed high semantic similarity but substantially lower lexical similarity — remaining grounded while varying in surface expression — with systematic variation in response length by verbosity preference and greater response complexity associated with higher self-assessment, verbosity, reflective processing, and technical abstraction.
• Human evaluation: five evaluators rated responses on quality, perceived complexity, abstraction, and processing style across 13 profiles per question (390 evaluations). Mixed-effects models found abstraction and processing preferences significantly predicted perceived response complexity, abstraction, and processing style — consistent with their intended effects — while self-assessment showed weaker effects. Bloom's level also predicted ratings, with more complex questions generally scoring lower.

The authors frame the results as preliminary evidence that prompt-based personalization produces measurable, perceptible differences in response characteristics, supporting its potential for adaptive behavior in LLM-powered educational agents. Deployment is planned for Spring 2027 with a pilot in Georgia Tech courses, followed by larger classroom evaluation, A/B testing, and assessment of impacts on learning outcomes and engagement.

What this means for practice

• Designers. Encode learner preferences as runtime prompt attributes rather than retraining or domain-specific authoring, so personalization can be updated between sessions with no model changes.
• Designers. Prioritize abstraction and processing-style conditioning when resources are limited: those dimensions produced the perceptible shifts in complexity and style, while self-assessment showed weaker effects.
• Designers. Keep every personalized response grounded in one shared retrieved knowledge base and vary only surface form; responses stayed semantically similar while differing lexically, which is what "personalized but accurate" should look like quantitatively.
• Researchers. Report lexical and semantic similarity together when claiming personalization worked — high semantic similarity alone would be evidence that the prompt attributes changed nothing.
• Designers. Plan the classroom evaluation to measure learning outcomes and engagement, since the current evidence covers response characteristics rather than learning.

Limitations

• The human evaluation used five evaluators recruited from current and former Georgia Tech KBAI students; the authors call the findings exploratory rather than population-level and state that the small evaluator sample limits generalization.
• The design had power to estimate main effects only — interaction effects among the personalization dimensions could not be estimated, and larger, more diverse samples are needed.
• The NLP evaluation covers 2,910 generated responses to 30 real student questions from a single Spring 2023 Knowledge-Based AI course, measuring response characteristics rather than learner outcomes; effects on learning and engagement are unmeasured.
• Classroom deployment had not occurred at the time of writing: integration into Jill Watson was targeted for Spring 2027 and pilot deployment in selected Georgia Tech courses for Summer 2027, with broader rollout in 2027–2028.

Citation

Basu, S., Kakar, S., & Goel, A. (2026). A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant. arXiv:2609.03402 (IAAI-27).