Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons

Synthesis: This preprint (Watanabe et al., arXiv:2608.20822) examines how learner characteristics interact with dialogue format in LLM- and TTS-generated dialogue-based lessons, applying the Aptitude-Treatment Interaction (ATI) framework. Across three formats — teacher–student (TS), student–student (SS), and teacher–teacher (TT) — 222 first-year high school students rated lessons on ARCS-based Motivation, self-reported learning outcomes, and overall evaluation. Using linear mixed-effects models, the authors found that the effect of learners' Experiential Learning style on motivation was moderated by format: the Concrete Experience (CE) factor interacted positively with the TT format, while the RCE factor (active experimentation through reflection and conceptualization) had a weaker positive effect in the TT than the TS format. Notably, the TT format raised motivation yet received a significantly lower overall evaluation than TS, a divergence the authors link to intrinsic cognitive load. The study argues that dialogue format should be selected according to learner characteristics, supporting Personalized Learning and AI conversational instruction in generative AI education.

Study Design and Materials

The lessons were delivered as part of "General Inquiry Time" in a data-driven career-education program at a public high school, covering data science content across three parts. Each part used a distinct dialogue format — TS (what data science is, ~25 min), SS (what data science can do, ~19 min), and TT (data-driven decision making, ~24 min) — for roughly 68 minutes of total viewing. Scripts were generated by an Large Language Models (LLMs) (Claude 3.5 Sonnet) with format-specific prompts (Expert-Novice, Peer-Peer, and Expert-Facilitator), and speech was synthesized via the Gemini TTS API with distinct prebuilt voices assigned to each speaker role.

Turn counts and utterance lengths differed across formats (178 turns in TS vs 30 in SS and 37 in TT), but a Friedman test found no significant difference across formats in perceived content difficulty or clarity of delivery, supporting the equivalence of conditions. A within-subjects design had all students view TS → SS → TT in a fixed order within one session; because all six classes received lessons simultaneously, counterbalancing was not feasible, confounding format with content and order.

Measuring Learner Characteristics and Outcomes

Learner characteristics were measured with a 16-item experiential learning style scale based on Kolb's theory, adopting the two-factor structure of Ikejiri et al. (CE factor: 4 items, α = .75; RCE factor: 12 items, α = .84), and the 10-item Critical Thinking attitude in Inquiry (CT-I) scale (α = .78), which loaded on a single dimension. Three outcome measures were collected per format: a 4-item ARCS motivation scale (one item each for Attention, Relevance, Confidence, Satisfaction; α = .82–.89), a 2-item self-reported learning-outcomes index (α = .68–.73), and a single-item overall evaluation.

Linear mixed-effects models with participant random intercepts were fit in R (lme4, lmerTest) with grand-mean-centered learner characteristics, the TS format as reference, and VIF-confirmed absence of problematic multicollinearity. The task-relevant measures tie directly to Student Engagement: the RCE factor and critical-thinking disposition showed significant positive main effects on both motivation and learning outcomes, while the CE factor's weak independent main effect was interpreted as a suppression effect given its moderate correlation with RCE.

Interaction Effects on Motivation

For ARCS-based Motivation, both the SS and TT formats elicited significantly higher motivation than the TS format, likely because a peer-to-peer exchange heightened perceived Relevance and Satisfaction. Critically, two interactions reached significance: TT × CE was positive (b = 0.162, p < .001) and TT × RCE was negative (b = −0.238, p = .002), with small-to-medium effect sizes (Cohen's f = .17 and .15). Simple-slope analysis showed the CE factor's slope shifted from negative in TS (−0.151) to near zero in TT (0.011), while the RCE factor's positive slope in TS (0.341) and SS (0.257) shrank to a non-significant 0.102 in TT.

Interpreting this through ATI, the authors argue that for CE-high learners a two-expert dialogue functions as a direct source of concrete, sensory expert knowledge and raises motivation, whereas for RCE-high learners an expert-led exchange leaves little room for reflection and conceptualization — making the TT format a relatively poorer motivational fit. The interaction terms for critical-thinking disposition were not significant.

Motivation–Evaluation Divergence and Free-Response Insights

For learning outcomes, interactions of format with the CE and RCE factors showed only trends toward significance (R² marginal = .129), which the authors attribute to limited power for detecting interactions and to self-reported comprehension differing from motivation's immediate response. For overall evaluation, no learner-characteristic interaction emerged; instead, the TT format received a significantly lower rating than TS (b = −0.126, p = .005). This divergence — the TT format both raising motivation and scoring lower in overall evaluation — is a central finding, interpreted via Cognitive Load Theory as reflecting higher intrinsic load from dense, stiff expert-to-expert dialogue.

The free-response analysis supports this: in the TT format, "stiffness of speech" was the most common improvement label, with difficulty of content and abundant difficult vocabulary also flagged; in the SS format, "unnatural manner of speaking" dominated, likely reflecting a mismatch between the TTS voice and the intended high-school peer register. Both SS and TT drew "unclear speaker roles" comments, suggesting extraneous cognitive load from inferring speaker relationships.

Implications and Limitations

The authors argue for format selection based on learner characteristics rather than uniform delivery: prioritizing the TT format for CE-high learners to enhance motivation, while anchoring TS or SS formats for RCE-high learners to sustain stable motivation and outcomes. Practical recommendations include an LMS routing function that administers a brief experiential-learning-style pre-Assessment and recommends a format, and, for the TT format, presenting an observation frame before viewing plus a post-viewing reflection activity to manage intrinsic cognitive load. This aligns with Self-Determination Theory-informed, Active Learning-oriented design and the broader push toward personalized instruction with Conversational AI.

Limitations include self-report-only learning outcomes (no objective or retention tests), a single-school single-session sample, confounding of format with content and fixed presentation order, and abbreviated one-item-per-component ARCS and single-item overall-evaluation measures. Because all effect sizes for significant interactions were small to medium, the authors frame their results as preliminary evidence for personalized learning.

Connected Concepts

• Language Learning
• Student Engagement
• Motivation
• Critical Thinking
• Experiential Learning
• Generative AI
• Conversational AI
• Self-Determination Theory
• Higher Education
• Active Learning
• Speech and Voice Technologies

Connected Articles

• AI-Guided Learning: Research on Knowledge and Skill Acquisition Support Methods Using Deep Learning Audio-Video Processing Techniques — AI-guided audio/video learning systems and learner adaptation
• Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation — Teacher-authored prompts for student-AI dialogue
• Building AI Companions that Prioritise Learning over Performance — Framework for AI learning companions
• EduGage: Methods and Dataset for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning — Assessing student engagement in video-based learning
• Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency — AI feedback and critical thinking in writing

Citation

Watanabe, F., Suko, T., Ishida, T., Kuma, Y., Kobayashi, M., Hirasawa, S., & Kumoi, G. (2026). Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons.