Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons

Synthesis: This preprint (Watanabe et al., arXiv:2608.20822) examines how learner characteristics interact with dialogue format in LLM- and TTS-generated dialogue-based lessons, applying the Aptitude-Treatment Interaction (ATI) framework. Across three formats — teacher–student (TS), student–student (SS), and teacher–teacher (TT) — 222 first-year high school students rated lessons on ARCS-based Motivation, self-reported learning outcomes, and overall evaluation. Using linear mixed-effects models, the authors found that the effect of learners' Experiential Learning style on motivation was moderated by format: the Concrete Experience (CE) factor interacted positively with the TT format, while the RCE factor (active experimentation through reflection and conceptualization) had a weaker positive effect in the TT than the TS format. Notably, the TT format raised motivation yet received a significantly lower overall evaluation than TS, a divergence the authors link to intrinsic cognitive load. The study argues that dialogue format should be selected according to learner characteristics, supporting Personalized Learning and AI conversational instruction in generative AI education.

Study Design and Materials

The lessons were delivered as part of "General Inquiry Time" in a data-driven career-education program at a public high school, covering data science content across three parts. Each part used a distinct dialogue format — TS (what data science is, ~25 min), SS (what data science can do, ~19 min), and TT (data-driven decision making, ~24 min) — for roughly 68 minutes of total viewing. Scripts were generated by an Large Language Models (LLMs) (Claude 3.5 Sonnet) with format-specific prompts (Expert-Novice, Peer-Peer, and Expert-Facilitator), and speech was synthesized via the Gemini TTS API with distinct prebuilt voices assigned to each speaker role.

Turn counts and utterance lengths differed across formats (178 turns in TS vs 30 in SS and 37 in TT), but a Friedman test found no significant difference across formats in perceived content difficulty or clarity of delivery, supporting the equivalence of conditions. A within-subjects design had all students view TS → SS → TT in a fixed order within one session; because all six classes received lessons simultaneously, counterbalancing was not feasible, confounding format with content and order.

Measuring Learner Characteristics and Outcomes

Learner characteristics were measured with a 16-item experiential learning style scale based on Kolb's theory, adopting the two-factor structure of Ikejiri et al. (CE factor: 4 items, α = .75; RCE factor: 12 items, α = .84), and the 10-item Critical Thinking attitude in Inquiry (CT-I) scale (α = .78), which loaded on a single dimension. Three outcome measures were collected per format: a 4-item ARCS motivation scale (one item each for Attention, Relevance, Confidence, Satisfaction; α = .82–.89), a 2-item self-reported learning-outcomes index (α = .68–.73), and a single-item overall evaluation.

Linear mixed-effects models with participant random intercepts were fit in R (lme4, lmerTest) with grand-mean-centered learner characteristics, the TS format as reference, and VIF-confirmed absence of problematic multicollinearity. The task-relevant measures tie directly to Student Engagement: the RCE factor and critical-thinking disposition showed significant positive main effects on both motivation and learning outcomes, while the CE factor's weak independent main effect was interpreted as a suppression effect given its moderate correlation with RCE.

Interaction Effects on Motivation

For ARCS-based Motivation, both the SS and TT formats elicited significantly higher motivation than the TS format, likely because a peer-to-peer exchange heightened perceived Relevance and Satisfaction. Critically, two interactions reached significance: TT × CE was positive (b = 0.162, p < .001) and TT × RCE was negative (b = −0.238, p = .002), with small-to-medium effect sizes (Cohen's f = .17 and .15). Simple-slope analysis showed the CE factor's slope shifted from negative in TS (−0.151) to near zero in TT (0.011), while the RCE factor's positive slope in TS (0.341) and SS (0.257) shrank to a non-significant 0.102 in TT.

Interpreting this through ATI, the authors argue that for CE-high learners a two-expert dialogue functions as a direct source of concrete, sensory expert knowledge and raises motivation, whereas for RCE-high learners an expert-led exchange leaves little room for reflection and conceptualization — making the TT format a relatively poorer motivational fit. The interaction terms for critical-thinking disposition were not significant.

Motivation–Evaluation Divergence and Free-Response Insights

For learning outcomes, interactions of format with the CE and RCE factors showed only trends toward significance (R² marginal = .129), which the authors attribute to limited power for detecting interactions and to self-reported comprehension differing from motivation's immediate response. For overall evaluation, no learner-characteristic interaction emerged; instead, the TT format received a significantly lower rating than TS (b = −0.126, p = .005). This divergence — the TT format both raising motivation and scoring lower in overall evaluation — is a central finding, interpreted via Cognitive Load Theory as reflecting higher intrinsic load from dense, stiff expert-to-expert dialogue.

The free-response analysis supports this: in the TT format, "stiffness of speech" was the most common improvement label, with difficulty of content and abundant difficult vocabulary also flagged; in the SS format, "unnatural manner of speaking" dominated, likely reflecting a mismatch between the TTS voice and the intended high-school peer register. Both SS and TT drew "unclear speaker roles" comments, suggesting extraneous cognitive load from inferring speaker relationships.

What this means for practice

• Instructional designers. Assign dialogue format by learners' experiential learning style rather than delivering one format to everyone: the TT format raised Motivation for CE-high learners (TT × CE b = 0.162, p < .001), while TS or SS anchored Motivation more reliably for RCE-high learners (TT × RCE b = −0.238, p = .002).
• Edtech designers. Build a routing function into the LMS that administers a brief experiential-learning-style pre-Assessment and recommends or delivers a matching format, and let learners choose among multiple formats to support self-directed learning.
• Instructors. When using the TT format, present an observation frame before viewing (for example, what to take away from a discussion between experts) plus a post-viewing reflection activity, since TT scored significantly lower on overall evaluation than TS (b = −0.126, p = .005) despite its motivational advantage.
• Designers. Disclose speaker roles before SS and TT playback and check the TTS voice register against the intended peer audience: "unclear speaker roles" recurred in free-response comments for both formats, and "unnatural manner of speaking" dominated for SS.
• Instructors. Treat a motivational gain as separate from comprehension — in this study the two diverged — and do not read higher Student Engagement as evidence of learning.

Limitations

• The study ran in a single session (November 12, 2025) at one public high school with 222 first-year students in six classes of roughly 40 each, so findings do not extend across school types, grade levels, or time.
• Learning outcomes rested on two self-report items with no objective test, delayed post-test, or retention/transfer task, and self-report may overestimate learning precisely in the TT format that also raised motivation.
• Format was confounded with lesson content (TS, SS, and TT each carried a different part of the curriculum) and every student viewed lessons in the fixed order TS → SS → TT with no counterbalancing, making format and presentation order statistically inseparable.
• ARCS motivation used one item per component (versus 12 items on the RIMMS scale) and overall evaluation a single item, and all significant interaction effect sizes were small to medium (Cohen's f = .17 and .15), which the authors treat as preliminary evidence.

Citation

Watanabe, F., Suko, T., Ishida, T., Kuma, Y., Kobayashi, M., Hirasawa, S., & Kumoi, G. (2026). Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons.