---
source_url: https://www.nber.org/papers/w35397
ingested: 2026-08-03
sha256: 25bd8cab2324b26d45a43a4682b3a375da190eb99f0a4bc4d4c7f7b2be327241
---

# Let's Chat: Leveraging Chatbot Outreach for Improved Course Performance

Katharine E. Meyer, Lindsay C. Page, Catherine Mata, Eric Smith, Brendan Tyler Walsh, Candice L. Fifield, Michelle Tyson, Amy E. Eremionkhale, Michael Evans, Shelby Frost, Eye Eoun Jung. NBER Working Paper No. 35397, July 2026. JEL I21, I23, O33.

## Design

Two pre-registered RCTs (REES 8160 Government; 13760 Microeconomics) at Georgia State University (GSU), in large-enrollment asynchronous online courses: Introduction to American Government and Principles of Microeconomics (multiple sections, two professors). Half of enrolled students randomly assigned to receive a **non-generative AI academic chatbot** (Mainstay) outreach: 2-3 scheduled customized text messages/week (assignment due dates, targeted nudges for missing work, encouragement), with personalization (e.g., "Hi FIRSTNAME") and targeting (differentiated messages by missing-assignment status). Students could text the bot 24/7 and receive AI-generated responses from a pre-programmed content knowledge base built with administrators and the instructional team; messages the bot couldn't answer were flagged to the course TA.

## Results

- Across both courses: **+4 percentage points in likelihood of earning an A or B**; similar across courses.
- **Microeconomics women: final grades +7 points** vs control women; **+11 pp more likely to earn A/B**; **10 pp less likely to DFW**. No treatment effects for men (control men and women performed similarly, so treated women outperformed both).
- Otherwise generally similar effects across demographics.
- **Mechanisms:** treated students more likely to attend university tutoring; suggestive evidence of higher homework completion in Microeconomics (multiple weekly assignments); no differential assignment completion in Government.
- **No spillover:** no effects on other courses taken that semester, nor on next-semester enrollment/performance — improves course-specific outcomes without (yet) developing underlying habits/study skills.
- **Student enthusiasm:** 82% of survey respondents recommended continuing the chatbot and expanding to other GSU courses.

## Significance

- Evidence that virtual, low-touch, non-generative AI outreach can improve course performance in large-enrollment online courses — a modality with documented negative outcomes.
- Contrasts with limited effects found for other low-touch outreach (e.g., Oreopoulos & Petronijevic 2019): message timing/customization likely matters.
- Task-completion channel: ongoing, dynamic tasks (weekly assignments) respond to nudge-style outreach better than one-shot academic inputs.