To Facilitate or not to Facilitate: Human and LLM Facilitator Tendencies in Online Discussions

Synthesis: This study asks when (not just how) LLMs should facilitate online discussions, creating PEFK, a corpus standardizing and aggregating facilitation datasets, and running the first survey on facilitation timing with expert facilitators and Large Language Models (LLMs)-as-a-judge models.

Key asymmetry: humans are more cautious while LLMs are excessively eager to facilitate, although both are more certain when judging that facilitation is not needed.

Corrective attempts found trained ModernBert classifiers more reliable than alternative LLM setups, though existing datasets impose a relatively low performance ceiling — a Benchmark-quality finding for automated discussion facilitation.

For online learning, the work informs when AI should intervene in discussion forums (MOOC-style and classroom), connecting facilitation timing to engagement and moderation research.

Key Findings

• The study defines facilitation timing as the decision of when to intervene in a discussion, observing that not intervening when needed can lead to topic derailment or escalation, while intervening when a discussion is going well can irritate participants.
• The authors create PEFK, a corpus standardizing and aggregating all relevant facilitation datasets, enabling direct comparison across prior work.
• They are the first to run a survey on facilitation timing, executed with expert facilitative participants and LLM-as-a-judge models.
• Humans are more cautious about intervening, while LLMs are excessively eager to facilitate; both are more certain when judging that facilitation is not needed.
• Trained ModernBert classifiers perform more reliably than alternative LLM setups for predicting facilitation needs, although current datasets impose a relatively low performance ceiling.

Study Design & Method

Automating facilitation has been attempted with encoder-only classifiers, and LLMs have more recently been championed as the eventual solution; however, prior work indicated LLM facilitators are too eager to intervene, rendering them unusable as autonomous agents — a finding the authors contrast with human tendencies for the first time. The study operationalizes what facilitation is, observes when humans decide to facilitate, and compares those decisions with LLM decisions. Corrective alternatives (different LLM setups) and classifier training on established datasets are then evaluated against the aggregated PEFK corpus.

What this means for practice

• Instructors. Decide when to intervene, not only what to say: in MOOC-style forums and classroom discussion spaces the timing of an intervention matters as much as its content, since stepping in during a discussion that is going well can irritate participants while staying out when it is not can let a topic derail or escalate.
• Instructors. Keep moderation human for now — expert facilitators in this study were cautious about intervening, while LLM judges were excessively eager, so an uncalibrated model will interject where a person would hold back.
• Designers. Calibrate automated facilitation toward human caution before treating a model as an autonomous agent, and treat the intervene/hold decision as a first-class design parameter of Human-in-the-Loop features in Collaborative Learning platforms.
• Researchers. Invest in annotation infrastructure and larger professional facilitation datasets rather than better prompting: trained ModernBert classifiers were the most reliable approach tested, yet the aggregated PEFK corpus imposed a relatively low performance ceiling.

Limitations

• Ten expert raters (11 were recruited; one did not finish) judged 1,224 discussion excerpts sampled from 1,226 chunks, and the authors had to analyze a "worst-case" half-positive/half-negative split because raters disagreed on the same excerpts.
• Raters saw only excerpts of each discussion rather than full threads, while real facilitation decisions are shaped by long-term dynamics, participant history, and community norms that the excerpts omit.
• Rationales were optional to provide and very few were collected, and only two intervention types (positive and negative reinforcement) were labeled, restricting what the labeling can support.
• PEFK's datasets are predominantly English-language and drawn from specific online or institutional settings, so the authors state the findings may not generalize to multilingual environments or culturally distinct moderation norms.

Citation

Tsirmpas, D., Korre, K., & Pavlopoulos, J. (2026). To facilitate or not to facilitate: Human and LLM facilitator tendencies in online discussions.