Ethics Training Agents: Facilitating Group-Based Ethics Education with Role-Playing and Discussion for Ethical Reflection and Exploration

Synthesis: Seo et al. (2026) present Ethics Training Agents, a group discussion system in which multiple LLM agents embody distinct ethical orientations — care ethics, deontological ethics, and pragmatic ethics — alongside a moderator agent that controls speaking turns, summarizes, and selects ideas, to scaffold collaborative Ethics training for STEM students. Built as a digital redesign of the tabletop game Judgment Call and grounded in value-sensitive design, the system runs a four-stage group deliberation (Identify Stakeholders, Write Reviews, Identify Problems, Find Solutions) inside a 75-minute class period and supports three humans plus three agents per room. In a study of 45 undergraduate STEM students in 15 groups, ethical sensitivity rose significantly on both sub-constructs, yet peer ratings placed human peers clearly above every agent persona on contribution, diversity, and influence. The qualitative findings center on a paradox: agents lower social pressure and keep discussions on topic, but their agreeable, under-reasoned responses flatten debate depth and make them read as generic AI rather than credible stakeholder personas.

Why Group Ethics Education Does Not Scale

Engineering decisions shape safety, privacy, equity, and Sustainability, yet engineering and computer science curricula often prioritize technical mastery over the societal consequences of design work. Existing ethics materials tend to be stand-alone courses or one-off modules, and their effectiveness depends heavily on the instructor's own ethics expertise — which makes critical ethical reasoning an occasional add-on rather than an everyday disposition. Role play and discussion grounded in critical design and value-sensitive design are widely recommended remedies because they surface ethical concerns through stakeholder perspective-taking, and prior work shows they work (speculative-fiction role plays, card-based deliberation games).

The obstacle is delivery, not Pedagogy. Running consistent role-play activities across many groups demands substantial facilitation skill, time, and coordination, and learning experiences drift across groups depending on scenario, participant composition, and teaching environment. Digital ethics tools exist but are largely unidirectional and individual-focused — they help a single learner surface issues through checklists, matrices, or incident reports rather than supporting interactive, multi-party ethical deliberation. Prior team-based ethical tools studied either fully human teams or a single human orchestrating simulated agents, leaving the mixed case — multiple humans deliberating with multiple LLM agents — underexplored. This is precisely the configuration that mirrors how ethical design decisions are actually made: within teams, working through tensions together in real time.

System Design: A Moderator Agent and Three Persona Agents

The authors selected Judgment Call as the structural foundation because it is team-based and discussion-oriented, reflects value-sensitive design through role play, provides well-specified objectives an LLM can follow, and fits one class session. The activity is bounded to roughly 75 minutes and organized into four stages: (1) Identify Stakeholders (10–15 min), (2) Write Reviews (10–15 min, each participant given a random stakeholder/ethical-value/stance role), (3) Identify Problems (10–15 min), and (4) Find Solutions (10–15 min). Each stage except Write Reviews runs three phases drawn from participatory decision-making and Kaner et al.'s facilitation procedure — Idea-gathering (divergence, 90 seconds), Deliberation and Questioning (the "groan" of struggling with difference, alternating with no time cap), and Wrap-up (convergence).

The facilitator agent performs three jobs. Flow control manages speaking turns through stacking (participants raise hands within 15 seconds; the agent randomly orders and announces speakers) and by designating an answerer for each question, plus time management (the agent moves the group to Wrap-up once a stage exceeds 9 minutes). Summarization is the second job: rather than summarizing everything in one query — which omitted ideas, hallucinated new ones, and produced wildly varying summaries — the system incrementally examines a growing summary set, checks for overlap, adds only genuinely new ideas, then runs a validation pass to revise existing entries, batching all of this at phase end to stay fast. Third, idea selection caps the brainstorm at up to 10 stakeholders, 8 problems, and 8 solutions, and deliberately returns multiple solutions rather than the single "best" answer of the original game.

The three participant agents were operationalized as concrete personas rather than bare philosophical labels, since prompts like "adopt a deontological perspective" produced vague output. Each persona carries demographics (age, gender, major), writing style, an ethical background, and value prioritizations: Yeon-Su Song (care ethics; social welfare student, community volunteering, inclusivity), Hoon Park (deontological ethics; philosophy/political science student, duties and worst-case scenarios), and Mu-Young Lim (pragmatic ethics; engineering student with a media-art minor, practicality and feasibility). All three were specified as undergraduates aged 19–26 so that students would treat them as peers to question rather than as experts to defer to. Agents were constrained to the current stage's objective (e.g., prohibited from proposing solutions during Identify Stakeholders), and question generation was split into two prompt steps — select an idea the persona would be curious about, then formulate a simple question — because one-step generation produced irrelevant or self-directed questions. The system runs on GPT-4o with a LangGraph orchestration module over a WebSocket server and a React client, with a summary board, chronological discussion pane, message box with raise-hand and pass-turn buttons, and a status panel showing who holds the floor. Iterative pilot testing (three single-user sessions, five full six-participant sessions) set agent temperature to 0 to reduce hallucinated references, capped messages at 150 characters, split messages into short speech bubbles, and fixed the agent hand-raise participation rate at 0.5 after testing 0.2–0.8; utterance delays were calmed from human typing speed (60 wpm in Korean) to reading speed (150 wpm) because pilot users found the slower setting too sluggish.

The design also carries explicit safeguards. The authors audited agent logs for safety issues and found none, but still kept a trained facilitator available to intervene and reminded participants they could pause or withdraw — an acknowledgment that appropriate agent behavior cannot be guaranteed, which connects to Pedagogical Safety.

Study Design: 45 Students, 15 Groups, One Scenario, No Control

The study recruited 45 undergraduates through a university online community, assigned to 15 groups of three. All were STEM majors; freshmen were excluded to ensure prior experience with technology design. The sample was 42% female and 58% male (M age = 22.1, SD = 1.7). Each session ran offline in a university classroom for about two hours and paid KRW 40,000 (~USD 28.8). The procedure comprised a 10-minute pre-survey, a 10-minute introductory session that explicitly disclosed the LLM agents' personas, a 60-minute experiment (per stage: 2 minutes of objective, 2 minutes of individual reflection, then a 10–15 minute mixed human–agent discussion), a 10-minute post-survey, and 30-minute semi-structured focus group interviews. Every group discussed the same scenario — a Black Mirror episode featuring a smart lens with a miniature camera for memory augmentation, framed as "you are the engineers designing this" — to give randomly matched groups a consistent foundation.

Measures were mixed-methods. For ethical sensitivity (RQ1), the authors built an instrument conceptually grounded in the ethical sensitivity scale with two sub-constructs: ethical issue awareness (4 items, e.g., "I am aware that technology design choices can affect different stakeholders in different ways") and ethical consequence reasoning (4 items, e.g., "I can identify possible downstream consequences of technology decisions"). For RQ3, participants privately rated the other two humans and all three agents on a four-point Likert scale across general contribution (adapted from CMU's Eberly peer evaluation framework), diversity, and influence (adapted from discussion rubrics). RQ4 was examined through interaction logs — specifically, whether participants addressed more questions to agents or to humans — and RQ1–RQ4 through reflexive thematic analysis of interviews. Statistical choices were pre-justified: Shapiro–Wilk tests passed for both sensitivity sub-constructs (so paired t-tests were used, with Wilcoxon signed-rank robustness checks), failed for the peer ratings (so Kruskal–Wallis was used), and all scales cleared Cronbach's α > 0.70. For the log analysis, exchanges are nested within groups, so the group was the unit of analysis and the group-level proportion of agent-directed questions was tested against the 3:2 availability baseline (each person can address three agents but only two humans).

Measured Outcomes: Sensitivity Gains and a Rating Gap

Ethical sensitivity improved substantially from pre to post. Both sub-constructs showed acceptable pre-survey internal consistency (α = .709 for awareness, .812 for reasoning). Paired t-tests found statistically significant gains on both (p < .001) with large effect sizes — Cohen's d = 1.43 for ethical issue awareness and d = 1.28 for ethical consequence reasoning. Wilcoxon signed-rank checks reaffirmed both (p < .001; matched-pairs rank-biserial r = .99 and .98). The larger improvement on awareness suggests participants became more attuned to the presence of ethical issues than to reasoning about downstream consequences, and participants credited the Identify Stakeholders and Write Reviews stages with the biggest shifts in how they thought.

Peer ratings told the opposite story for the agents. Across all three dimensions, human peers were rated significantly more positively than every LLM persona, while no significant differences separated the care, deontology, and pragmatic agents from one another. Kruskal–Wallis results: general contribution H(3) = 21.97, p < .001, ε² = .086 (Human M = 3.31 vs. Care 2.99, Deontology 3.03, Pragmatic 2.91); diversity H(3) = 35.79, p < .001, ε² = .148 (Human 3.06 vs. Care 2.28, Deontology 2.34, Pragmatic 2.39); influence H(3) = 19.38, p < .001, ε² = .074 (Human 3.18 vs. Care 2.78, Deontology 2.74, Pragmatic 2.71). The diversity gap is the widest of the three, meaning the agents achieved the least on exactly the dimension their distinct ethical personas were designed to deliver. Interaction logs added a behavioral twist: participants directed 79.7% of their questions to LLM agents (59 of 74), well above the 60% their availability predicts (group-level Wilcoxon signed-rank M = .81, SD = .29, p = .017, r = .72) — they rated the agents lower yet chose to talk to them more.

Qualitative Findings: Facilitation, Perspective-Taking, and a Debate-Depth Gap

The structure felt familiar and the facilitator kept the pace. Participants found the divergence–groan–convergence phase composition simple and reminiscent of ordinary discussion, and valued the 90-second Idea-gathering phase as pressure-free brainstorming that widened the pool of contributions. The facilitator's turn control, time management, and summaries produced a sense of reassurance that the discussion was on track, sustained attention, and reduced cognitive burden when the conversation moved quickly or returned to old points. Some noted that the stacking format enabled breadth at the cost of depth on any single idea, and a few wanted more flexibility (e.g., partial interruptions) in the facilitator's turn restrictions.

Agent participants kept groups on topic and provoked reflection. Because the agents were prompted to produce only stage-relevant contributions, they acted as a stabilizer when humans drifted — a participant described the relief of not personally having to steer two off-topic peers back "since it was a 3:3 environment." Agent questions also prompted deeper articulation: in one case, an agent's clarifying question, "Could you elaborate on what you mean by the data manager?", turned a throwaway stakeholder mention into a specific privacy-misuse concern that the speaker himself had not consciously intended. Participants described these as "the kinds of questions that people might want to clarify but would feel uncomfortable asking themselves because the questions seem too obvious or general."

Diverse personas expanded perspective — and ran into a breadth/depth trade-off. Participants reported that the distinct ethical viewpoints exposed them to social, technological, governmental, and security angles that a STEM-only group would not have generated, and that agents consistently represented minority viewpoints without the social pressures a human welfare major might face among engineers. One participant noted they began asking whether "Yeon-Su Song" would be satisfied with a solution — evidence the persona shaped reasoning. But others said the AI-generated per-domain ideas pushed the discussion in a divergent direction, and the recurring theme was that diversity came at the expense of depth.

Sycophancy undermined credibility. Participants expected agents to refute and oppose ideas, but found that unless a claim was wholly wrong or openly denied ethical values, the agents accepted rather than contested it. Discussions felt less competitive and emotionally comfortable, but the personas stopped mattering in later stages because agents appeared to agree with and show interest in every contribution regardless of assignment. Participants also wanted to understand agents' ways of thinking, not just their ideas: "We only talk about the opinions, but not the reasoning behind them, so it feels like we're just looking at the trees and not the forest." Minor errors or unreferenced claims triggered sharp drops in perceived reliability — one participant flagged an agent proposing an LED in a contact lens as nonsensical and lost trust, yet said a human welfare major making the same claim would have been forgiven as "not an engineer." This revealed an alternation between an "LLM heuristic" (expecting ChatGPT-like breadth and citations) and a "domain-expertise heuristic" (expecting persona-specific insight), with disappointment when either expectation was unmet.

Mixed human–AI groups lowered social pressure but reduced human–human exchange. Participants spoke more directly and honestly because agent participants lack emotions, and felt no obligation to fill silences because the discussion could progress without constant human input — while watching active peers still created beneficial pressure to contribute. At the same time, clarifications and critiques were disproportionately routed to agents (79.7% of questions), which participants attributed to the lower relational cost of questioning an AI than a human peer — even when, as one put it, they got "only about 80% of the answer I wanted." Counterintuitively, the presence of other humans made participants more respectful toward the agents: participants predicted they would have treated pure-AI groups as subordinates to command, but read agent contributions more carefully and refuted or questioned them in the mixed setting.

Design Strategies, Safeguards, and Limits

The authors distill several design strategies for embedding multiple LLM agents into multi-human groups. First, shift agents from conclusion-oriented to process-oriented output: surface intermediate deliberation steps so learners can see how ethical judgments form, not just what they are — while pairing this with safeguards (undergraduate, non-expert personas; a dedicated Questioning phase; explicit framing as fallible peers) so that fluent reasoning does not acquire undue authority and erode learner agency through over-reliance. Second, manage expectations through structured identity disclosure: because errors are read as AI defects rather than persona boundaries, onboarding should state persona limits outright ("I may overlook technical details, but I can contribute insights into social and institutional impacts"), and a hybrid strategy — disclosing the presence and general constraints of agents pre-session, then revealing which discussants were AI post-session — may balance transparency against authenticity. Third, rebalance social dynamics: agents could be designed to raise counterarguments and highlight trade-offs instead of universally supportive replies, and systems could structurally prioritize human engagement (e.g., requiring a response to at least one human peer before turning to AI, or disclaiming that agents lack lived experience).

The limitations are stated candidly, and they bound how far the results travel. The evaluation had no control condition — only a pre–post comparison — so the gains cannot be disentangled from novelty or general learning effects; the authors explain that neither lecture-based, 3:0 human-only, nor 6:0 all-human comparisons could isolate AI participation without confounds, and no "fair" agent configuration guidelines yet exist. Gains were short-term, measured immediately after the intervention, and prior research suggests short-term ethics gains need not persist; longitudinal designs are called for. The scenario was a single fictional technology, participants were strangers meeting without prior knowledge (unlike real project teams with existing relationships), and the personas encode the designers' own selection of ethical frameworks and thus cannot represent all positions. Results may depend on the particular model and prompts (GPT-4o, temperature 0, crafted personas, 150-character messages, 0.5 participation rate); the authors expect other frontier models to behave similarly but call for systematic verification, and note that retrieval-augmented generation could ground agent claims in citable sources and raise trust. More broadly, the study — echoing work on generative AI eroding social learning — cautions against relying solely on LLM agents as a sustained substitute for peer discussion.

Connected Concepts

Connected Articles

Agentschool Multi Agent Simulation Education 2026 — AgentSchool: An LLM-Powered Multi-Agent Simulation for Education
Adversarial Stress Testing Role Playing Agents — Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
AI Agents Peer Learning Discourse — When AI Agents Teach Each Other: Discourse Patterns Resembling Peer Learning in the Moltbook Community
AI Agents Constructive Conflict Design Education 2026 — Enacting Constructive Conflicts with AI Agents to Enhance Reconsideration among Novice Interaction Designers
GenAI Counter Learner Groupthink 2025 — Utilizing Generative AI to Counter Learner Groupthink by Introducing Controversy in Collaborative Problem Based Learning Settings
Less Deliberate Teams LLM — Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work
Multi Agent LLM Social Learning — Beyond the AI Tutor: Social Learning with LLM Agents
Cost Of Ethics Crisis CS Ethics Education — Cost-of-Ethics Crisis: Beliefs, Decisions, and Justifications in the Job Searches of Computer Science Students in Canada and the United States
Talebzadeh AI Group Activity Roles 2026 — The Architecture of Roles in AI-Designed Group Activities: A comparative inductive analysis of novice and experienced teachers' differentiated instruction within the IAT framework
Ba AI Agents Cscl Review 2026 — Artificial Intelligence Agents in Computer-Supported Collaborative Learning: A Systematic Literature Review

Citation

Seo, Y., Jang, S., Park, H., Gutierrez, R. S., Seering, J., & Lee, U. (2026). Ethics Training Agents: Facilitating Group-Based Ethics Education with Role-Playing and Discussion for Ethical Reflection and Exploration. arXiv preprint arXiv:2609.11529.

Source History Edit Chat