AI Empathy as a Human Disposition in Generative AI Interaction: Development and Preliminary Psychometric Evaluation of a Cognitive–Affective–Motivational Scale

Synthesis: Kang, Lee and Kang developed and preliminarily evaluated the 42-item AI Empathy Scale for Generative AI Interaction (AIES-G), which treats AI empathy as a human disposition expressed within Generative AI interaction rather than as a property of the system. Two hundred one participants in four intact educational cohorts completed the scale before and after GenAI-supported coursework, and the authors examined the post-test factor structure with exploratory factor analysis alongside internal consistency and pre-test–post-test responsiveness. Parallel analysis and Velicer's MAP criterion both supported six factors (48.2% of item variance), but the items did not reproduce the six intended subfactors: Prosocial Intention formed no factor of its own and two retained factors did not represent intended content dimensions. Internal consistency was acceptable to strong and scores excluding Personal Distress increased (dz = 1.46), while Personal Distress met the prespecified equivalence criterion. The key qualification is that one exploratory sample supplied the structure, reliability and responsiveness evidence, so the framework is provisional.

Key Findings

• The post-test items were factorable and six factors were retained. KMO = 0.859, Bartlett's χ2 (861) = 4225.91, p < 0.001, and all 42 items had individual MSA values above 0.70 (range 0.724–0.931; M = 0.851). Parallel analysis and MAP both indicated six factors, accounting for 48.2% of cumulative item variance with RMSR = 0.041, and maximum absolute pattern loadings ranged from 0.327 to 0.793.
• Six retained factors did not match the six intended subfactors. All seven Imaginative Self-Projection items and all seven Personal Distress items loaded primarily on separate factors, as did five of the seven Empathic Concern items. Perspective Taking and Reflective Inquiry items loaded primarily on the same factor. Prosocial Intention formed no factor of its own, and its items were distributed across four factors. One retained factor consisted mainly of negatively keyed items (PT7, EC7, PI7, RI7), and another comprised three items from different subfactors (EC6, PI2, RI3); two of the six retained factors did not represent intended content dimensions.
• Removing the negatively keyed items left five factors. For the remaining 36 items, parallel analysis and MAP both indicated five factors (48.6% of cumulative variance; RMSR = 0.044); three Perspective Taking items loaded with Imaginative Self-Projection and two Prosocial Intention items loaded with Empathic Concern. The authors conclude that the items do not empirically support the proposed six-subfactor distinction, particularly within the motivational domain.
• Fourteen items were flagged for cross-loading or weak differentiation. Eight had absolute loadings of at least 0.30 on two factors, and six more had a difference below 0.20 between their two largest absolute loadings, even though every item had a maximum absolute loading of at least 0.30.
• Reliability was high for the total scale and acceptable to good for the subfactors. Cronbach's alpha was 0.923 at pre-test and 0.916 at post-test for the 42-item total. Subfactor alphas ranged from 0.762 (Perspective Taking) to 0.844 (Imaginative Self-Projection) at pre-test and from 0.756 (Empathic Concern) to 0.867 (Imaginative Self-Projection) at post-test. At post-test McDonald's omega tracked alpha closely (PT 0.786, ISP 0.869, EC 0.792, PD 0.832, PI 0.781, RI 0.802), and total omega was 0.949 at pre-test and 0.945 at post-test.
• Change was differentiated by domain. Cognitive empathy showed the largest increase (dz = 1.49), followed by motivational empathy (dz = 1.25), with affective empathy more modest (dz = 0.42). The total score excluding Personal Distress rose from 2.946 to 3.528, t(200) = 20.67, p < 0.001, dz = 1.46. Perspective Taking showed the largest subfactor change (+0.869, t(200) = 22.52, dz = 1.589).
• Personal Distress met the equivalence criterion rather than merely failing to change. PD showed no mean change, t(200) = −0.26, p = 0.795, dz = −0.02, and a TOST equivalence test placed the difference within ε = ±0.365 (0.5 × SDpre), with the 90% CI [−0.068, 0.049] entirely inside the bounds. All four cohorts increased in total score: Group 1, dz = 1.54; Group 2, dz = 1.36; Group 3, dz = 1.09; and Group 4, dz = 1.44. The only significant Cohort × Time interaction was for Prosocial Intention, F(3, 197) = 5.88, p < 0.001.

How the scale and its framework were built

The AIES-G operationalizes a 3 × 2 framework that crosses cognitive, affective and motivational domains with other- and self-oriented responses, yielding Perspective Taking, Imaginative Self-Projection, Empathic Concern, Personal Distress, Prosocial Intention and Reflective Inquiry. The authors are explicit that the construct describes the human side of Human AI Collaboration: GenAI is the eliciting context, not a sentient target, and references to the system's "perspective" mean a user's functional model of how prompts, context and constraints may shape a response.

The paper reports the development sequence as a 54-item candidate pool (9 items per subfactor), a two-round Delphi review by three experts using an operational I-CVI ≥ 0.67 retention criterion, deletion of 12 items for insufficient content validity, unclear wording or overlap with adjacent constructs, and a think-aloud pilot with 15 students. The finalized instrument has 42 items, seven per subfactor, with one negatively keyed item per subfactor, rated 1–5.

What the factor analysis adds to the construct claim

The retained-factor count matched the 3 × 2 framework, but the item-level pattern did not; the paper presents the six-subfactor structure as provisional and the instrument as a preliminary research tool rather than a finalized diagnostic. Three additional results bound the interpretation. First, Harman's single-factor diagnostic produced a first unrotated factor with an eigenvalue of 10.53, accounting for 25.1% of total item variance, which indicates shared variance among dimensions without a single dominant factor; the authors note the procedure cannot establish the absence of common method bias. Second, a polychoric sensitivity analysis reproduced the Pearson solution with Tucker's congruence coefficients of 0.996 to 0.999 (M = 0.997). Third, pre-test and post-test solutions agreed strongly for Empathic Concern (0.970) and Personal Distress (0.981) but ranged from 0.752 to 0.859 for the other four factors, suggesting the cognitive and motivational items may have behaved differently across occasions.

The scale is also positioned against neighboring constructs rather than as a replacement for them: AI Literacy covers knowledge and competence, Critical Thinking in AI use covers verification and reflection, and Explainable AI work targets users' mental models of system behavior. The authors argue that AI empathy remains distinct because it adds affective response and self- and other-oriented motivational readiness, and they flag discriminant and incremental validity as unresolved. Within the instrument, Personal Distress should not be read as uniformly desirable: it indexes self-oriented anxiety, frustration, confusion or perceived loss of control, and relates to Anxiety and Stress rather than to constructive engagement.

What this means for practice

• Read AIES-G profiles at the subfactor level, not as a single total. Prosocial Intention did not form its own factor, so a PI score should not be treated as a clean measure of that construct, and Personal Distress is not a positive outcome even when other scores rise.
• Treat high Perspective Taking or Imaginative Self-Projection scores cautiously. The authors warn these may reflect useful modeling of system behavior or anthropomorphic overinterpretation if not paired with AI literacy, epistemic caution and verification.
• Use the scale to describe a cohort's self-reported orientation, not to evaluate your teaching. The course activities overlapped directly with PT, PI and RI item content, and no untreated comparison group was included, so pre–post gains cannot be attributed to instruction.
• Plan for item revision before confirmation. The authors recommend prioritizing Prosocial Intention and the negatively keyed items, then testing correlated six-factor, three-domain, higher-order and bifactor models in a new sample.
• Do not assume a one-off pre/post shift means a durable change. The paper calls for test–retest intervals without targeted GenAI instruction and for behavioral validation against what users actually do with AI outputs.

Limitations

• The study used one sample of 201 participants for 42 items, about 4.8 participants per item, and the pooled group was unbalanced: Group 4 contributed 110 of 201 participants (54.7%), with cohorts differing in educational level, course purpose and AI-supported activities, so the pooled covariance structure may partly reflect between-cohort differences.
• The same sample supplied the factor structure, internal consistency and responsiveness evidence, no confirmatory sample or test–retest interval was included, and the study was not preregistered; the instrument was developed and administered in Korean, so cross-cultural measurement invariance is untested.
• There was no untreated control group and no random assignment, and activities such as prompt formulation, output evaluation, revision and reflection overlapped with PT, PI and RI item content, leaving task familiarity, response shift, maturation and repeated testing as alternative explanations for the score increases.
• All measures were self-report. Harman's procedure produced a first factor accounting for 25.1% of item variance but cannot rule out common method bias, and the six negatively keyed items need further scrutiny for comprehension difficulty and method variance.

Citation

Kang, S. J., Lee, J., & Kang, S.-J. (2026). AI Empathy as a Human Disposition in Generative AI Interaction: Development and Preliminary Psychometric Evaluation of a Cognitive–Affective–Motivational Scale. Behavioral Sciences, 16(10), 1717.