AIEd Knowledge BaseAIEd KB
On this page

Concept

Qualitative Research

created 2026-08-24 · updated 2026-10-03

Qualitative research — the family of empirical methods that study how people experience, interpret, and make meaning of phenomena, typically through words, observations, and artifacts rather than numbers. In AI in education, qualitative methods reveal how students and teachers actually experience AI tools — the meanings, tensions, harms, and mechanisms that standardized measures miss. Because AI-in-education is fast-moving and its effects are often mediated by context, perception, and contested constructs like Trust and Learner Agency, qualitative work is essential alongside quantitative designs (see Research Methods in AIED).

Questions to Consider

  • A survey tells you 40% of students distrust an AI tutor; a focus group tells you why they distrust it. Which number feels more actionable to you, and what does the 'why' add that the percentage cannot?
  • Qualitative findings are usually not statistically generalizable, yet they're often 'conceptually generalizable' — mechanisms and dynamics that transfer elsewhere. Before you read, what does it mean for a finding to be generalizable in concept but not in statistics?
  • The page cautions that human–LLM coding agreement is not the same as coding quality when human consensus isn't ground truth. Have you ever treated 'two reviewers agreed' as proof something was correct? When is agreement a signal of truth, and when is it just shared error?
  • If AI can now assist qualitative coding at scale, does that threaten the interpretive depth that makes qualitative research valuable, or merely automate its drudgery? How would you decide which is happening in a given study?
  • Qualitative work centers under-represented voices — ethnic-minority students, skeptical nonusers — that large surveys often miss. Think of an AI-in-education claim you've heard. Whose experience of it is probably not captured by the headline number?
  • Choose one contested construct you care about — trust, agency, or harm. Before reading, sketch how you'd study it with words and observations rather than numbers, and note what you'd lose by doing so.

Introduction

Qualitative research is not a single method but a family organized by what they study and how evidence is gathered and analyzed. What unites them is an emphasis on meaning-making, context, and depth over breadth and causal control. Qualitative findings are typically not generalizable in the statistical sense, but they are often conceptually generalizable — revealing mechanisms, categories, and dynamics that transfer to other settings. In the knowledge base's corpus, qualitative work is prominent for studying AI acceptance, trust, harm, teaching practice, and learning processes.

Major qualitative approaches

Thematic analysis

Thematic analysis identifies, codes, and interprets patterns ("themes") across qualitative data — typically interview or focus-group transcripts, open-ended survey responses, or documents. It is the most widely used approach in the knowledge base's qualitative studies. Interviews and open-ended responses are themselves self-report data, so they share the limits on what can be claimed about behavior — see that page for where self-report evidence is strong and where it breaks down. A study of the trust–utility gap in physics uses thematic analysis of student interview data to surface domain-specific skepticism and adoption preferences; a comparison of GenAI vs. teacher feedback analyzes student perceptions of usefulness and trustworthiness; and guidelines for adult AI learning derive design principles from thematic coding of expert and learner input. How AI changes teaching workflows relies on thematic analysis of educator accounts.

Coding reliability can be made an ongoing procedure rather than a one-time check. Galiç et al. (2026) code 304 noticing statements at Krippendorff's α = .803, monitor agreement on five overlapping cases (pooled κ = 0.937), and re-code disputed statements whenever agreement fell below a 0.85 recalibration threshold before resuming; the coded statements then enter an epistemic network analysis that models which dimensions co-occur rather than a flat list of themes.

Grounded theory

Grounded theory builds a theory from the data rather than testing an a priori framework, using iterative coding (open → axial → selective) until theoretical saturation. It is ideal for constructing new theory about emergent AI-in-education phenomena. Liu et al. develop a grounded theory of tool, tutor, or crutch — a three-mode typology of how students cognitively scaffold or offload onto AI in programming education — directly theorizing Cognitive Offloading. A grounded-theory study of critical AI tutors examines whether such tutors empower or enslave learners. Human-centered GenAI feedback design uses grounded analysis across a multisite study. See also Theory Development in AI in Education for how such grounded theories feed the field's theory building.

Phenomenology and phenomenography

Phenomenological approaches study the lived experience of a phenomenon — what it is like to learn with AI — while phenomenography studies the qualitatively different ways people experience and understand a phenomenon (producing "categories of description"). The Absent Cognitive Baseline draws on students' lived experiences of self-assessment under AI to theorize a structural gap; a phenomenological study captures the experience of completing work with AI while knowingly not understanding it; and a socio-cultural, interpretive study analyzes how GenAI becomes a "runaway object" in mathematics academic practice.

Discourse analysis

Discourse analysis examines how language-in-use constructs meaning, identities, and power — analyzing classroom talk, written text, or interactional sequences. NSPA conducts long-horizon classroom discourse analysis (here computationally assisted) to mitigate dialect bias in understanding classroom interaction; a study of ethnic-minority preparatory students analyzes collaborative discourse in Prompt Engineering tasks. Discourse analysis bridges qualitative interpretation with computational methods when combined with Educational NLP.

Poudyal (2026) shows how discourse coding becomes auditable when it is rendered as countable rules: across 366 abstracts and 91,405 tokens an association counts only when an actor precedes a predicate within six intervening words, and nominal proximity is excluded so that "AI governance" never counts as evidence that AI governs — a rule-based approximation, not a dependency parse, whose limits the design states.

Observations and ethnography

Observation studies watch behavior in context; ethnography extends this to sustained, immersive study of a setting, often with the researcher as participant-observer. A trio-ethnography of LLM-supported programming education traces how students' interpretations evolve; a classroom case study observes AI Literacy integration in biology. Observational methods capture actual behavior (what learners do with AI) rather than reported behavior — complementing the self-report surveys that dominate quantitative measurement of attitudes.

Case studies

A case study is an in-depth investigation of a bounded case (a course, an institution, a single learner) using multiple data sources. A business-school case study generates a student-informed teaching and learning framework for GenAI; a biology case study documents AI-literacy integration. Case studies trade breadth for depth and are strong for theory generation and transferable insight rather than generalization. Khlaif et al. (2026) offer a qualitative case study of 21 visually impaired undergraduates across three Palestinian universities, using thematic analysis of semi-structured interviews to show how GenAI functions as an assistive technology for inclusive learning — an example of case-study research surfacing mechanisms (personalized adaptation, teacher augmentation, educational parity) that quantitative measures miss.

Swidan (2026) triangulates video, the alert system's own logs, and stimulus-recall interviews for one teacher and eight students, showing that an affective signal acquired meaning only through the teacher's response; the two-stage episode coding by a single author, without independent coding or validated affect detection, marks the limit of what a one-coder case analysis can warrant.

Interviews and focus groups

Semi-structured interviews and focus groups are the primary data-collection instruments across all the above approaches. They elicit rich, contextual accounts. The knowledge base's qualitative corpus is built substantially on interviews (e.g., expertise pathways) and focus groups (e.g., the text-to-image competence paradox, Guidelines for Designing AI Technologies to Support Adult Learning). Quality depends on careful question design, sampling for variation, and rigorous analysis.

How qualitative research appears in the knowledge base

AI and qualitative analysis

A distinctive recent development is using LLMs to assist qualitative coding. The knowledge base's evidence is cautionary: Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis shows LLM and human coding diverge, with errors cascading through temporal analysis; Agreement Is Not Quality shows that human–LLM coding agreement is not the same as coding quality when human consensus is not ground truth. LLM-assisted coding can scale and accelerate qualitative analysis, but its outputs require verification against human judgment — an important intersection of qualitative research with Educational NLP and AI Ed Evaluation.

The fluent-output problem and warrantability. Holster (2026) sharpens why accuracy and disclosure are insufficient standards for AI-assisted qualitative work. Because LLMs reorganize corpora in minutes into fluent topics, quotations, and prevalence claims, they can conceal the analytic pathway that produced them — and people tend to rate easily processed, fluent output as more true. Holster proposes warrantability as a complement to accuracy and disclosure: an AI-assisted interpretation is warrantable when the pathway from source data to claim remains inspectable, contestable, and revisable. Its constructive machinery is semantic lenses (documented reorganizations of a corpus across levels of abstraction) and a claim-relative repertoire of warrant artifacts — source-linked topic tables, lens stacks, and evidence rivers — designed into tools so that fluent analysis also produces a retraceable pathway record. This extends the field's audit-trail tradition into the generative era and gives reviewers something concrete to contest beyond a final interpretation.

  • Reliability is not accuracy in multi-agent LLM coding. A literature-informed pipeline in which two AI coders coded independently, debated and reconciled disagreements produced inter-coder agreement above Cohen's kappa 0.85 on every dataset and label while criterion F1 ranged from 0.31 to 0.89 (mean 0.68, SD 0.16) — high agreement between agents that were both wrong. Longer codebooks (t = -11.702) and more similar excerpts (t = -9.249) reduced initial accuracy, and discussion turns correlated positively with accuracy (t = 7.997) while correctly resolved conflicts and collaborating modes correlated negatively (t = -9.720 and -8.420), so convergence is a weak proxy for correctness. The practical implication is that AI-assisted coding needs checklist-level adjudication against a human reference rather than agent-to-agent agreement as its quality signal. (How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding)

Strengths and limitations

  • Strengths: deep ecological and conceptual insight; surfaces unexpected phenomena, risks, and mechanisms; essential for theory-building (see Theory Development in AI in Education); captures meaning, context, and contested constructs; centers under-represented perspectives; strong for studying fast-moving phenomena where standardized measures lag.
  • Limitations: limited statistical generalizability; interpretive and researcher-dependent (reliability concerns); small samples; weaker support for causal claims; findings can be hard to synthesize across studies; time- and labor-intensive.

Qualitative and quantitative methods are complements, not rivals — see Research Methods in AIED for how they contrast and triangulate, and mixed methods for designs that combine them.

Connected Concepts

Connected Articles

Metadata

Study designInterviews, Case Study
Research methods and evaluationQualitative Research, Research Methods in AIED
SourceHistoryEditChat

Embed this page

Copy the code below to embed this page in a learning management system or other website. The embedded version shows only the page content — it hides the site header, navigation, and footer.