The Chain Behind the Claim: Warrantability in AI-Assisted Qualitative Research

Synthesis: This essay (Holster, EdArXiv 2026) takes on the credibility problem created when large language models let education researchers reorganize qualitative corpora in minutes, producing fluent topics, quotations, and prevalence claims whose analytic pathway is concealed. Extending the field's evidence debates and the audit-trail tradition into the generative era, it proposes warrantability as a standard that complements accuracy and disclosure: an AI-assisted interpretation is warrantable when the pathway from source data to claim remains inspectable, contestable, and revisable. To make that concrete it introduces semantic lenses — documented reorganizations of a corpus across levels of abstraction — and a claim-relative repertoire of warrant artifacts, from source-linked topic tables to lens stacks and evidence rivers, designed into research tools so that different inferential moves become available for examination, strengthen peer assessment, and widen access to accountable AI-assisted inquiry.

From Boyle's air-pump to the fluent summary

The essay opens with a seventeenth-century analogy: Robert Boyle faced a credibility problem when his air-pump produced phenomena almost no one could witness directly. His response was a "literary technology" of circumstantial reports, engravings, and candid accounts of failed trials, so that distant readers could become virtual witnesses — trust was attached to a documented pathway a reader could inspect and contest, rather than to the product itself. Three and a half centuries later, an education researcher can paste hundreds of open-ended responses into an LLM and receive a polished summarization within minutes; AI can propose codes, identify themes, select quotations, organize categories, and narrate implications with striking fluency. The open question is whether a reader can determine what such AI-assisted analyses warrant.

Why accuracy and disclosure are insufficient

The essay's key move is that existing standards are not enough. A claim can be accurate (the output happens to match the data) yet still conceal which analytical choices produced it — which subset of responses was weighted, how a theme was aggregated, how a prevalence number was derived. Disclosure (declaring "AI was used") names the tool but not the pathway. Warrantability adds a third requirement: the interpretive pathway must remain inspectable (a reader can retrace it), contestable (a reader can challenge a specific inferential move), and revisable (the interpretation can be corrected). LLMs compound the problem because their outputs are biased in ways that resist quick detection, and people tend to rate easily processed, fluent information as more true — so the very fluency that makes AI-assisted analysis seductive is also what conceals its warrant.

Semantic lenses and warrant artifacts

The essay's constructive proposals are representations that make inferential moves examinable:

• Semantic lenses — documented reorganizations of a corpus across levels of abstraction. Rather than a single polished theme list, the analyst records the successive "lenses" through which the data was read and reorganized, so the movement from raw responses to high-level claims is traceable.
• Claim-relative warrant artifacts — the repertoire of records an analyst produces to support a specific claim, including source-linked topic tables (each topic linked back to the quotes and sources that ground it), lens stacks (the layered sequence of transformations), and evidence rivers (continuous, auditable trails of how evidence flowed into conclusions).

These are not audit paperwork for its own sake; they are designed into research tools so that producing fluent AI-assisted analysis also produces, by default, the pathway record that makes it warrantable. In doing so they extend the audit-trail tradition of Lincoln and Guba and adjacent transparency infrastructures in political science, into the generative era.

Benefits for peer review and access

Two benefits follow. For peer review, warrant artifacts give reviewers something to check beyond the final interpretation — they can inspect the semantic lenses and source links to test whether a particular theme or prevalence claim is actually grounded, rather than taking the fluent summary on faith. This extends the field's emerging standards for reporting AI-assisted analysis in evaluable detail. For access and equity, making the pathway available for examination can widen participation in accountable AI-assisted inquiry: if the analytic route is transparent, more researchers (and more critical readers) can engage with and contest AI-mediated qualitative findings, rather than being shut out by the authority of a fluent output. The essay's stance is deliberately methodological rather than adversarial: AI-assisted qualitative research is not inherently illegitimate, but its claims need a visible chain of warrant, and that chain is currently too often missing.

Contribution to the knowledge base

This essay adds a procedural epistemology for AI-assisted qualitative research that complements the wiki's existing treatments of AI-in-research. Where Research Methods in AIED pages and reviews catalog methods, reporting models (e.g. transparency frameworks), and evaluation practice, Holster supplies a concrete standard (warrantability) and concrete artifacts (semantic lenses, lens stacks, evidence rivers) for judging whether an AI-mediated interpretation is trustworthy. It connects to trust (knowing when to trust an AI-assisted analysis), to research methods and reporting conventions, and to broader concerns about integrity and rigor in the generative era. It is an essay/proposal rather than an empirical study, so its value is conceptual: giving the field a shared vocabulary and a design target for transparent AI-assisted qualitative tools.

What this means for practice

• Researchers. Log the full prompt–response sequence and the corpus version behind every LLM pass, because fluency makes a generated theme list look inevitable long before its evidentiary work has begun.
• Pair each prevalence claim with a complete or explicitly sampled unit–topic record, keeping quotations verbatim and linked to stable record identifiers, and leave units that fit multiple topics, fit none, or contradict a topic visible rather than dropping them.
• Record the successive semantic lenses through which the corpus was reorganized so a reader can retrace the path from raw responses to the finished claim, not only inspect the claim.
• Keep warrant artifacts claim-relative: an exploratory map may need only a source-linked topic table and a saved model interaction, while a study touching identity or community representation calls for participant response or explicit counterexample tracking.
• Report AI use as a pathway rather than a disclosure, stating which inferential moves the model made, which the researchers made, and where human judgment entered.

Limitations

• It is an essay/proposal rather than an empirical study: neither the warrantability standard nor the artifact repertoire (source-linked topic tables, lens stacks, evidence rivers) is tested against a corpus, a tool build, or reviewers.
• The authors leave privacy, ownership, consent, and model bias unresolved, noting that sensitive data may require local models, institutional agreements, or synthetic examples.
• The framework gives a model no authority to decide whose experience matters, so low-frequency concerns, culturally specific language, irony, and relational meaning remain vulnerable to erasure.
• Lower technical barriers do not by themselves produce equity: subscription costs, institutional data agreements, and uneven language coverage still stratify use, and the authors concede that a fully documented analysis can still be shallow or wrong.

Connected Concepts

• Qualitative Research — the methodology whose claims AI assists
• Research Methods in AIED — the field's methods and reporting debates
• Large Language Models (LLMs) — the tool class that enables fluent corpus reorganization
• Generative AI — the broader technology
• Trust Calibration — knowing when to trust AI-assisted output
• Peer Assessment — strengthened by warrant artifacts
• Academic Integrity — rigor in AI-assisted scholarship
• Ethics — accountable AI-assisted inquiry
• Learning Analytics — the data-rich context where such corpora arise
• AI Detection — transparency in the use of AI

Connected Articles

• Evaluation in the Age of AI: Output as Evidence of Learning — process-based evaluation in the age of AI
• Can AI Evaluate Assessment? A Study of Large Language Model Meta-Assessment Performance — evaluating LLM-based assessment
• Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content — detecting AI-generated content
• AI-Mediated Learning and the Restructuring of Interpretive Cognition: A Developmental-Critical Model for Social Sciences and Humanities Education — AI interpretive cognition in social sciences & humanities
• Presumed Effective: The Manufacturing of an Evidence Base for AI-in-Education Through Flawed Meta-Analysis — audit of meta-analytic claims in AIED

Citation

Holster, J. D. (2026). The chain behind the claim: Warrantability in AI-assisted qualitative research. EdArXiv preprint.