B. Data Capture: Collection, Processing, and the Micro-Discovery Loop

Purpose: Data capture is the phase where research meets its material — archives, interviews, corpora, structured records. It is largely human work, domain-specific in ways AI cannot replicate. But AI transforms what happens around collection: preparation before, processing after, and — most significantly — the feedback loop between incoming material and the direction of further collection. This document maps those roles and introduces the micro-discovery pattern that makes AI-assisted data capture different in kind from simply faster.

Part of the B.lifecycle series.

What data capture covers

Data capture is broader than "fieldwork." Depending on the research, it includes:

Across all of these, the same basic structure holds: there is collection itself (largely human, largely irreplaceable), and there is processing of what was collected (engineering-like, increasingly AI-assisted).

AI in preparation: before collection begins

Before you go to the archive, launch the interview series, or start building the corpus, AI can help with the design work:

These are relatively engineering-like tasks with clear outputs. Getting them right before collection starts reduces rework later. The judgment about whether the design is actually appropriate for your research question — that remains yours.

AI in post-collection processing: the engineering-like end

After material is collected, a substantial range of processing tasks sit firmly on the engineering-like end of the gradient and are highly productive uses of AI:

Transcription and OCR correction. Converting audio recordings to text, correcting OCR errors in scanned documents, cleaning up machine-transcribed text. These are mechanical tasks where AI assistance is mature and reliable — though outputs require checking, particularly for specialist terminology, proper names, and historical language.

Named entity extraction. Identifying and tagging persons, places, organisations, dates, and events across large document sets. Powerful for building structured representations of unstructured historical or qualitative material.

First-pass categorisation. Applying a defined coding scheme across a corpus — a first pass that you then verify, correct, and use as a starting point rather than a finished product. See B.lifecycle.4.dataanalysis for the specific cautions around qualitative coding.

Format conversion and structuring. Converting transcripts, notes, or extracted text into structured formats (tables, JSON, Markdown) for further processing or import into analysis tools.

Batch processing at scale. For corpora that no individual researcher could process manually — hundreds of archival scans, thousands of interview pages — AI batch processing is not just faster but makes otherwise impossible research possible. This is the autonomy-gain case: AI expands what a researcher can do independently.

The micro-discovery loop: collection and processing interleaved

The traditional model treats data capture as a sequential phase: collect everything, then process, then analyse. AI disrupts this sequence in a productive way.

When processing happens close to collection — the same day, the same week — the processed material becomes active input to subsequent collection decisions. This is the micro-discovery loop:

Process today's archival finds tonight → notice a name appearing repeatedly that you did not know was relevant → search the index for that name tomorrow → find a new thread.

Transcribe and do first-pass coding of Monday's interviews → realise a category is splitting into two distinct phenomena → revise the interview guide for Tuesday's sessions.

Run entity extraction on the first batch of sources → a geographic concentration emerges → redirect the corpus-building toward that region.

In each case, AI-assisted processing is not just faster — it is enabling a tighter feedback loop between what you are finding and what you look for next. Research becomes more responsive to its own discoveries. This is closer to how experienced researchers actually work — always half-interpreting while collecting — but now with AI handling the processing that makes that responsiveness possible at scale.

This is one of the strongest arguments for AI in research that goes beyond efficiency: it enables more agile research, not just more efficient research.

What stays human

The engineering-like processing tasks can be delegated. The judgment tasks cannot:

Corpus selection. Deciding what counts as your dataset — which sources are included, on what criteria, with what justification — is an intellectual and methodological decision that defines the research. AI can help you articulate the selection criteria and think through their implications; it cannot make the selection for you.

The archival encounter. What you notice in the archive, what you choose to photograph, what you recognise as significant given your specific question and your knowledge of the field — this is the exercise of disciplinary expertise that no processing tool replicates. The archival find that changes the project is almost never the result of a systematic search; it is the result of a trained eye noticing something unexpected.

The interview itself. The conversation, the follow-up question, the decision to pursue a thread or let it drop, the interpretation of hesitation or emphasis — these are human relational and interpretive acts. AI can help you prepare for interviews and process the recordings afterward; the interview is yours.

The "is this actually data?" judgment. Not everything collected is data. Deciding what constitutes evidence for your specific claim, what is noise, what is context — this is the interpretive work that gives data collection its meaning.

Data capture feeding back to earlier phases

Data collection is where the research plan meets reality, and reality rarely matches the plan exactly.

Sources you expected to exist do not. Sources you did not expect reveal an entirely different picture. Interview subjects say things that make your working hypothesis look naive. The corpus that seemed sufficient turns out to have a systematic gap. These are not failures — they are how research works. They are also the moments that send you back:

AI-assisted processing makes these feedback signals legible faster. The sooner you see the gap or the contradiction, the sooner you can redirect — rather than discovering it after months of collection in the wrong direction.

Related