Calibration: what AI is actually good at for researchers

Who this is for: Researchers approaching AI sceptically, or those who have tried it and found the results disappointing. This document gives an honest account of where AI performs well, where it underperforms, and how to tell the difference in your own work.

The organising frame: Tom Pepinsky's distinction between execution and interpretation is the most useful lens. AI is excellent at execution — following rules, applying patterns, transforming structured inputs into structured outputs. It is unreliable at interpretation — generating original arguments, producing factual claims without grounding, or exercising the evaluative judgment that research demands.

This is not a failure of the technology. It is the boundary of what it can currently do. The mistake is to confuse fluency with reliability.

Where AI demonstrably helps

These are areas where performance is consistent and documented — not hype.

Pattern recognition across large text corpora. LLMs are trained to find patterns. Give Claude 20 papers or 40 archive documents and ask what themes recur, what tensions exist, which sources cluster around similar positions — this is genuine capability. It will miss nuances a specialist would catch, but it will find patterns a human reader might miss through fatigue or selective attention across a large set.

First-draft generation. AI produces text fast. The blank-page problem is solved. The first draft of a section, a summary, an email, a grant paragraph — the result is not publication-ready, but it is material to revise. For researchers who find writing aversive, this is a real gain in time and momentum.

Structured extraction from text. Identify all named persons, extract all dates and locations, pull all quotations mentioning a specific term — AI does this reliably on clearly defined tasks. Errors occur but are catchable through spot-checking. The consistency across a large corpus is something manual work cannot sustain.

Consistency across volume. A human coder applies rules differently on document 40 than on document 1 — fatigue, drift, context-dependence. AI applies the same rules throughout. This consistency has value even when absolute accuracy is imperfect: a systematic error is recoverable; a drifting one is not.

Format and structure transformation. Convert PDFs to markdown, reformat a bibliography, restructure a document from one outline to another, generate a table from running text — these transformations are reliable and save significant time.

Multilingual access. Working with Latin sources, German historiography, or field-adjacent literature in languages you read slowly — AI provides access that would otherwise require specialist assistance or significantly more time.

Dialogue and stress-testing. Claude as a thinking partner — not as an expert, but as an interlocutor. Ask it to steelman a position you disagree with, play hostile reviewer, find the weakest link in your argument. The quality of the pushback is limited, but it surfaces things you may not have caught alone.

Where performance drops

Factual reliability without grounding. When asked to recall specific facts, statistics, or citations from general knowledge, Claude halluccinates. The probability is not trivially low: hallucination rates run at 6% on general tasks and rise to 48% on specialised academic queries. Citation accuracy is around 50–60%. More troublingly, AI uses more confident language when it is wrong — 34% more likely to assert with high confidence when hallucinating. Never trust a factual claim you have not independently verified.

→ A.critical.limitations has the full statistics.

Qualitative interpretation. Ask Claude to interpret an ambiguous historical source, assess the significance of a finding, or exercise the kind of evaluative judgment that makes a researcher's argument original — and you get something that looks like interpretation but is not. It reproduces dominant framings. It suppresses minority positions and unconventional readings. It is, by construction, optimised for plausibility, not originality.

Original argument generation. Claude can help develop and articulate an argument you have already formed. It cannot generate a genuinely novel research argument from scratch. What it produces will feel persuasive and coherent while lacking the original insight that makes research valuable.

Literature search and discovery. Claude's training data has a cutoff and does not include all published literature. Do not use it to discover what exists on a topic — use it to work with sources you have already assembled. For discovery, use Gemini Deep Research, Perplexity, or Elicit.

Self-knowledge about errors. Claude does not know when it is hallucinating. It does not flag uncertainty reliably. You cannot ask "are you sure?" and receive a meaningful answer. Verification must be external, systematic, and against primary sources.

The honest trade-offs

Speed vs. depth. Claude is fast. What it produces at speed is seldom deep. The value proposition is not "better output" — it is "faster material to work from." The depth comes from what you do with it.

The most striking illustration of this comes from Google DeepMind's 2025 AI co-scientist paper. Their system recapitulated a significant biological discovery — a novel gene transfer mechanism in bacterial evolution — in two days. The headlines wrote themselves. But the paper adds a caveat that is easy to miss: "While the co-scientist generated this hypothesis in just two days, it was building on decades of research and had access to all prior open access literature on this topic." The two days is real. The accumulated knowledge underneath it is ten years. The acceleration is in synthesis speed, not in the discovery of what no one had yet thought. That distinction matters when evaluating capability claims about AI in research.

Gottweis, J., Weng, W.-H., Daryin, A., Tu, T. et al. (2025). Towards an AI co-scientist. arXiv:2502.18864.

Breadth vs. reliability. Claude engages with a wide range of tasks. Its reliability varies significantly across them. The mistake is to assume that because it performs well on task A (structured extraction), it will perform equally well on task B (qualitative interpretation).

Fluency vs. accuracy. AI output is always fluent. Fluency does not signal accuracy. A confident, well-written paragraph may contain a hallucinated citation, a misread source, or a subtly wrong claim. Fluency is the most seductive source of false confidence in AI-assisted work.

Delegation vs. skill atrophy. Every skill you delegate is a skill you practice less. This is not unique to AI, but the scale and ease of delegation makes it worth naming explicitly. Researchers who outsource first-draft writing to Claude, and do it for long enough, report that writing feels harder when they return to it. → C.dangers for the full treatment.

Signs you are using it well

Signs you may be fooling yourself

The bottom line

AI is a capable assistant for well-defined execution tasks and a genuinely useful thinking partner when used critically. It is not a researcher and should not be treated as one. The researchers who get the most from it are those who treat it as a fast, fluent, non-expert collaborator whose output they systematically verify and regularly override.

The benchmark is not "does it do this better than I can?" It is "does this get me somewhere useful faster, with effort I am willing to invest in checking?"

Related