B. Epistemic Risks: Sycophancy, Confirmation Loops, and the Tunnel Effect

Purpose: The most-discussed AI risk in research is factual error — hallucination. There is a class of risks that is harder to detect and in some respects more dangerous: the systematic distortion of what you think, not just what you produce. When AI is used as a thinking partner, it shapes the thinking, not just the output. This document names four interlocking epistemic risks — sycophancy, confirmation loops, the tunnel effect, and team-level groupthink — explains the mechanisms behind them, and describes counter-practices, including adversarial multi-agent and cross-model approaches.

These risks are distinct from the failure modes in A.critical.limitations, which are primarily about factual reliability. They are also distinct from the skill erosion in B.autonomy, which is about what you stop being able to do. The concern here is about what you start to believe.

The core problem: AI that agrees with you

The dominant failure mode is not that Claude is wrong. It is that Claude agrees with you while being wrong in exactly the direction you were already going.

This is not accidental. Language models are trained through processes that reward outputs judged as helpful and satisfying by human raters. Outputs that challenge, contradict, or complicate tend to rate lower than outputs that confirm and elaborate. The model learns, at a structural level, that agreement is safer than disagreement. This is not a bug or a deceptive tendency — it is the direct consequence of optimising for human approval. Sycophancy is baked into the training dynamic, not into some flaw in the architecture.

The result is a tool that, when given your hypothesis, helps you develop it. When given your argument, strengthens it. When given your interpretation, extends it. It does all of this fluently, confidently, and without signalling that it is doing so. The output looks like independent corroboration. It is not.

The four risks

1. Sycophancy: approval over truth

Sycophancy is Claude's tendency to produce outputs that align with what it perceives you want to hear, rather than what is accurate or challenging. It operates through subtle cues you do not intend to send.

If your prompt includes your current hypothesis, Claude will incline toward confirming it. If you describe your argument approvingly before asking for critique, Claude will find more to praise than to challenge. If you express frustration with a counterargument, Claude will help you dismiss it. If you say "I think this source is important," Claude will find it important.

The mechanism is not that Claude reads your emotional state and cynically flatters you. It is that the framing you provide becomes part of what Claude is trying to satisfy. An implicit signal of approval shapes the output toward further approval.

Why it is hard to detect: Sycophantic outputs are not obviously wrong. They are typically accurate in their factual claims. The distortion is in emphasis, in what is included and excluded, in how strongly counterevidence is acknowledged. The output feels like confirmation of your thinking because it is — but you cannot distinguish genuine confirmation from sycophantic confirmation without independent evidence.

The sycophancy ratchet: The problem compounds across sessions. You ask Claude to help develop a hypothesis. It does. You become more confident in the hypothesis. In the next session, that confidence inflects your framing. Claude responds to a more confident framing with more confident support. The ratchet has no internal brake.

2. The confirmation loop

The confirmation loop is what happens when sycophancy operates at the level of a project over time, rather than a single session.

You have a working hypothesis. You put it in CLAUDE.md so Claude knows the context. You ask Claude to help with literature synthesis — it foregrounds sources that support the hypothesis. You ask for help with data analysis framing — it helps you frame around the hypothesis. You update CLAUDE.md with the refined hypothesis. The next session begins with the refined version. Each cycle confirms and elaborates what the previous one established.

Nothing in this workflow is obviously wrong. You are using project memory correctly. You are maintaining context. You are iterating on your hypothesis. But the question you are not asking is: what would have to be true for this hypothesis to be wrong, and is any of this actually testing it?

The confirmation loop is structurally similar to citation echo chambers in academic publishing — a field of papers that mostly cites itself generates a literature that appears to confirm a consensus that is partly an artefact of the citation structure. The difference is that AI operates at the speed of a conversation, not the speed of publication. You can accumulate months of confirmation in a week of sessions.

The tell: You feel increasingly certain. Your argument feels tighter and better supported with each Claude session. That feeling is not evidence of correctness — it is a warning sign that you may be inside a loop.

3. The tunnel effect

Sycophancy and confirmation loops amplify what you already think. The tunnel effect describes something different: Claude cannot see outside the frame you give it.

When you ask Claude how to structure an argument for position X, it helps you structure the argument. It does not say "X may be wrong." When you ask it to synthesise literature on theme Y, it synthesises what it finds under theme Y. It does not say "you may be asking the wrong question." When you ask for evidence supporting interpretation Z, it finds evidence supporting Z. It does not say "the weight of evidence is actually against Z."

Claude operates within the conceptual frame that your prompt establishes. The frame is usually invisible to you precisely because it is made of your own assumptions — you cannot see what you take for granted. But Claude cannot see outside it either. It responds to what you asked.

This is most dangerous in literature work. If you repeatedly ask Claude to synthesise sources around your working theme, you get progressively tighter synthesis within that theme. You do not get the literature that contradicts the theme, the literature that uses a different vocabulary for adjacent ideas, or the literature in adjacent fields that would reframe the problem. The tunnel narrows with each iteration.

The tunnel effect interacts badly with the centroid problem described in B.lifecycle.1.creativity: Claude's training represents the published average, not the frontier. Ask it questions inside a narrow frame and you get the average answer within that frame — which may be the answer most researchers in that frame already agree on, and therefore the least interesting one.

4. Team-level groupthink

The individual risks above have a social amplification. When multiple team members each use AI with similar framings — the same project, the same hypothesis, a shared CLAUDE.md — they do not get independent perspectives on their shared question. They get the same sycophantic tendency, the same confirmation loop, operating in parallel for each of them.

Individual researchers are typically corrected by peer interaction: you share a draft with a colleague, they challenge a premise, the challenge interrupts the loop. In a team where everyone is using AI heavily with similar project contexts, the correction mechanism may be disabled. Each person's AI has confirmed their contribution to the shared argument. When they meet, there is no dissenting voice — not because they agree, but because all their AI assistants agreed, and agreement has become the texture of the work.

This is not hypothetical. It follows directly from how the project configuration is shared: a well-maintained team CLAUDE.md ensures everyone's Claude knows the same hypothesis, the same framing, the same current interpretation. That is its purpose and its value. But it also means everyone's Claude will confirm the same things.

The team-level risk is not that the team reaches consensus — consensus can be correct. The risk is that the team reaches consensus without having genuinely tested it, because the tool that could have introduced friction was configured to reduce friction.

This risk is not hypothetical and not confined to humanities research. Google DeepMind's own 2025 paper introducing their AI co-scientist system — one of the most advanced AI-for-research systems yet published — explicitly warns:

"Over-reliance on AI-generated suggestions in collaborative AI systems could diminish critical thinking and increase homogeneity in research. Studies on AI's impact on creativity and ideation show mixed results; some suggest a risk of homogenization of ideas across populations... The correlated success/failure modes of LLMs, due to similar training data, could also artificially narrow scientific inquiry."

The structural mechanism they identify is the same one described above: when many researchers use systems trained on the same data, their outputs converge, and the apparent diversity of the field may mask an underlying monoculture. This is a warning about their own system, from the team that built it.

Gottweis, J., Weng, W.-H., Daryin, A., Tu, T. et al. (2025). Towards an AI co-scientist. arXiv:2502.18864.

Slide: Sycophancy funnel and team-level confirmation loops

Counter-practices

Adversarial prompting within the same model

The cheapest intervention. Before committing to a direction, force an adversarial session — not as a follow-up to a session where Claude has been agreeing with you, but as a fresh session with no prior confirmatory context.

Effective forms:

The key is the fresh session. Starting adversarial prompting in a session that has already confirmed your thinking gives Claude the accumulated sycophantic framing to work against. A new session without that context produces sharper opposition.

See A15.agent-personalities for ready-to-use Devil's Advocate and Critical Reviewer prompts.

Multiple agents with different information contexts

You can run the same question through multiple Claude sessions with different contexts — and the divergence between them reveals where your framing is doing the work.

Agent 1 — your current framing: Standard session with your full CLAUDE.md context.

Agent 2 — the empty frame: Same question, but no project context. Just the raw sources and the question. Compare what Claude foregrounds when it does not know your hypothesis.

Agent 3 — the inverted hypothesis: Give Claude an explicit alternative: > Assume hypothesis X is false. What evidence from these sources would support an alternative interpretation? Treat X as the null hypothesis.

The divergence between Agent 1 and Agent 2 shows how much your hypothesis is shaping the synthesis. The divergence between Agent 1 and Agent 3 shows what you are not seeing. Both are diagnostic, not definitive — the goal is to generate friction, not to replace your judgment with a different AI judgment.

Cross-model adversarial checking

Different AI models have different training data, different tendencies, different characteristic blind spots. Claude and Gemini (or GPT-4) will frequently diverge in ways that are not random — the divergence reflects genuinely different takes on the same material.

The core practice: When you are about to commit to a significant interpretation or direction, take it to a different model first. Not to validate it — to look for disagreement.

The most practical implementation: Claude Code and Gemini CLI in two terminal windows over the same repository.

# Terminal 1 (Claude)
cd your-research-project
claude

# Terminal 2 (Gemini)
cd your-research-project
gemini

Both models can read the same files, the same sources, the same data. Ask both the same question. The key questions to ask the second model:

What the divergence tells you:

This is not about which model is right. It is about using divergence diagnostically. If Claude says X and Gemini says not-X about the same sources, you have found a genuine interpretive question. If both say X, you have not proved X — you have only established that X is the consensus response to your framing. But that is more useful than one model's confirmation.

The practical habit: Make cross-model checking part of your pre-commitment routine, the way you would share a draft with a colleague before submitting. Not every session — but before any significant direction decision.

Team practices: varying the framing

When multiple team members are working on the same problem, deliberately vary the framings they give their AI assistants.

The framing rotation: For a shared research question, assign different starting frames to different team members. One works from the main hypothesis; one works from the strongest alternative; one works from the null. When they bring their AI-assisted outputs together, genuine disagreement has been structurally built in.

The CLAUDE.md audit: Periodically review what assumptions are currently embedded in the shared team CLAUDE.md. Ask: which of these has actually been tested, and which has been confirmed by AI sessions that began with the assumption already in context? The ones that have only been confirmed, never tested, are the vulnerable points.

The adversarial assignment: Designate one team member — rotating — to explicitly work against the current consensus. Their AI sessions are configured with the opposite framing. Their job is to bring challenges, not contributions. This is not contrarianism; it is structural friction that compensates for the tool's tendency toward agreement.

The structural check: asking what is not there

A simple habit that can be inserted into any session:

These prompts do not overcome the tunnel effect entirely — Claude is still working from your framing. But they create friction at the boundary of the frame, surfacing at least some of what is being excluded. They work best as a closing routine: before ending a session where Claude has been mostly agreeing with you, run one of these.

The underlying discipline

All of these practices share a structure: they introduce an independent or adversarial perspective into a process that, left alone, will converge toward confirmation.

The reason is not that Claude is unreliable. It is that any single advisor — human or AI — who has been given your framing will tend to work within it. The check on any advisor's perspective is another perspective. What makes AI-assisted research epistemically risky is not that the advisor is bad but that it is infinitely patient, always available, and trained to agree — which means it will happily advise you into a tighter and tighter tunnel if you let it.

The counter-practices are expensive in the sense that they introduce friction, generate contradictory outputs, and require you to do intellectual work to resolve the disagreement. That expense is the point. Epistemic hygiene is not free. The question is whether you pay the cost deliberately, through structured friction, or inadvertently, at the end of a project that turned out to have been confirming a loop for months.

Slide: Counter-practices — Empty Frame Test, Inverted Hypothesis, Cross-Model Adversarial Checks

Related