Token Efficiency: Getting More Done Without Hitting Limits

Who this is for: Researchers who have run into Claude slowing down mid-task, hitting usage limits, or getting "context too long" errors — and want to understand why and what to do about it.

Two different limits, two different problems

"Token limits" actually means two separate things, which have different causes and different solutions:

1. Context window limit — how much Claude can hold in one session at once (~200,000 tokens). When you fill this up, Claude starts losing track of earlier parts of the conversation, or refuses to continue.

2. Usage/rate limit — how many tokens you can use per hour or day on your plan. When you hit this, Claude slows down or asks you to wait. This is about your subscription quota, not the session length.

Most day-to-day friction comes from the usage limit, not the context window. The context window is large enough for most research tasks. But processing many large PDFs in a row, or running many sessions in a day, can exhaust your hourly quota.

Understanding your usage

Check current session usage:

/cost

Shows tokens used and approximate cost for the current session.

Signs you are hitting the rate limit:

Signs you are hitting the context window:

Strategy 1: Start fresh for each distinct task

Every /clear resets the context and starts a new session. This is the single most effective habit for staying within limits.

Bad habit: One long session for everything — reading 5 papers, then editing a draft, then organising files, all in one continuous conversation.

Good habit: One session per task. Read papers → /clear. Edit draft → /clear. The context for each task is small and focused; nothing from previous tasks accumulates.

# Session 1: paper reading
> Read papers in /reading/ and write notes to reading_notes.md
[done]
/clear

# Session 2: draft editing
> Read draft.docx and give me feedback on section 2
[done]
/clear

The output of each session is saved to a file. The next session starts fresh but can immediately read those files if needed.

Strategy 2: Use /compact for long sessions you cannot break up

When you need to continue a long session — a complex task that must run continuously — /compact asks Claude to summarise the conversation so far and replace the full history with the summary. This frees up significant context space.

/compact

Claude writes a summary of what has been done and decided, replaces the conversation history with it, and continues. You lose the fine detail of earlier exchanges but keep the essential thread.

Use this when:

Strategy 3: Convert documents to markdown before processing

A PDF with layout, images, and metadata may be several times larger in tokens than the same content as clean markdown. Converting first reduces token cost for the same information.

> Convert paper.pdf to paper.md using markitdown, then read paper.md and summarise it.

vs.

> Read paper.pdf and summarise it.

For a single paper the difference is small. For a batch of 20 papers it compounds. See A.markdown-central for conversion tools and the /convert-folder skill.

Strategy 4: Be specific about what to load

Claude reads what you tell it to read. If you say "read everything in this folder", it loads everything. If your folder has 30 files and you only need 5, that is 25 files of unnecessary context.

Vague (loads everything):

> Read my notes and summarise the state of the project.

Specific (loads only what is needed):

> Read CLAUDE.md and progress.md and summarise the state of the project.

Similarly, for long documents, ask for targeted reads:

> Read pages 40–60 of the_register.pdf — only that section.

Rather than:

> Read the_register.pdf [200 pages]

Strategy 5: Write output to files, not the terminal

When Claude writes a long output to the terminal — a detailed summary, a full extraction, a revised draft — that output also occupies context. In a long session with many outputs, this accumulates.

Always specify an output file for substantial results:

> Read paper.pdf and write the summary to paper_notes.md

The summary is saved to disk. If you need to refer to it later in the session, Claude reads the file — but only when you ask, not continuously.

Strategy 6: Process in chunks for large corpora

For large batch jobs — 40 PDFs, a long register — split across multiple sessions rather than one marathon session.

# Session 1
> Process PDFs 1–15 in /sources/. Extract named persons to extractions.md.
/clear

# Session 2
> Continue extracting named persons from PDFs 16–30 in /sources/.
  Append results to extractions.md (do not overwrite).
/clear

# Session 3: consolidate
> Read extractions.md. Remove duplicates, sort alphabetically, save back.

This keeps each session within comfortable limits and produces the same final result. The append instruction is important — it adds to the existing file rather than starting fresh.

Strategy 7: Put standing context in CLAUDE.md, not in conversation

Pasting your project background, conventions, and folder structure at the start of each conversation wastes tokens every session. That text sits in the context the entire time, consuming space.

Inefficient:

> Here is my project background: [500 words pasted]
  Here are my extraction conventions: [300 words pasted]
  Now read source.pdf and extract persons...

Efficient:

[CLAUDE.md contains project background and conventions — loaded automatically]

> Read source.pdf and extract persons following the conventions in CLAUDE.md.

The content is the same. The second approach references CLAUDE.md rather than pasting it — Claude reads it from disk when needed rather than carrying it in conversation history throughout.

Strategy 8: Use targeted Zotero searches

If Zotero MCP is connected, asking Claude to search your entire library for a vague topic loads many results into context. Be specific:

Loads more:

> Search my Zotero library for anything relevant to heresy.

Loads less:

> Search my Zotero library for sources specifically about Cathar perfecti
  published between 1990 and 2010. Return the top 5.

The second query returns fewer items and uses fewer tokens for the same quality of result.

When you hit the rate limit: what to do

If Claude slows significantly or tells you to wait:

  1. Finish the current task cleanly — write any pending output to files before the session degrades further
  2. Wait — rate limits on Pro typically reset within an hour
  3. Use the pause productively — review what Claude produced, plan the next session
  4. Consider the Max plan if you regularly hit limits on intensive days — it has significantly higher quotas

You cannot "bank" unused tokens from quiet days. The quota resets on a rolling basis regardless of use.

Quick reference: which strategy for which problem

Problem Strategy
Running out of context mid-session /compact, or split into multiple sessions
Rate limit hit during batch work Split corpus across sessions; convert to markdown first
Slow responses, seems distracted /clear and start fresh; conversation may be too long
Same context pasted every session Move it to CLAUDE.md
Large PDFs taking long / missing content Convert to markdown first, or read specific pages
Output cluttering the context Always write substantial output to files

Related