The Context Window: Why Agents Forget and When to Start Over
An AI agent does not have unlimited memory: everything it is working with sits on a desk of fixed size. An explanation with everyday pictures of why long conversations make an agent worse, when to clear, summarize or start over, with a worked calculation, a handoff template and a checklist.
For anyone who uses an AI agent to write code or do multi-step work, beginners included. No knowledge of how models work is needed. Every term comes with an everyday picture, and the calculation is marked so you can skip it if you do not need it.
Numbers come from public sources listed at the end. Examples labeled "illustration" are constructed and name no real person or company. How a model uses long input internally is not fully understood by anyone, so the pictures below are memory aids only. Command names come from the Claude Code documentation at the time of reading; other tools and later versions may differ.
Quick summary#
- An agent only "remembers" what sits inside its context window: your request, the files it read, command output, project instructions. The window has a limit, and everything you open takes up room.
- Several studies found models use long input less reliably than short input. Liu et al. (2023) found results are often best when the needed information is at the start or end and drop when it sits in the middle. A Chroma report (July 2025) tested 18 models and found reliability fell as input grew longer. The authors themselves say their tasks are simpler than real work.
- When a conversation is nearly full, the tool may compact it into a summary. The Claude Code documentation says the summary keeps your requests and key code, but details of early instructions can be lost.
- In this post's worked example, a long session fills about 91.5% of the window, and about 43% of that is finished, old work. Starting over with a short handoff brings usage down to about 8%. These are illustration numbers for you to recompute.
- Lasting rules belong in the project instruction file, not in the conversation. Switching to unrelated work: clear the conversation. Same task but long: compact with guidance.
- Starting over has a cost too: the agent has to reread files, and the handoff may be incomplete or wrong. The "when" part of this post is a rule of thumb, not a measured result.
Hard words, plain meaning#
| Term | Plain meaning |
|---|---|
| Context window | The agent's work desk: everything open on the desk is what it can see. The area is limited. |
| Token | The unit models use to count text. A short word is usually one token, a long word can be several. Like counting bricks instead of walls. |
| Context rot | A name for the observation that accuracy often drops as input gets longer. Like a cluttered desk where the paper you need is hard to find. |
| Compaction | The tool summarizes a long conversation into a short one and continues from it. Like copying notes onto a fresh sheet and throwing out the pile. |
| Handoff | A few lines you write so a new session knows what is being done, what is decided and what is left. Like shift notes. |
| Subagent | Another agent that works at its own desk and sends back only a summary. Like asking a colleague to look something up. |
| Instruction file / memory | A file such as CLAUDE.md or AGENTS.md that the tool loads every session. Like house rules posted on the wall. |
| Session | One conversation from opening to closing. A new session starts with a clean desk. |
1. A picture first: a desk of fixed size#
Picture working at a desk that is not very big. You spread out your boss's request, a few files, the results of some lookups, your notes. At first the desk is empty and you see everything. Two hours later the desk is full of paper and the sheet you need is under three others. You can still work, but slower and with more mistakes. If it gets completely full, you copy the important points onto a fresh sheet, clear the desk and carry on. The copy is sure to be missing a few details.
An agent works much the same way. The Claude Code documentation describes the context window as holding conversation history, file contents, command output, instruction files, automatic memory, loaded skills and system instructions. As you work, the window fills. This is only a picture to help you remember: inside the model it is certainly more complicated than a desk.
Takeaway: "the agent forgot" is often not laziness or stupidity but limited room, and what occupies the room is not always what you need. Managing the desk is your job.
2. What takes up room, and what is left after compaction#
2.1 What takes up room#
According to the Claude Code documentation, before you type anything some things are already on the desk: instruction files, automatic memory, the names of MCP tools and skill descriptions. After that every file read, every command output and every exchange adds more. The /context command shows usage by category with suggestions for reducing it.
2.2 What is left after compaction#
When nearly full, the tool compacts the conversation into a summary. The table below is a digest of the "What survives compaction" section of the documentation, in the version I read. Details may change between versions.
| Kind of content | After compaction (per the documentation) |
|---|---|
| Instruction file at the project root | Reloaded from disk |
| Automatic memory | Reloaded from disk |
| Files read or edited | Up to five most recently modified are reread, very large files come back as a path only |
| Path-scoped rules, instruction files in subfolders | Reloaded on demand, content loaded earlier may be summarized away |
| Instructions spoken in the conversation | Inside the summary, details from early on may be lost |
Two takeaways from the table: if a rule must survive, put it in the project-root instruction file, not in speech. And if you want control over what is kept, compact on purpose with guidance, for example /compact focus on the auth bug fix, instead of letting the tool guess what matters.
Takeaway: lasting rules belong on the wall poster, not in the pile of paper on the desk. See Writing AGENTS.md for how to write that poster.
3. Why long conversations make agents worse#
Picture reading a 300 page file and then being asked about a detail on page 150. You may manage, but it is much harder than when you were handed just one page. Research on language models shows something similar, though why nobody has explained completely.
3.1 Liu et al. (2023): information in the middle gets missed#
The authors (including Percy Liang) tested answering questions from many documents and retrieving a value from a list of keys and values. According to the arXiv abstract, performance is often highest when the relevant information is at the beginning or end of the input and drops significantly when the model must use information from the middle of a long input, even for models advertised as supporting long contexts. This is an experiment with 2023 models. The abstract page I read does not name the individual models, so newer models may behave differently.
3.2 Chroma (July 2025): 18 models, reliability drops as input grows#
Chroma's technical report (Hong, Troynikov, Huber) tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3 families, on three kinds of tasks: an extended "needle in a haystack" test, question answering over long conversations (LongMemEval, 306 prompts averaging about 113,000 tokens) and repeating a string of words. The authors write that performance grows increasingly unreliable as input length grows. They also found distracting content makes results worse, even when there is a single distractor, and that models react differently: according to them, Claude models tended to abstain when things were ambiguous, while GPT models tended to answer wrongly more often with distractors present.
Read this alongside the authors' own words: their tasks are simpler than real work, so they expect the effect to be worse in practice, but it is also hard to separate the effect of length from task difficulty, and the report does not explain the mechanism. It is a company's technical report, and I have not seen an independent review.
3.3 Anthropic (September 2025): a finite "attention budget"#
Anthropic's post on context engineering (29 September 2025) uses the term context rot and describes context as a finite resource, where every added token uses up part of the model's "attention budget". It proposes a few directions: compact when nearly full, keep notes outside the window to read back later, and use subagents with their own desks. This is the technical view of a company that sells tools, not a controlled experiment.
Takeaway: the three sources point the same way: longer does not usually mean better. How much, and why, is uncertain, and it differs by model.
4. A calculation you can recheck (illustration, skippable)#
This is an illustration with constructed numbers, not measurements. Suppose a 200,000 token window. A session consists of: 12,000 tokens loaded at the start, 25 file reads of 3,000 tokens each, 40 command outputs of 1,500 tokens each and 60 exchanges of 600 tokens each.
window = 200_000
startup = 12_000
files, file_size = 25, 3_000
cmds, cmd_size = 40, 1_500
turns, turn_size = 60, 600
used = startup + files * file_size + cmds * cmd_size + turns * turn_size
print(used, round(used / window * 100, 1))
# 183000 91.5
# suppose half of the work is an old, finished task
old = 0.5 * (used - startup)
print(old, round(old / window * 100, 1))
# 85500.0 42.8
# start over with a 4,000 token handoff
fresh = startup + 4_000
print(fresh, round(fresh / window * 100, 1), (window - fresh) // file_size)
# 16000 8.0 61 (how many more 3,000 token files fit)
Reading the result: the long session uses about 91.5% of the window, and about 43% of it is old work. Only about 5 more files fit (17,000 divided by 3,000). After starting over with a 4,000 token handoff, usage is about 8% and about 61 more files fit.
Note clearly: all the token sizes above are assumptions, including the assumption that half the session is old work. Window size, the automatic compaction threshold and how tokens are counted depend on the model and the tool. The calculation does not say 8% gives better results than 91.5%, only how much room old work takes. Run /context in your real session to see real numbers.
5. When to do what: clear, compact, split or start over#
Picture the desk: sometimes you only need to push some paper aside, sometimes copy notes onto a fresh sheet, sometimes move to a different desk.
| Situation | Action | Basis |
|---|---|---|
| Switching to unrelated work | Clear the conversation (/clear) |
Claude Code docs: the old conversation crowds out the files you need next and costs tokens on every message |
| Same task but long and nearly full | Compact with guidance (/compact focus on <topic>) |
Claude Code docs: the summary keeps what you choose instead of what the tool guesses |
| Need to read many files only to look something up | Hand it to a subagent | Docs: file contents stay on the subagent's desk, you only get a summary |
| A rule keeps being forgotten | Put it in the project-root instruction file | Docs: that file is reloaded after compaction |
| Fixing the same error repeatedly without success, or the agent starts looping | Write a handoff and open a new session | My own rule of thumb, no measurement |
| Continuing on another day | New session plus a handoff | Docs: each new session starts with a clean window |
The "basis" column shows which parts are in the documentation and which are personal experience. For specific thresholds such as "two corrections" or "more than half the window" I have not seen a study that measures them, so treat them as a starting point to adjust yourself.
5.1 Signs it may be time to start over#
- You switch to a task unrelated to most of what is on the desk.
- The agent breaks a rule you stated and you have to remind it a second time.
- The agent keeps editing the same spot or proposes again something already rejected.
- You no longer remember what the conversation settled (the agent will find it harder too).
- After an automatic compaction, quality visibly changes.
/contextshows most of the room is command output and files from finished work.
6. A handoff template (copyable)#
Picture shift notes: the next person reads for a minute and can continue. Paste this block at the start of the new session, or save it as a file in the project to read again later.
## Handoff
Goal: <one sentence>
Done and checked: <what, with the command run and result>
In progress: <file, function, next step>
Decided and why: <decision, one line each>
Tried and failed: <approach, why dropped>
Do not: <boundaries, files not to touch>
Unchecked or assumed: <what the agent guessed rather than ran>
Command to rerun the check: <one command>
The "unchecked or assumed" part comes from Make AI Output Prove Itself: separate what was run from what was only guessed, so the new session does not treat a guess as fact. You can ask the agent for a draft and then read and fix it yourself, because a summary written by AI can also be incomplete or wrong.
A guided compaction command, for when you stay on the same task:
/compact focus on: current goal, decisions made, files being edited,
the check command, and what must not be touched
7. Illustration: one morning, two ways of working#
This is an illustration, naming no real person or product.
Without clearing the conversation:
- 09:00. A dev asks the agent to fix a rounding bug in invoices and says: "New function names must start with
calc." - 09:40. The agent has read 25 files and run many commands. The bug is fixed, tests pass.
- 10:30. In the same session, the dev asks for a button colour change on another page.
- 10:45. The agent adds a function named
computeColor, against the rule given at 09:00. The dev reminds it and the agent fixes it, but later proposes again an approach that was rejected at 09:20.
With clearing and a handoff:
- 09:40. Fix done, the dev commits and saves a short handoff to the project's notes file. The
calcrule is already in the root instruction file. - 10:30. The dev runs
/clearand opens the new task with a clear request. - 10:45. The agent reads the instruction file and names the function correctly the first time.
What to do in both: lasting rules go in the instruction file, finished work gets committed and then the desk is cleared. This is an illustration, so it proves nothing. The evidence is the research in section 3, and it has limits too: older models, tasks simpler than real work. See also Seven Mistakes When Writing Requests for Agents.
8. Where following this still breaks#
| Situation | Mechanism | How to catch it |
|---|---|---|
| Compaction loses an important detail | The summary keeps what the tool judges important | Compact with guidance, put required details in the instruction file or handoff |
| The handoff contains something wrong | An AI-written draft can omit or misread | Read it yourself, mark assumptions, check with the rerun command |
| Starting over too early | The agent has to reread files, which costs effort and tokens | Only do it when a sign in 5.1 appears |
| A rule tied to a subfolder gets summarized away | It is loaded on demand into the conversation history | Required rules go in the root instruction file |
| Believing a big window is enough | Accuracy can still fall as input grows | Still clear, still hand lookups to a subagent |
| A huge file or output refills the window at once | Automatic compaction may stop after a few tries | See the troubleshooting docs, split the file, use a subagent |
9. Numbers and documents used#
| Source | Says | How to read it | Caveats |
|---|---|---|---|
| Liu et al., July 2023 (v3 Nov 2023) | Performance highest when information is at the start or end, lower in the middle | Position in a long input matters | 2023 models, abstract page does not name them |
| Chroma, 14 July 2025 | 18 models, reliability falls as input grows | Longer is often not better | One company's report, tasks simpler than real, no mechanism explained |
| Anthropic, 29 Sept 2025 | Context is a finite resource, suggests compaction, outside notes, subagents | Basis for the practices | View of a tool vendor |
| Claude Code docs (How Claude Code works, Explore the context window) | What loads first, what survives compaction, how /clear, /compact, /context work |
Basis for sections 2 and 5 | Command names and behavior change by version |
Read it correctly: all three research sources hint at "longer is often less reliable". I did not find a controlled study that measures directly how much "clearing the conversation" or "writing a handoff" improves results in real work. The thresholds in section 5 are rules of thumb.
10. Habits to apply from tomorrow#
10.1 Eight questions mid-session#
- Is the current work still related to most of what is on the desk?
- Are the required rules already in the root instruction file, or only spoken?
- What does
/contextshow taking up the most room? - Has finished work been committed and checked?
- Do I need the agent to read many files only for lookup, and should a subagent do it?
- If I compact, what do I want kept (have I written it as guidance)?
- If I start over, does my handoff have an "unchecked or assumed" part?
- Am I starting over because of a clear sign, or out of habit?
10.2 Three things to do now#
- Run
/contextin your current session and note the three biggest items. - Move the three rules you keep repeating into a short block in the root instruction file.
- Save the handoff template from section 6 as a file and try it on one task this week.
11. A balanced view: what starting over does and does not do#
It does: reduce how much old work takes up room, avoid repeating what was already rejected, make you write down what was decided (useful for later readers too) and give important rules a durable home in the instruction file.
It does not:
- It does not guarantee better results. A new session loses details the handoff did not record, and the agent has to reread files, costing effort and tokens.
- I have no measurements from real work for the "when" thresholds. Try them and compare.
- Tools differ in how they compact, reload and set thresholds, and this changes between versions. The documentation you read today may not be right in a few months.
- A summary written by AI can also be wrong.
Do not overstate: this is not a cure for every case of agent mistakes, many come from vague requests rather than a full window. Do not dismiss it: a conversation that runs on with nobody looking at the desk is where small errors pile up, and clearing is cheap compared with guessing what the agent forgot.
12. Conclusion#
An agent does not have unlimited memory: it works on a desk of fixed size, and research suggests a cluttered desk often means less reliable results. Put lasting rules in the instruction file, compact with guidance when you stay on the same task, clear when you change tasks, and when you start over bring a short handoff that you have read yourself.
References#
Numbers and descriptions come from public sources as of October 2026. Command names and tool behavior change by version, so open the originals to check before quoting.
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv, July 2023, version 3 on 20 November 2023)
- Hong, Troynikov, Huber (Chroma), Context Rot: How Increasing Input Tokens Impacts LLM Performance (14 July 2025)
- Anthropic, Effective context engineering for AI agents (29 September 2025)
- Anthropic, Claude Code documentation: How Claude Code works and Explore the context window
Frequently asked questions
- What is a context window?
- It is all the text the model can see in one working turn: your request, the files the agent read, command output, system instructions and memory files. It has a size limit measured in tokens. Picture a desk of fixed size where every open document takes up space.
- Why does the agent forget what I said at the start?
- Two common reasons. First, when the conversation is nearly full the tool may compact it into a summary, and the Claude Code documentation says details of early instructions can be lost. Second, studies such as Liu et al. (2023) and the Chroma report (2025) suggest model accuracy often falls as input grows longer, especially for information in the middle. The deeper cause is not fully understood.
- When should I use /clear and when /compact?
- According to the Claude Code documentation, run /clear when you switch to unrelated work, because the old conversation crowds out the files you need next. Run /compact with guidance, for example /compact focus on the auth bug fix, when you are still on the same task but the conversation has grown long. These are Claude Code commands, other tools use different ones.
- Where should important rules live so the agent does not forget them?
- In the project instruction file (such as CLAUDE.md or AGENTS.md), not in the conversation. According to the Claude Code documentation, the CLAUDE.md at the project root is reloaded from disk after compaction, while spoken instructions from early in the chat can be lost. Rules tied to a subfolder or a path pattern load on demand and can be summarized away.
- Does a bigger context window solve this?
- Only partly. The Claude Code documentation lists some models with a one million token window, but the Chroma report on 18 models found accuracy still tends to fall as input grows, to different degrees across models. A larger window gives more room, not a guarantee that the model uses all of it well.