Context window
Why a bigger context window does not fix agent quality, and the habits teams use to keep each session's window small.
Written by
-
Benedikt Stemmildt Founder & Co-CEO
Updated:
Glossary 33
The context window holds everything a model reads when it produces its next step: the system prompt, instruction files such as CLAUDE.md, the definitions of every loaded tool, the files it has read and the conversation so far. Its size is counted in tokens and has a fixed limit per model. Whatever falls outside it, the model does not see.
Why is a bigger window not the answer?
Accepting a large input and reasoning across it are two different things. Models recall facts at the start and end of a long context more reliably than facts in the middle, an effect measured as “lost in the middle” (Lost in the Middle: How Language Models Use Long Contexts (opens in a new tab), 2023). As the window nears its limit, a practitioner reports, Claude Code also starts cutting corners to finish the task before the window runs out (I Didn’t Know These Claude Code Rules (Big Mistake) (opens in a new tab), Sean Kochel, December 2025). Loading the whole repository therefore works against the agent.
How do teams keep it small?
They keep unneeded tokens out of the main session and move bulky work into other windows. Load tool servers only for the task that needs them, keep the always-loaded instruction file short, and let a subagent do the searching and reading and return a summary. Order matters too: fixed constraints early, the immediate task last. Context Engineering is the practice built around this budget.
Related
- Context Engineering: deciding what enters the window and when.
- Protocol and context cost: what MCP tool schemas take from the window.
- Just-in-time retrieval: loading data when a step needs it.
Resources
Papers and reports
- Lost in the Middle: How Language Models Use Long Contexts (opens in a new tab) - arXiv paper from 2023 measuring that models recall facts at the start and end of a long context better than in the middle.
Talks and videos
- I Didn’t Know These Claude Code Rules (Big Mistake) (opens in a new tab) - Sean Kochel’s video on Claude Code rules, reporting that the agent cuts corners as it approaches its context window limit.