How context fails
The context is everything the model is reading right now: your prompt, the chat history and any tool results. Long contexts do not just fill up. They go wrong in specific, nameable ways. Several of these show up mostly in agents, programs that run a model in a loop with tools, which the next chapters cover.
Words on this page (7)
- prompt
- : The input text you give a model, including instructions, examples and data.
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- agent
- : A model run in a loop that chooses its own tools and steps until the task is done.
- hallucination
- : A confident answer that is not supported by the model's knowledge or the provided sources.
- token
- : A piece of text the model reads and writes, usually a word or part of a word. About 4 characters of English on average.
- attention
- : The mechanism that lets each token weigh all earlier tokens by relevance when computing its next representation.
- system prompt
- : Standing instructions placed at the start of every call that set the model's role and rules.
The failure modes
Poisoning: a wrong fact or enters the context and later steps treat it as true. Distraction: so much accumulated history that the model repeats past actions instead of reasoning anew. Confusion: irrelevant or documents make the model pick the wrong one. Clash: the context contains contradictory information, such as an old and a new version of a spec. Rot: accuracy drops as inputs get longer, even on simple tasks (the causes are not fully understood).
What to do about each
Poisoning: validate facts before saving to memory and let the start fresh when a run goes wrong. Distraction: summarize or drop old turns. Confusion: expose only the tools needed for the step. Clash: keep one source of truth and remove stale versions. Rot: retrieve only what the step needs instead of pasting everything.
Measure it
Test with your real at different lengths: add unrelated pages of text (about 5,000 each) and watch the accuracy. If answers get worse as you add context, you have found the limit for that task. Place the key instructions at the start and the end, where models pay the most .
A worked example: clash
Your context holds the old spec ("orders ship in 5 days") and a newer message ("orders now ship in 3 days"). The model sees both and sometimes uses the old one. Fix: remove the outdated item or write one clear sentence, "Ignore the earlier shipping time. Use 3 days."
Telling the failures apart
Work through a short test to find which failure you have.
- Run the failing step again with a minimal, clean context. If it now succeeds, the cause is clutter: distraction, confusion or rot.
- If the wrong fact appears in the output, search the context for where it first appeared. If it came from an earlier step, that is poisoning.
- If the context holds two versions of the same thing, that is clash.
- If it still fails with a clean context, the problem is the task or the model, not the context. Clutter calls for trimming. Poisoning and clash call for removing the bad item. Diagnose before you trim anything.
A second example: confusion from too many tools
An agent has 40 tools. A user asks for last month's sales total, and the agent calls "get_customer" because its description mentions sales. Many descriptions overlap, so the model cannot tell them apart. The fix is to expose only the five tools the step needs and rewrite each description so it says clearly what it does and when not to use it. Symptom to watch for: the right tool exists, but the wrong one gets called.
Trimming has a cost
Removing old turns can remove something that still matters. Suppose the user said early on, "never use PDFs". After trimming, the agent produces a PDF. The symptom is a broken earlier constraint. Keep standing rules in a separate stable block that is never trimmed (in a chat app, that is the ), and trim only the history around it.
Common mistake
Blaming the model for forgetting when the real problem is clutter in the window.
A coding agent misreads one error message in step 3 and writes a wrong note: "the database is down". For the next 40 steps it builds on that note and never questions it. This is context poisoning: one wrong item in the notes drags everything after it off course.
Check yourself
An old and a new version of a spec are both in the context. Which failure is this?
Not quite. Contradictory information makes behavior unpredictable.
Where do models use information best in a long prompt?
Not quite. Research found accuracy dips for facts in the middle of a long context.
An agent writes a wrong note early and builds on it for 40 steps. Which failure is this?
Not quite. One wrong item in the notes drags everything after it off course.
You got 0 of 3 right.
Go deeper
- ArticleHow Long Contexts FailArticle
Drew Breunig · Names poisoning, distraction, confusion and clash with fixes.
- ArticleContext RotArticle
Chroma · Measures degradation as input length grows.
- PaperLost in the MiddlePaper
Liu et al. · Models use information at the start and end better than the middle.
Spotted a mistake or something unclear? Report it on GitHub.