You paste the whole specification in. Two hundred thousand tokens of room, and the document fits with space left over: the API contract at the top, forty pages of worked examples, and somewhere around page nineteen, one sentence saying that all timestamps are UTC. You ask for a parser. What comes back is clean, well organised, and treats the timestamps as local time. So you go looking for the sentence, expecting to find it cut. It is there. It was in the request you sent, whole and unmodified, and the model read straight past it. Nothing was truncated. Nothing was dropped. The answer is wrong anyway.

A long set sags in the middle

This has a name. Models weigh the beginning and the end of a long context more heavily than the part between them, and the effect is strong enough that information sitting in the middle can be present and still unused. The window is a capacity limit. This is not one. The question here is not how much fits, but how much gets weighted.

Think of a band playing a ninety-minute set. People leave humming the opener and the encore. Songs seven through twelve were played well, at full volume, to a room that was listening — and by the car park they are gone. Adding four more songs does not fix this. It makes the middle longer.

So you have two moves, and both are about running order. You can reorder — pull the passage that actually decides the answer up to the front or down to the end, where it will be heard. Or you can cut, so the useful material stops being buried in filler. LongLLMLingua does both to retrieval prompts and gets up to 21.4% better results using a quarter of the original tokens. Not more context. Less of it, better placed.

The opening stays fixed for a reason

There is a second force pushing on the same real estate, and it comes from the bill. Prompt caching lets the provider keep the stable head of your context — the standing instructions, the files already read — and re-read it cheaply on the next turn. With DeepSeek’s automatic caching, that head costs $0.004 per million tokens after the first request instead of $0.44, roughly a hundred and twenty times less on the repeated turns of a loop. The catch is that the saving is fragile. Change one thing in the cached part and the discount is gone for everything after it.

Read those two facts together and the layout writes itself. The front is where attention is strongest and where the cheapest tokens live, so it wants material that never moves. The end is where each new observation lands, and where attention picks up again. Whatever you keep shuffling drifts into the middle, which is exactly the place that is read least.

Cutting is the other lever. LLMLingua scores each token with a small model and drops the ones carrying no weight; a 2365-token maths prompt becomes 211 tokens, eleven times shorter, and still solvable. That compressor is a model too — it costs something to run, and it will happily throw away a number or a name if you do not protect them.

Someone will cut your set eventually

If the loop runs long enough, the decision stops being yours. Anthropic’s API takes a threshold — say 150,000 tokens — and when the conversation crosses it, an extra sampling pass writes a summary. The next request drops everything before that summary and carries on. You write no summarisation code at all. You also do not choose the running order any more: the material that used to sit in the middle comes back condensed, in a shape a default prompt decided, and custom instructions replace that prompt rather than extend it.

Which is why the window size on a spec sheet is the least interesting number on it. A bigger window buys you more middle, and the middle is the part that gets read worst. It changes what you are able to send. It does not change what gets weighted once it arrives.

None of this is a reason to avoid long contexts. It is a reason to stop treating them as a shelf you can keep stacking, which is the assumption underneath most of what goes wrong in an agent loop that has been running for an hour.

Length buys you room. Position decides what gets used.