An hour into a run you go and look at the bill, braced for it. Every turn re-sends the whole thing: the standing instructions, the tool definitions, the four files the agent read in its first minute, then the growing pile of everything it has done since. Fifty turns of that. The number you find is a fraction of what you multiplied out in your head. Most of those tokens were not charged as new — the provider had seen them before, in that exact order, and read them back for almost nothing. Then you fix a typo in one tool description, and the rest of the run costs full price.

The courses below never get laid twice

This is prompt caching, and the mechanism is dumber than it sounds. Before answering, the provider compares your request against ones it has already handled, looking for the longest run of identical text starting from the very beginning. Not similar text — identical, in an order that is fixed for you: tool definitions first, then the standing instructions, then the conversation. Everything up to the first difference gets reused. Everything after it is fresh work.

So picture a bricklayer coming back to a job. What went up yesterday is standing; nobody lays those courses again, and starting the morning costs nothing. A new course along the top is quick. But pull a brick out near the ground and it is never one brick — what sits above it comes down and goes back up, in order, one course at a time.

Which is why a context that only grows at the tail is cheap, and one that gets edited is not. That is a design rule with a name. When something shifts mid-session — the agent moves from read-only to approved writes — you do not go back and rewrite the message that set the old policy. You add a new one saying the policy changed. Same information reaches the model, and nothing has been demolished.

The mark goes below whatever still moves

None of this happens because you were hoping for it. You place a mark in the request meaning: everything up to here is settled, keep it. What sits before that mark is what gets matched next time.

Where you put it is the whole trick. It belongs after the last thing that will still be there, byte for byte, on the following turn. Put it after today’s date, or after the user’s question, and you have marked a line that lands somewhere different every turn — the mark is real and the saving is nil. Nothing warns you. You are simply billed.

The numbers do come back, though. Every response reports how much was written into the cache, how much was read back out of it, and how much was paid for in full. If that middle figure sits at zero across a long run, something near the front is moving and you can go and find it.

Which promotes a housekeeping habit into an architectural one. Renaming a tool, reordering the definitions, adding an example to a description: all of it used to be free, and none of it is. It is the brick near the ground.

Something upstream will change on its own

Sooner or later the front of the context changes without you touching it. The run goes on long enough that the harness compacts, and the accumulated history is replaced by a shorter account of itself. That is an alteration high up in the request, and everything below it is new again.

You cannot prevent that and would not want to, since a session unable to shed weight dies at the window limit. What you can do is put a mark immediately after the standing instructions, above whatever the summary replaces. A two-thousand-token system prompt then holds its own place and stops being re-read at full price every time the conversation underneath it is rewritten.

The rest is what survives the rewrite. Shortening is the easy half; what makes a compacted session worth continuing is that the task, the decisions already taken and the next checkable step come through intact — and that stale tool output, a directory listing from a hundred steps back, gets thrown away before the summary is written rather than faithfully summarised. Every pass of the loop re-sends all of it, so the shape of what you send is the shape of what you pay.

Grow the context at the end and it costs you almost nothing. Edit the beginning and you buy the whole thing twice.