Where the Model Ends and Your Software Begins
Your agent has behaved for three weeks. Then one afternoon it runs a shell command that reaches two directories above the project and removes a build folder that was never its business. You open the transcript. The reasoning is not deranged: it wanted a clean rebuild, and what it wrote does exactly that. So you start rewriting the system prompt, adding a paragraph about staying inside the working folder. Then you stop. That command did not run because the model misunderstood its instructions. It ran because nothing standing between the model and the shell was ever asked whether it should.
The appliance decides, the building sets limits
Think about a heater in a rented flat. The heater decides what to do with the current it receives: how hot, how long, when to cut out. It has no opinion about how much current the room can carry. That question was settled by someone else, years earlier, in the gauge of the wiring, in the shape of the socket, in the breaker that trips when the draw goes wrong. Two pieces of engineering, two owners, one seam between them.
An agent has the same seam. On one side is the model, which decides whether and how to use what it is handed: which file is worth reading, whether a patch is worth proposing, when to stop. What it cannot do is reach anything by itself. The tools it can call, the observations that come back, the folders it may touch, the actions that wait for a human to approve them — all of that is the software around it, which you wrote and which runs on your machine. The loop in the middle carries out each call and reports what happened. It does not replace the judgment with a rigid script.
That seam decides who answers for what
The distinction stops being philosophical the first time something goes wrong. Take the deleted folder. The model picked a reasonable action out of what it was offered, and what it was offered was a shell with no edges. That is a wiring fault, not a heater fault, and the repair is on the wiring side: restrict the paths, hand back a diff so the change is visible, require an approval before anything destructive runs.
The opposite overcorrection is just as common and harder to undo. Burned once, teams start moving judgment into code, growing decision trees that try to anticipate every situation the agent might meet. That is rewiring an entire building so a heater can only ever heat one room to one temperature. Rules at the seam exist to bound action and risk, not to simulate the thinking. Which is what the job actually is: not building the intelligence, but designing the world it works in, so that the work is reliable and somebody can be held to it.
You can wrap the seam from outside
None of this requires opening up the agent. A wrapper can sit around the model client you already use, the way a metered extension lead sits between the socket and whatever you attach to it. The calling code keeps the same interface; the one line that builds the client changes, and nothing else does. From then on every request and every tool call passes through the same pipeline, parsed, classified by risk, checked against permissions, run, its output cleaned before the model sees it, and written to an audit file line by line.
That afternoon’s command would have met all of it. Classified, blocked for acting outside the workspace, logged with a timestamp, its output filtered on the way back. Because the checks live at the seam and not inside the agent, one policy can follow several agents and does not care which provider serves the model. Three ways to lose the benefit: making the loosest mode the default because it is convenient, filtering what goes in while ignoring what comes out, and trusting text patterns to authorize things that matter. The alternative is guardrails scattered through every application, each slightly different. Draw the line on purpose, because everything on your side of it is yours to answer for.