Not Every Action Deserves the Same Gate
Your support agent holds two tools pointing at the same order system. One reads the status of an order. The other cancels it. In the source they are twins: same client, same credentials, same shape of argument, twelve lines apart in one file. The runtime cannot tell them apart either: nothing attached to either one says what happens if it fires at the wrong moment. Reading a status back costs a second. A cancellation is a refund, a mail to a customer and a phone call on Monday. The difference is real and written down nowhere.
The category is printed on the box
A firework arrives with its class printed on the side. One kind is made for a table indoors and burns at arm’s length. One belongs in a garden, with a keep-back distance printed beside the class. One is not sold to you at all: it leaves the depot against a licence, and whoever fires it writes a plan for the evening first. The sorting happened once, upstream, by whoever filled the tube, and everyone after that runs on the printing.
Risk annotations are the same trick in software. A tool declares what sort of thing it is — read-only, destructive, safe to run twice, or reaching past your own systems into somebody else’s — and a client reads those declarations as signals. It auto-approves the harmless ones, retries where a repeat cannot damage anything, and warns on calls that cross out of your trust boundary. They belong with the rest of a tool’s description.
The catch is what a printed class is. It is a claim by whoever made the thing, not a property of the powder. A crate from a supplier you have never met can say indoor on the side and still put a shell through the hedge. An annotation is worth what the server that wrote it is worth: read-only, declared by a server you do not trust, is a sentence rather than a guarantee.
The ban outranks the licence, every time
A licence does not settle the evening. If the council has banned open flame in a dry week, nothing is fired, and nobody weighs one document against the other. The ban is consulted first, and nothing later gets to overrule it. The sequence is the design.
Permissions for tools work that way, in a fixed order. A hook runs ahead of everything and can stop a command before any rule sees it. Deny rules refuse next, and no permissive rule further down quietly undoes one. Ask rules pull an action towards a person. The mode sets the standing posture for whatever survives, broad allow rules clear ordinary traffic, and only what none of that settled reaches the callback. Most restrictive first, deterministically. The payoff is not only fewer accidents: afterwards you can name the rule that decided.
The human gate is where a case leaves the machine, and it is not somebody watching a screen. It is a classification. The proposed action is matched against written configuration — is this path on the denylist, is this action on the auto-merge allowlist — and if it clears, the loop commits, opens the pull request, merges. If it is risky or ambiguous, it escalates to a person carrying the full context. A small command applies that file mechanically, so nothing depends on the agent’s reading of its own behaviour. A rule that lives only in the prompt is not a gate.
Low risk is not the same as free
Ratings earn their keep in silence: a harmless read runs and is logged, a middling call picks up extra checks, and destructive ones stop for a person. Tiers go wrong in familiar ways. Rate a tool by its name and anything that deletes reads as dangerous while anything that syncs reads as harmless, when the second touches far more. Watch single calls and you miss the version where nothing crossed a line and the sum of a hundred cheap ones did. Block without an escalation route and you have built a dead end, which people solve by switching it off.
The trigger is not only what a tool is. It is also how the run is going. An agent that has failed three times to work out which order the customer means should hand the case over, though no single step was ever high risk. Repetition is as much a limit as any label.
None of this lives in the model. Reversibility, write access, financial effect and required permissions are annotated in a registry beside the tool. The order of evaluation is configuration. The gate is a file you can read and argue with. A model cannot know what it is allowed to break. Somebody has to print it on the box.