Your agent has read from the same internal wiki for three months. On Tuesday it starts coming back empty. The search tool returns an error, the agent apologises, tries again, then writes a confident summary of nothing. So you tighten the instructions. Retry on failure. Check the results before summarising. Say so when you have no data. Nothing improves, because nothing you touched was ever broken. Overnight the wiki connector went up a minor version, one field in the response changed shape, and the tool contract the agent was handed now describes something the tool no longer returns.

The silent note is rarely the pipe

An organ tuner walks into a church because one note will not sound. They do not start at the pipe. If a whole rank has gone quiet, they go to the wind supply, because a rank is fed as a unit. If the same note is dead under every stop, they open the key action, because that linkage is what every stop shares. Only when one note fails under one stop do they climb up and look at the pipe itself. None of this is intuition. It is a catalogue built out of years of call-outs, and it converts a sound into a place to open the case.

Agent faults have been catalogued the same way, by reading through issues and pull requests in real agentic applications rather than reasoning about what could go wrong. What comes out is not sorted by severity or by how annoying the bug felt. It is sorted by architecture: faults in how the agent is initialised, in what it carries between steps, in how work is handed from one step to the next, and in the seam where it touches an outside system. The prompt is the pipe. It makes the sound, so it gets the blame, and it is usually the last thing wrong.

A symptom points upstream, not inward

What you actually observe is small and repetitive. Structured output that will not parse. A tool call that comes back an error. An exception nobody handled. A taxonomy earns its keep by attaching each of those to the causes that keep producing it — a schema that no longer matches, a dependency that drifted, state that grew past what anyone models, a model interface that changed under you — and to say which component to read first. Tuesday’s wiki failure has a shape the catalogue already knows: a tool that worked and then stopped, right after a deploy. That sends you to compare the connector’s version against the contract the tool declares, before you write a single new instruction.

The difference is what an hour of debugging buys. Rewriting the prompt is cheap to start and unbounded to finish, and no wording will reconcile two schemas that disagree. It also decides what you are willing to leave running by itself. A failure you can name and locate is one you can put a check around, the way you can guard the shape of what comes out. A failure you attribute to the model’s mood is one you can only supervise.

Break the environment before it breaks you

Naming faults gets you halfway. The other half is meeting them before a customer does, which is hard when the job is financial analysis or anything else with no safe public sandbox. One approach lets a language model play the environment: it answers the agent’s tool calls in a way that stays consistent with the domain and with what the agent has already done, over tasks built to be solvable, graded for difficulty and grounded in documents. Then faults get injected on purpose — an explicit error, quietly degraded data, or both at once.

That last one is the test worth stealing. A run replaces one field with a missing value and says nothing about it. A good agent notices the hole, goes looking for confirmation, and declines to reach a conclusion it cannot support. A weak one fills the gap and sounds just as certain as always. The scoring keeps the agent’s completion and robustness apart from the simulator’s own quality, because a sloppy environment will misjudge a careful agent, and you would never know which one you were measuring. A fault with a name is something you can hunt for. A fault without one just becomes another line in the prompt.