An afternoon goes badly and you open the dashboard to find out where. One agent reports how long a model call took and files it under latency, in milliseconds. Another reports the same number and files it under duration, in seconds. The third wrapped its entire run — the model call, two tool calls, a retry — inside one event named after the run, so there is nothing inside it to open. Every team instrumented its agent. Nobody skipped the work. You still cannot say which tool got slow, and the fix is not more measurement. None of the three is talking about the same thing.

Every bank writes the payment down identically

Send money to another country and three institutions touch it before it lands. For a long time each of them wrote that down in its own words. One recorded the day the money left, another the day it settled, a third typed the purpose into a free line by hand. The same fact went under three names, so when a payment sat somewhere for two days nobody could say where, and comparing what two institutions charged for the same movement meant learning both their dialects first.

What fixed it was dull. An agreed message, where the fields carry the same names and the same meanings for the operation that actually happened — a transfer, a currency conversion, a charge — and one reference that every institution along the way copies onto its own record. Nothing new is observed. The records can now be laid side by side.

Agent telemetry has the same agreement available, in the GenAI semantic conventions. Standard names and attributes describe the operation that is really occurring — a model invocation, one step of an agent, a tool call, traffic over MCP — instead of fields each agent invents for itself. The runtime keeps the ordinary trace context it already had. An agent takes a request, invokes a model, calls two remote tools, and every span shares one trace ID, so the dashboard attributes latency and errors to the right tool and compares token usage across providers without rewriting the queries for each SDK.

A complete log is also a surface

The interesting part is what the contract records: the identity of the operation, its timing, its outcome, its usage. Prompts, completions and tool output are the part everyone wants next, and they come with a rule attached. They carry personal data, they carry secrets, and they carry instructions from sources you do not control — text a fetched document wrote, not text you wrote. They get collected only under an explicit data policy that says what is classified, what is stripped, and how long the rest is kept. Exporting whole prompts and whole tool results because they were there is the most common mistake in this area, and it is not a tidiness problem. A record complete enough to replay everything is also a record worth stealing.

The bank message is the same shape. It says who, how much, when, and what happened. It does not say why the sender needed the money.

Two other habits quietly undo the contract. One span covering an entire run keeps the total and throws away which operation caused it. And treating one vendor’s attribute names as the agreement between teams works until somebody changes framework, at which point a year of telemetry stops meaning anything.

The pilot configuration is not a promise

A search tool call looks like it failed on a model error. The correlated trace shows a latency spike in the sandbox network and retries by the runtime in the same interval. What gets corrected is the external dependency, not the prompt.

That correlation stops being a convenience the moment volume arrives. A customer service agent grows from a hundred requests to a hundred thousand. A legacy API starts timing out, the retries push token consumption up, and quality falls on the rare cases — the ones nobody wrote a test for. Not one of those three shows up as a failure. They show up as cost per task, escalations to humans, and a tail that got heavier, all of which you can only read if the words hold still while the traffic changes. A setup that carried the first hundred requests proved exactly that much, and the trade-offs have to be watched over time rather than assumed once.

Until every part calls the same thing by the same name, you are not measuring one system. You are collecting opinions about it.