Someone from the support team asks which conversations the agent handled badly last month, and what it did inside them. You have a tool for exactly this. It shows latency percentiles, token counts, error rates, and a viewer where you can open one conversation at a time and read it. What it cannot do is put those conversations next to the satisfaction scores, because the scores live in the warehouse with everything else the company measures. So you answer with an anecdote. The problem is not the question. It is that your traces are sitting inside a tool instead of a table.

Every crossing was logged, none compared

For most of the age of sail, every ship kept a record of its own passage. At noon the mate wrote down the position, the wind, the set of the current, the state of the sea, and at the end of the voyage the book went into a cupboard in the shipping office. Thousands of crossings, all of them observed, none of them comparable: each book was a private account of one journey, readable by whoever thought to pull it out. Then someone copied the pages into a single sheet organised by patch of ocean and month of the year, and a question that had been a matter of opinion — which way to go in April — became something you could look up. No captain changed what he did on watch. Only where the pages landed changed.

An agent runtime can land them the same way. Instrumentation built on OpenTelemetry intercepts the model client and the vector store, attaches semantic attributes to each span, and exports through the collector your company already runs, so a tool call that queries a vector database and then invokes a model arrives as a tree in the observability backend you had before. An exporter with a defined schema writes the same events — sessions, tool calls, outcomes — into the warehouse. The application logic is untouched. The destination is different.

The join a dashboard cannot draw

Once they are records in a warehouse, the interesting questions are the ones nobody built a screen for. A team joins the transcripts of its support agent with CSAT scores and looks for the tool calls that keep turning up in the conversations that ended badly. No observability product offers that view, because half of it is not telemetry at all: the scores come from a survey, and the survey is in the warehouse for reasons of its own. The join works because both sides are tables.

The same is true inside the stack. With traces, metrics and logs queryable in ordinary SQL, you can pull every request that failed validation in a given hour and set the duration of the inference beside the database time for the same window. That comparison usually tells you whether the fault sits in the agent’s reasoning or in the software around it. Keep only the model calls and you cannot make it: the root cause stays ambiguous, and the optimisation lands on the wrong component. One caution, because this material is not neutral. The tables hold prompts and user data, so redaction and retention are decided before the first query, not after.

A finding becomes a test that repeats

The last step is what makes the trouble worth it. A regression you notice in production — the agent has started inventing an order status — becomes a case in an evaluation dataset. The case runs in continuous integration and fails the pull request that would bring it back. The same evidence feeds an experiment: a new version of the prompt, the old one, the same cases, two numbers to compare. Observing, checking and improving stop being three products that do not speak to each other, because all three are reading the same rows.

That loop has one honest failure mode, and it is optimising against offline scores alone. A variant that wins on the dataset and costs twice as much to serve, or drifts on the traffic the dataset never contained, has won nothing; real behaviour and real spend have to stay in the picture. Which is a great deal easier when the evaluation results and the production bill are two queries against the same place.

Agent logs are not a format that needs a special viewer. They are events in a table, and a table answers questions you never thought to ask when you wrote it.