Application observability answers "is it up, and how fast?" Agent observability must also answer "why did it do that?" and "was that the right thing?" Those need different data.
Add a decision layer to your traces
Standard distributed tracing gives you spans for model calls and tool calls. Wrap them in spans that represent the agent's *reasoning structure*:
- Run: one end-to-end task, with the requesting user and agent version.
- Plan step: the intent, such as "look up the customer's open orders".
- Model call: prompt version, model, tokens, latency, and structured output.
- Tool call: name, arguments, result summary, policy decision.
- Outcome: the final result, plus any evaluation score attached afterwards.
With that tree, a bad outcome can be walked backwards: which plan step went wrong, what the model saw, which tool result misled it.
Capture what you will need during an incident
- Inputs to each decision, redacted for sensitive data but reconstructable by authorized staff.
- Policy decisions, including denials. A pattern of denied calls often reveals an attack or a confused prompt.
- Version identifiers for prompts, models, tools, and policies.
- Cost and token counts at the step level.
Privacy is part of the design
Traces contain customer data by nature. Apply the same classification and retention rules as the data itself, with role-based access to full payloads and aggregated views for everyone else. Decide redaction rules before launch, not after the first review finds an account number in a log.
Close the loop with evaluation
Attach scores to traces: automated grader results, human feedback, downstream signals such as "customer reopened the ticket". Then you can query for the worst one percent of runs and turn them into eval cases. Observability that does not feed evaluation is just expensive logging.
Alerts worth having
- Rising rate of policy denials or approval rejections.
- Cost per run drifting upward.
- Step counts climbing, which often signals looping.
- Drop in grader scores after a deploy.
When you can answer "what changed?" and "show me the run" in under a minute, you can operate agents in production with confidence.