All posts

Observability for Agents: Tracing Non-Deterministic Systems

AI AgentsBest PracticesSoftwareDevelopmentDevOps

With ordinary code, a bug reproduces. With an agent, the same input can take a different path every run. You cannot debug what you cannot see — so you have to capture the whole reasoning trail.

TL;DR

  • Agents are non-deterministic; you cannot rely on reproducing a failure.
  • Trace the full trajectory: prompts, tool calls, results, decisions, and tokens.
  • Log inputs and outputs at every step so failures are reconstructable after the fact.
  • Treat evals and production traces as one continuous feedback loop.

Why traditional logging falls short

For deterministic code, you reproduce a bug by re-running with the same input. Agents break that: temperature, changing context, and model updates mean the same starting point can branch differently each time. A failure you saw yesterday may not recur today. So observability shifts from "reproduce it" to "capture enough that you never need to reproduce it."

What to capture

An agent's run is a trajectory — a sequence of reasoning and actions. Capture the whole thing:

  • The full prompt sent at each step, including the assembled context — not just the user's message.
  • Every tool call with its exact arguments and the result returned.
  • The model's decisions — what it chose to do and, where available, why.
  • Token usage and latency per step, for cost and performance.
  • The final outcome and whether it met the success criteria.
Trace: one agent run
  step 1  prompt → model → tool_call(search, {...}) → result
  step 2  prompt(+result) → model → tool_call(edit, {...}) → result
  step 3  prompt(+result) → model → final answer
  totals: tokens, cost, latency, success?

You are not debugging a function. You are reconstructing a decision trail. Capture the trail or you are guessing.

Make traces reviewable

Raw logs are not enough; you need to see trajectories. Useful capabilities:

  • Step-through views of a run, so you can find exactly where it went off the rails.
  • Searchable traces by outcome, tool, error, or cost, so you can find patterns across many runs.
  • Diffing similar runs to see why one succeeded and another failed.

Close the loop with evals

Production traces and offline evals are the same feedback loop seen from two angles. Real failures captured in traces become new eval cases; the eval suite then guards against regressions on those exact failures. Over time your eval set becomes a distilled memory of every way your agent has gone wrong — and your best defense against it doing so again.

What to watch in production

  • Success rate on the task the agent exists to do.
  • Cost and token trends per run, to catch creep and runaway loops.
  • Tool-call patterns — unexpected tools or arguments can signal prompt injection or misbehavior.
  • Latency and step counts — an agent taking many more steps than usual is often a sign something is wrong.

The takeaway

Observability is not optional infrastructure for agents — it is the only way to operate something non-deterministic responsibly. Capture the full trajectory, make it reviewable, and feed real failures back into your evals. When an agent does something baffling at 2am, the difference between a five-minute diagnosis and a lost day is whether you wrote down what it was thinking.

More on running agents in production, on the blog. →