With ordinary code, a bug reproduces. With an agent, the same input can take a different path every run. You cannot debug what you cannot see — so you have to capture the whole reasoning trail.
TL;DR
- Agents are non-deterministic; you cannot rely on reproducing a failure.
- Trace the full trajectory: prompts, tool calls, results, decisions, and tokens.
- Log inputs and outputs at every step so failures are reconstructable after the fact.
- Treat evals and production traces as one continuous feedback loop.
Why traditional logging falls short
For deterministic code, you reproduce a bug by re-running with the same input. Agents break that: temperature, changing context, and model updates mean the same starting point can branch differently each time. A failure you saw yesterday may not recur today. So observability shifts from "reproduce it" to "capture enough that you never need to reproduce it."
What to capture
An agent's run is a trajectory — a sequence of reasoning and actions. Capture the whole thing:
- The full prompt sent at each step, including the assembled context — not just the user's message.
- Every tool call with its exact arguments and the result returned.
- The model's decisions — what it chose to do and, where available, why.
- Token usage and latency per step, for cost and performance.
- The final outcome and whether it met the success criteria.
Trace: one agent run
step 1 prompt → model → tool_call(search, {...}) → result
step 2 prompt(+result) → model → tool_call(edit, {...}) → result
step 3 prompt(+result) → model → final answer
totals: tokens, cost, latency, success?
You are not debugging a function. You are reconstructing a decision trail. Capture the trail or you are guessing.
Make traces reviewable
Raw logs are not enough; you need to see trajectories. Useful capabilities:
- Step-through views of a run, so you can find exactly where it went off the rails.
- Searchable traces by outcome, tool, error, or cost, so you can find patterns across many runs.
- Diffing similar runs to see why one succeeded and another failed.
Close the loop with evals
Production traces and offline evals are the same feedback loop seen from two angles. Real failures captured in traces become new eval cases; the eval suite then guards against regressions on those exact failures. Over time your eval set becomes a distilled memory of every way your agent has gone wrong — and your best defense against it doing so again.
What to watch in production
- Success rate on the task the agent exists to do.
- Cost and token trends per run, to catch creep and runaway loops.
- Tool-call patterns — unexpected tools or arguments can signal prompt injection or misbehavior.
- Latency and step counts — an agent taking many more steps than usual is often a sign something is wrong.
The takeaway
Observability is not optional infrastructure for agents — it is the only way to operate something non-deterministic responsibly. Capture the full trajectory, make it reviewable, and feed real failures back into your evals. When an agent does something baffling at 2am, the difference between a five-minute diagnosis and a lost day is whether you wrote down what it was thinking.