Seeing inside the loop
An agent run is a chain of decisions, tool calls, and observations. When it goes wrong, a single final answer tells you nothing. Observability captures the internal steps as structured traces so you can see exactly what happened.
Spans and traces
- A span records one operation: a model call, a tool call, or a retrieval, with its inputs, outputs, timing, and token cost.
- A trace is the tree of spans for one whole task, showing the order and nesting of every step.
- Metadata tags each span with the model, prompt version, and any error.
Together these let you replay a run and find where it diverged from the right path.
What it unlocks
- Debugging: pinpoint the exact step that produced a bad result.
- Cost analysis: see which spans burn the most tokens.
- Regression tracking: compare traces across versions to catch quality drops.
- Audit: a record of what the agent did and why, for compliance.
Doing it well
- Capture inputs and outputs at every step, not just the final answer.
- Use stable identifiers so spans link to the trace and to each other.
- Avoid logging secrets; redact sensitive fields in the span.
Without tracing, an agent is a black box you can only judge by its last token. With it, every decision becomes inspectable.
Key idea
Observability records each agent step as a span within a trace tree capturing inputs outputs timing and cost, turning a black box run into an inspectable record for debugging cost analysis regression tracking and audit.