AI Observability: What to Monitor for AI Agents
Published . Updated .
AI means artificial intelligence. AI observability means recording what AI systems do in production, so a team can see what an agent did, what it cost, whether the result was right and whether it stayed within its rules. Without it, a failure shows up as a customer complaint rather than an alert.
Why is monitoring AI agents different from monitoring software?
Ordinary monitoring asks whether a service is up and fast. An AI agent can be up, fast and wrong. It can return a confident answer built on a bad search, call the wrong tool, or take five steps where one would do. The same input can also produce different outputs on different runs.
Anthropic described this problem when it built its multi-agent research system:
"Adding full production tracing let us diagnose why agents failed and fix issues systematically."
Anthropic, How we built our multi-agent research system, June 2025
What should you monitor for an AI agent?
A planning aid. Each group needs an owner and an alert threshold.
| Signal | What to record | Alert when |
|---|---|---|
| Traces | Each step, tool call, input and output, linked into one run | A run fails, loops or takes far more steps than usual |
| Cost | Tokens and spend per task, per agent and per customer | Spend per task jumps or a budget is close |
| Quality | Evaluation scores on sample runs, human ratings, corrections | Scores fall below the agreed baseline |
| Safety | Policy violations, blocked actions, approvals and rejections | Blocked actions rise or a new type appears |
| Latency and errors | Response time, timeouts, tool and model errors | Users wait longer or errors cluster on one tool |
Not sure what your agents did last week? Talk to Chiri or see how Chiri Brain handles it.
Why does cost need its own signal?
Agents use far more tokens than chat. In Anthropic's data, agents typically used about 4x more tokens than chat interactions, and multi-agent systems about 15x more. Anthropic, June 2025 Tracking spend per task, not only per month, shows which workflows earn their cost. Chiri's token cost calculator helps estimate the starting point.
Field guide. The Field Guide to the AI Harness covers the systems that let an AI model act safely, including observability among the components to check. Read the field guide.
Why does quality need people as well as metrics?
Users often trust AI output without checking it. In a 2025 global study of more than 48,000 people, led by the University of Melbourne with KPMG, 66% of employees said they rely on AI output without evaluating its accuracy, and 56% said they had made mistakes in their work because of AI. KPMG, April 2025 If users will not catch errors, monitoring has to. That means scoring a sample of runs against agreed criteria and collecting corrections from the people who use the output. Why AI Agents Break After the Demo covers why quality slips after launch.
Why does safety belong in observability?
Many organisations cannot tell whether their AI systems have been attacked. In IBM's Cost of a Data Breach Report 2025, 13% of organisations reported breaches of AI models or applications, and another 8% did not know whether they had been compromised. IBM, July 2025 Recording blocked actions, approvals and policy hits is how a team finds out. Prompt Injection in the Data AI Agents Read and Human in the Loop AI describe the controls that produce these events.
Which standards help?
OpenTelemetry, the open standard many teams already use for software monitoring, is developing conventions for generative AI. Its GenAI semantic conventions cover spans, metrics and events for AI clients, the Model Context Protocol (MCP) and specific model providers, including spans for invoking an agent and executing a tool. OpenTelemetry, GenAI semantic conventions The conventions are marked as in development, so field names may still change. Using them keeps AI traces in the same tools as the rest of the company's monitoring.
How do you start with one agent?
Start with the agent that does the most work or carries the most risk, and add monitoring in this order:
- Trace every run. Record each step, tool call, input and output, linked by one run identifier.
- Attach cost. Add tokens and spend to each trace, so cost can be read per task and per workflow.
- Agree what good looks like. Write down a small set of quality criteria with the people who use the output, and score a sample of runs each week.
- Record policy events. Log every blocked action, approval and rejection alongside the trace.
- Set alerts and an owner. Choose thresholds for each signal and route alerts to a person who can act.
Once one agent is covered, the same setup extends to the next, and a shared dashboard shows all of them side by side.
Who should own AI observability?
One named owner per agent, with alerts that go to a person who can act. Monitoring that nobody reads is a log, not a control. For how Chiri Brain records each step of an agent run, see See Everything (Because Black Boxes Don't Get Approved).
How Chiri approaches this. Chiri builds, operates, and improves company-specific software as an AI managed service provider. Small teams of senior operators and engineers work with the customer and keep operating the system after launch. Chiri Brain records what each agent retrieved, which tools it ran and which model it called. See Chiri Brain
Running agents in production and not sure what they are doing? Talk to Chiri.
Questions and answers
What is AI observability?
Recording what AI systems do in production: the steps and tool calls of each run, its cost, the quality of its results and any policy or safety events.
What is the difference between AI observability and LLM observability?
LLM observability usually covers calls to a large language model (LLM). AI observability for agents also covers the steps, tools, data and actions around those calls.
What metrics matter most for AI agents?
Cost per task, quality scores on sampled runs, step count and failures per run, blocked actions, and response time.
Do we need a separate tool for AI observability?
Not always. OpenTelemetry's GenAI conventions let AI traces flow into the monitoring tools a company already uses, though the conventions are still in development.
Who should watch AI agent monitoring?
One named owner per agent, with alerts routed to a person who can pause the agent, change its permissions or fix the workflow.