AWS CloudWatch Omni Brings Purpose-Built Observability to AI Agents

AWS Launches CloudWatch Omni: Observability Built for AI Agents, Not Just Applications

Monitoring an AI agent is fundamentally different from monitoring a web app. A prompt change can degrade response quality even when error rates show zero problems. Teams spend hours reviewing logs across multiple systems, unable to pinpoint what changed or why. Amazon CloudWatch just shipped a purpose-built answer: CloudWatch Omni, a unified observability and evaluation platform for generative AI and agentic workloads.

CloudWatch Omni is now generally available. It captures every trace from an agent’s execution, including LLM calls, tool invocations, and reasoning steps, in a structured timeline. The Trace Explorer lets developers drill into any span to inspect inputs, outputs, token usage, and latency. A Compare mode shows two traces side by side to diagnose regressions when prompt or configuration changes alter agent behavior.

Key Takeaways:

• 17 built-in evaluators score agent responses across dimensions like coherence, helpfulness, faithfulness, and routing correctness — the quality signals standard metrics like latency cannot capture.
• Two surfaces: a VS Code/Kiro IDE extension (free, no AWS account required) and a standalone web experience for operators. Both share the same trace data, so the trace a developer debugs is the same trace an operator investigates.
• Broad framework support: LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, Vercel AI SDK, and Amazon Bedrock AgentCore — with OpenInference and ADOT open standards for instrumentation.
• Playground, Experiments, and Prompt Management tools let teams test prompt variants, run batch evaluations, and version prompt configurations before deploying changes.

The companion web experience also provides application monitoring alongside AI agent observability, giving teams a single pane of glass for both.

Read the full article on AWS News Blog