AWS releases failure detection tools for AI agents that automatically diagnose why they fail in production, reducing diagnosis time from hours to minutes.

What happened

AWS Strands Evals SDK now includes detector functions that automatically scan AI agent execution traces to identify failures and trace causal chains between them. The detectors categorize failures across nine parent categories—hallucination, incorrect actions, orchestration errors, task instruction non-compliance, execution errors, context handling errors, repetitive behavior, LLM output issues, and configuration mismatch—and assign confidence scores and fix recommendations.

Why it matters

When an AI agent's performance drops (for example, goal success rate falling from 85 percent to 70 percent after a deployment), teams previously had to manually inspect traces span by span to understand what broke. This manual diagnosis becomes a bottleneck for teams operating agents at scale. The detectors automate this workflow by answering 'why did it fail?' at the per-span level, complementing traditional evaluators that only answer 'how well did it do?'

What to watch

The detector pipeline uses a two-phase approach—failure detection scans each span against the taxonomy, and root cause analysis traces causal chains and classifies each failure's causality as PRIMARY, SECONDARY, or TERTIARY. It handles sessions of varying sizes through direct analysis, failure path pruning, and chunked analysis with merge for very large sessions. Fix recommendations specify whether changes belong in system prompt, tool description, or other configuration.