AWS Field Sales gets nine-stage agent eval on Amazon Bedrock

The AWS post frames agent quality as a lifecycle problem rather than a last QA step. The nine stages span from defining success to an automated feedback loop, each addressing a question teams face as agents mature. The framework applies at three levels simultaneously: tool, agent, and multi-agent, with each level running its own golden set, evaluators, and promotion criteria. The post highlights that agent-level quality alone misses a third failure surface that neither tools nor single agents can see.

Before launch, testing the Customer Insights Agent surfaced examples where the agent reported $410.08 as "$410.08 Million", labeled Sales Revenue as Sales Revenue excluding Adjustments, offered an Annual Recurring Revenue estimate not in the data, and presented its own calculations as sourced facts. Monitoring stayed green throughout, indicating the system responded but not whether the answer was right. A coordinated-release mismatch that dropped the pass rate from 78 percent to 38 percent is cited as a related risk that persists before launch and grows with data, prompt, code, or model changes, or upstream data refreshes every week.

AgentCore Evaluations is named as the connective tissue, alongside Amazon Bedrock Guardrails for safety, Amazon CloudWatch for metrics, alarms, and tracing, Amazon S3 and Amazon Redshift for evaluation persistence, and AWS Lambda for custom evaluators. The post notes that no agent ships or keeps running if it can't meet its safety bar, and that quality gets the most space because it is the hardest of five dimensions to get right. The remaining dimensions are performance, cost, and operational control.