Researchers release AutomationBench, a test suite that exposes how poorly today's AI agents handle real multi-app business workflows

Researchers (drawing from Zapier's real workflow patterns) published AutomationBench, a benchmark that grades AI agents on tasks spanning Sales, Marketing, Operations, Support, Finance, and HR — where a single workflow touches a CRM, email, calendar, and messaging app simultaneously. Even the best frontier models currently score poorly on these tests.

Unlike existing benchmarks, AutomationBench forces agents to discover API endpoints on their own (rather than being told which systems to use), follow layered business rules from policy documents, and navigate data noise where some records are misleading or irrelevant — mirroring the messy reality of enterprise software. Scoring is objective and end-state only: either the correct data landed in the right systems or it didn't.

Business teams relying on AI agents to automate workflows (like syncing a lead from a web form through a CRM, calendar, and email simultaneously) now have a public standard to measure whether their AI tooling actually works in production conditions — not just toy examples. This matters for anyone evaluating automation software before rolling it out to their team.