Amazon Nova Act enables automated UX testing at scale

Amazon has published a comprehensive guide for deploying an automated UX testing platform built on Nova Act, a multimodal foundation model capable of understanding web interfaces through vision and action. The traditional UX testing landscape faces multiple bottlenecks: manual testing does not scale (testers can only evaluate a limited number of user journeys, leaving edge cases unexplored), and traditional automation tools require hard-coded scripts that break whenever interfaces change, creating maintenance overhead that limits test coverage. Comprehensive testing across diverse user journeys, device types, and interaction patterns remains prohibitively costly and time-consuming for most organizations.

Nova Act offers a fundamentally different approach by mimicking human reasoning when navigating interfaces. Rather than relying on predefined element selectors like traditional tools such as Selenium or Playwright, Nova Act processes visual information from screenshots, analyzing page layout, identifying interactive elements through visual cues, and making contextual decisions about which actions to take next. This visual understanding allows Nova Act to adapt to interface changes and handle dynamic content gracefully. The model's reasoning and chain-of-thought logs also provide valuable insight into website design and intuitiveness.

The proposed solution is composed of four layers. The documentation processing layer ingests site documentation, user guides, and flow testing specifications from Amazon S3 into an Amazon Bedrock Knowledge Base for semantic similarity search. AWS Lambda then uses Claude 4.5 Sonnet in Amazon Bedrock to transform user flows into comprehensive testing scenarios—for example, converting the task "buy a coffee maker via search" into detailed step-by-step interaction paths at multiple levels of granularity. The orchestration layer manages test execution at scale using Amazon DynamoDB to store generated test flows, DynamoDB Streams to trigger batch processing, and Lambda functions to coordinate execution and spin up Amazon ECS tasks for parallel processing. The execution layer runs intelligent user flow testing using Amazon ECS with AWS Fargate for scalable, serverless compute, with Nova Act agents executing user flows in parallel browser sessions and real-time interaction logging capturing detailed behavioral data. The analysis layer transforms test results into metrics: AWS Lambda processes results using Amazon Bedrock to analyze flow execution patterns, calculate usability scores, and identify friction points, with results stored in DynamoDB and presented in a React dashboard.

The complete solution is available in the aws-samples GitHub repository and can be deployed using AWS CDK. Prerequisites include Node.js v20 or newer, npm v10.8 or newer, and an AWS account. After deployment, organizations can generate test flows through automatic generation from documentation (using Claude to convert tasks like "purchase a highly-rated stainless steel coffee maker" into precise multi-level instructions), manual flow definition by inserting JSON directly into DynamoDB (useful for edge cases and new features), or a hybrid approach combining both. The system supports authenticated user flows through persistent browser sessions, structured data extraction via Pydantic schemas, and file operation testing. Execution results include summary metrics (duration of each step, number of actions required, success/failure status, and extracted data), detailed chain-of-thought reasoning logs, screenshots, and behavioral data stored in Amazon S3.

UX testing has long been constrained by the labor intensity of manual evaluation and the fragility of script-based automation. Manual testers can only evaluate a limited set of journeys, leaving edge cases unexplored, while traditional tools like Selenium and Playwright require hard-coded scripts that break whenever a website's interface changes—a problem that scales poorly as applications evolve. This creates a practical ceiling: comprehensive testing across diverse user journeys, device types, and interaction patterns remains prohibitively costly and time-consuming for most organizations.

Amazon's solution leverages Nova Act's multimodal capabilities—its ability to understand and interact with web interfaces through vision and action—to sidestep both constraints. Rather than relying on element selectors or predefined scripts, Nova Act analyzes screenshots as a human tester would, identifying interactive elements through visual cues and making contextual decisions about next steps. This approach allows Nova Act to adapt when layouts change and to handle dynamic content gracefully. The architecture compounds this advantage by orchestrating parallel test execution across cloud infrastructure (using ECS with AWS Fargate), enabling the testing of many user flows simultaneously rather than serially. The system also automates test scenario generation from existing documentation via Claude 4.5 Sonnet, eliminating manual test-writing overhead for baseline coverage, while still allowing teams to manually define flows for edge cases or features not yet documented.