Developer builds open-source CLI to test whether SDKs work well with AI agents

A new Apache 2.0 licensed CLI tool lets developers create test suites to verify if their SDKs are compatible with agentic AI systems. Test suites can be created manually or by an AI agent based on source code and documentation.

The tool dispatches sandboxed agents to attempt each test, then grades results using a separate Judge Agent. Test-taker agents access only public information (guides, blogs, package metadata), while Judge agents can also access private information (source code, internal documents) to evaluate performance.

The CLI provides commands for the full workflow—generating, evaluating, and reporting on test suites—plus a local Web UI for inspecting results and editing test cases. Sandboxing ensures host machine secrets are not exposed and egress HTTP requests are monitored to prevent proprietary IP leakage.

Available on GitHub at https://github.com/PSPDFKit-labs/agentic-usability