Microsoft releases ASSERT, an open source framework that uses AI to turn natural-language descriptions of intended AI behavior into automated tests.

Microsoft on Tuesday unveiled ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), an open source framework designed to simplify testing of application-specific AI behavior by converting plain-language descriptions of goals, policies, or intended behaviors into scored test cases.

The framework takes natural-language descriptions, structures them into acceptable and unacceptable behaviors, generates problem scenarios and test cases, runs them against the target system, and scores the results. Developers can also customize evaluations by providing system context, tools, and constraints; ASSERT records the AI system's decision paths so developers can inspect where failures occur.

According to Microsoft's Sarah Bird, chief product officer of Responsible AI, ASSERT addresses a gap in broader evaluations by focusing on application-specific dimensions. The framework can be used when systems are being built, after deployment, and for continuous monitoring.

The release reflects a broader shift in the AI industry toward repeatable testing and regression checks, with efforts like Stanford's HELM, MLCommons' AILuminate, and evaluation groups like METR developing benchmarks to measure model behavior under different conditions.