Gray Swan launches AI security platform for language models

What happened

Gray Swan has raised funding and is positioning itself as a provider of AI safety and security tools. The company grew out of over a decade of research by Kolter and Fredrikson at Carnegie Mellon into vulnerabilities in deep learning systems. Their work includes the Gray Swan Arena (a red-teaming platform for testing model robustness), Shade (an adversarial red-teaming tool that Anthropic used to evaluate robustness against prompt injection attacks), and Cygnal (a guardrails product for policy enforcement).

Why it matters

AI systems have inherent vulnerabilities fundamentally different from traditional software—they can be tricked the way people can be tricked. The risk is especially acute because most organizations use a small number of foundational models (like Codex and Claude Code), meaning a vulnerability found in one affects many systems simultaneously. Kolter and Fredrikson argue that a separate security mindset and dedicated AI safety providers are needed alongside new AI platforms, distinct from traditional cybersecurity.

What to watch

The body frames AI security as an emerging domain where specialized red-teaming models can now outperform humans at breaking AI systems, and where frontier models do not automatically become safer as they scale. The founders suggest that AI security may become embedded in insurance and compliance frameworks, and that major prompt-injection breaches may be inevitable—events that are visible in advance but difficult to prevent.