OpenAI has created GPT-Red, an AI model trained to attack other AI systems in order to find and patch security vulnerabilities before deployment.
By using a self-play training loop where GPT-Red attempts attacks while other models practice defense, the company discovered new attack types—notably a 'fake chain of thought' injection—and demonstrated that its latest flagship model, GPT-5.6, now blocks more than 77% of these attacks, up from only 10% on the prior version.
The approach helps OpenAI keep pace with safety testing as AI models grow more complex and are used in increasingly risky real-world settings like code editing and web browsing.
What happened
OpenAI built GPT-Red, an LLM trained to attack other AI models in a self-play loop, and used it to strengthen GPT-5.6. The model discovered new attack types, including a 'fake chain of thought' injection that tricks models into acting on spoofed information. When tested on the same red-teaming task human testers performed in 2025 against an earlier GPT-5 version, GPT-Red was more successful at finding effective attacks than the humans.
Why it matters
As LLMs grow more capable and are deployed as agents that interact with files, websites, and code, human red-teaming teams alone cannot keep pace with the expanding attack surface. GPT-Red automates this safety evaluation at scale. OpenAI found that fewer than 23% of GPT-Red's strongest attacks worked against GPT-5.6, compared with more than 90% against GPT-5 released in August last year—showing the approach can measurably improve model robustness before release.
What to watch
GPT-Red has limitations: it struggles with back-and-forth conversational attacks and image-based prompt injections that human testers handle well. OpenAI says GPT-Red supplements rather than replaces human red-teamers and will not release the model itself, citing the high compute cost required to train such a system.
Ask the AI about this article →
OpenAI's development of GPT-Red reflects a fundamental challenge in AI safety: as language models become more capable and are deployed in increasingly complex real-world scenarios—such as agents that interact with code, websites, and external systems—the scope of potential attacks grows faster than traditional human-led testing can handle. The company's co-creators, Nikhil Kandpal and Dylan Hunn, frame this as a problem of both expanding risk surface and growing blast radius. Rather than scaling up human red-teaming teams indefinitely, OpenAI designed GPT-Red using a self-play training loop, a technique in which an attacker model and defender models iteratively improve against each other. This approach mirrors competitive training dynamics seen in other domains and allows the system to discover novel attack vectors that might not have occurred to human testers.
The concrete results support the value of the method. When GPT-5.6 was tested against the strongest attacks GPT-Red discovered, fewer than 23% succeeded—a dramatic drop from more than 90% success against the earlier GPT-5 released in August last year. This measurable improvement in robustness across versions demonstrates that the red-teaming process yielded actionable defense improvements. Notably, GPT-Red uncovered attack types researchers had not previously encountered, including the fake chain of thought injection, which exploits the model's reasoning process itself rather than only its input handling. The discovery of such novel attacks underscores the value of automated red-teaming at scale.
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
Aranya Inc., a startup founded last year, launched today with $11 million in funding
CBTS Technology Solutions LLC launched Forge Agents, a platform that turns a plain-language job description in…
Phonely Ltd. launched Alma, a large language AI model built for voice agents and trained on over 10 million re…
Imec CEO Patrick Vandenameele said at SEMICON Taiwan 2026 that the Belgian research center is broadening its c…
Alphabet's AI Overviews now reach over 2.5 billion monthly users through Google Search, and its ad business ge…
Sarah O’Connor's book 'We Are Not Machines' explores how mechanization and AI have transformed the workforce…
