OpenAI、自動攻撃AI「GPT-Red」発表 人間13％に対し84％成功

On July 15, OpenAI unveiled GPT-Red, an automated red-teaming AI designed to discover vulnerabilities in its own models before deployment. Red-teaming is a security validation technique where testers adopt an attacker's perspective to probe for weaknesses. Human red-teamers have historically been the standard, but they face two critical constraints: the process is time-consuming and difficult to scale, and it cannot generate the volume and variety of attack data required to meaningfully improve model robustness through training.

GPT-Red mimics human red-teamers by iteratively sending prompts to target models, observing responses, and refining attacks. The system was trained using self-play reinforcement learning, where an attacking GPT-Red model trains simultaneously against multiple defensive language models. The attack side earns rewards by successfully executing prompt injections (embedding malicious commands within instructions), while the defense side earns rewards by resisting attacks and completing its intended task. As defenders strengthen, attackers learn to devise more powerful and diverse attacks—creating a virtuous cycle. OpenAI invested computation resources equivalent to its largest post-training efforts purely into this safety work.

The performance gains are substantial. In testing on indirect prompt injection scenarios using previously unseen scenarios, human red-teamers achieved a 13% attack success rate, while GPT-Red succeeded in 84% of cases. In a real-world test, OpenAI tasked GPT-Red with attacking an AI vending machine agent running in its offices (similar to Anthropic's experimental Project Vend). GPT-Red achieved all three malicious objectives: modifying high-priced items to cost $0.50, ordering items over $100 and reselling them for $0.50, and canceling other customers' orders. These vulnerabilities have been disclosed and are now undergoing remediation testing.

GPT-Red's primary role is to feed discovered attacks into product model training to increase robustness. The latest model, GPT-5.6 Sol, benefited from adversarial training using GPT-Red attacks. On the most difficult direct prompt injection benchmark, GPT-5.6 Sol's failure rate dropped to one-sixth of the best product model from four months prior. Against direct prompt injection attacks from GPT-Red itself, the failure rate is just 0.05%. A novel attack type discovered by early GPT-Red versions—called "fake Chain-of-Thought attacks"—had a success rate exceeding 95% against GPT-5.1 but now falls below 10% on GPT-5.6 Sol. OpenAI stresses that this safety improvement came without sacrificing capability: the model retained its normal abilities while gaining resistance to malicious instructions.

To prevent misuse, GPT-Red remains an internal model, strictly separated from deployed systems, ensuring its trained attack capabilities do not reach bad actors. OpenAI framed the effort as initiating a "safety flywheel," using today's models to make tomorrow's models more robust and trustworthy. The company plans to expand compute and training data for stronger future versions of GPT-Red. A peer-reviewed paper detailing the work is scheduled for release later this week.

OpenAI's announcement of GPT-Red reflects a shift in how AI companies approach safety validation. Traditional red-teaming by human experts is labor-intensive and cannot generate the volume and variety of attack data needed to significantly improve model robustness. By automating this process with an AI opponent trained via self-play, OpenAI aims to create what it calls a "safety flywheel"—a cycle where today's models train tomorrow's models to be more resilient. The company invested computation resources equivalent to its largest post-training efforts solely into safety, underscoring the resource commitment required.

The real-world test on an AI vending machine agent in OpenAI offices demonstrated the practical stakes: GPT-Red successfully achieved three malicious goals (price manipulation, order fraud, and order cancellation) that previous defenses had not caught. This discovery then fed back into training GPT-5.6 Sol, which now exhibits dramatically lower failure rates on prompt injection benchmarks. The company emphasizes that this robustness gain came without degrading model capability—a critical distinction, since overly defensive models often become less useful. By keeping GPT-Red internal and planning to publish a peer-reviewed paper, OpenAI is attempting to balance transparency with security, allowing the research community to learn from the work while preventing attack tools from proliferating.

No comments yet. Be the first to share your thoughts!