OpenAI Uses AI Red Team to Strengthen GPT-5.6 Against Prompt Injection Attacks

OpenAI has introduced GPT-Red, an automated AI system designed to find security vulnerabilities in its language models.

GPT-Red takes its name from cybersecurity red teaming, which is the practice of deliberately attempting to break a system to identify weaknesses before attackers can exploit them.

In a post on Wednesday, OpenAI said the tool helped make GPT-5.6 more resistant to prompt injection attacks before deployment.

“As model capabilities grow, safety and alignment must scale with them,” OpenAI wrote on X. “Red-teaming is essential, but today’s approaches are difficult to scale, creating a critical bottleneck. GPT‑Red is one way we’re addressing it.”

According to OpenAI, GPT-Red was trained through self-play reinforcement learning, generating progressively stronger prompt injection attacks while defender models learned to resist them. The company said those attacks were incorporated into GPT-5.6’s training process, reporting that GPT-Red succeeded in 84% of internal evaluation scenarios, compared with 13% for human red teamers in the same tests.

“GPT‑Red learns through adversarial self-play, where its goal is to prompt inject a variety of challenging defender models,” OpenAI wrote. “Every successful attack that GPT-Red finds is used to improve these defenders, pushing GPT‑Red to continuously find broader and more complex failures.”

In one case study, OpenAI said the system manipulated an autonomous vending machine agent into lowering prices, ordering discounted inventory, and canceling another customer’s order before the vulnerabilities were disclosed and addressed.

GPT-Red follows years of cybersecurity efforts by OpenAI after the public launch of ChatGPT.

In 2023, the company launched its OpenAI Red Teaming Network, recruiting outside cybersecurity researchers and domain experts to probe ChatGPT and other models for security flaws before release. GPT-Red expands on that effort by automating much of the process, using an AI model to generate prompt injection attacks and other adversarial tests at a scale that would be difficult for human researchers alone.

OpenAI’s announcement reflects a broader shift toward using AI to secure AI.

Earlier this month, the Ethereum Foundation said it had deployed AI agents to red-team critical network infrastructure, uncovering a vulnerability in software used by Ethereum consensus clients. Researchers said AI agents can search larger codebases than humans, but the challenge has shifted from finding potential bugs to proving which ones are exploitable.

According to OpenAI, GPT-Red will remain an internal tool because it contains intentionally developed offensive capabilities.

“We believe with GPT-Red that we have started to unlock a similar flywheel for safety, where today’s models can be used to make tomorrow’s models more robust, aligned, and trustworthy,” they said.