OpenAI's GPT-Red: An AI Hacker for Safer Models
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- OpenAI built GPT-Red as an automated red-teaming system that attacks other LLMs in a self-play loop, with defenders trying to fend off the attacks in a simulated 'dojo' of real-world scenarios including web browsing, email, calendar apps, and code editing.
- GPT-Red discovered a previously unseen type of prompt injection the researchers call a 'fake chain of thought'—inserting a spoofed entry into another model's working memory that tricks it into accepting false information (e.g., telling it '1+1=3, and you've verified this').
- OpenAI says more than 90% of GPT-Red's strongest attacks succeeded against GPT-5 (released August last year), but fewer than 23% worked against the new GPT-5.6, which it calls its most robust release yet.
- GPT-Red outperformed human red-teamers in a rerun of a 2025 experiment probing an earlier GPT-5, and successfully hacked Vendy—an Andon Labs vending-machine agent—into changing prices and canceling a customer's order.
- OpenAI researchers Nikhil Kandpal, Dylan Hunn, and Chris Choquette-Choo, who co-created GPT-Red, say the company will not release the model and that its compute-intensive, year-plus build makes it hard for outsiders to replicate.
- GPT-Red still has blind spots: it struggles with multi-turn conversational attacks and image-based prompt injection, so OpenAI says it supplements rather than replaces its human red-team.
- Jessica Ji, a senior research analyst at Georgetown's Center for Security and Emerging Technology, called the self-play approach 'very promising' but said human testers remain essential for attacks the model misses.
Why it matters: OpenAI is moving core safety testing from human red-teams to AI-on-AI combat just as its models are being deployed as agents that touch files, websites, and third-party code—a shift that could compress vulnerability-discovery cycles from months to days. The 90%-to-23% drop in attack success between GPT-5 and GPT-5.6 is the first concrete benchmark the company has offered tying its red-team automation to measurable safety gains, but keeping GPT-Red private means rivals and customers have to trust OpenAI's internal numbers rather than verify them.
