Black Hat: AI Finds Novel Hacks Only With Human Help
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- James Kettle presented findings at the Black Hat security conference in Las Vegas on Wednesday, concluding that agentic AI is 'minimally capable but extremely limited' in devising new attack paths autonomously, yet extremely powerful when paired with human insight.
- Kettle's experiments, which began in September 2025 using Anthropic's and OpenAI's latest models, led him to discover an entirely new vulnerability class he dubbed Shared-Parser Confusion—rooted in web servers using shared code to process both requests and responses.
- The Shared-Parser Confusion finding is significant because requests to a website are untrusted while responses are trusted, creating 'a major attack surface' that 'potentially spills into a lot of different attack types,' Kettle told WIRED.
- Kettle found that as more powerful AI models debuted over the course of his research, systems generated findings at a rate far surpassing his own, producing notable results 'maybe every two days without me even logging into the system.'
- The human-AI collaboration produced more proven vulnerability examples in a few months than Kettle could likely find in a few years, though the AI could hypothesize but not independently prove the Shared-Parser Confusion attack—Kettle evaluated and confirmed it.
- Kettle scoped his tests narrowly within his own web security expertise to prevent AI from passing off existing research as original, saying few people discuss AI's limits in security 'because there aren't incentives to talk about that angle.'
Why it matters: For defensive security teams evaluating AI-augmented tooling, Kettle's findings are a direct reality check: agentic AI dramatically accelerates proven bug-hunting but cannot independently conceptualize novel attack strategies. The Shared-Parser Confusion vulnerability—described as 'absolutely massive' and potentially spanning many attack types—was only uncovered through human-AI collaboration, meaning expert human oversight remains essential in any AI-driven security workflow.
