Anthropic Researcher Quits Citing Extinction-Level AI Risk — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Jacob Coxon resigned from Anthropic, accusing the company and OpenAI of 'racing straight to self-improving superintelligence and gambling with our lives' despite believing AI 'could kill us all by the end of the decade.'
- Evan Hubinger, who leads one of Anthropic's AI safety teams, agreed with Coxon's assessment and personally estimated the chance of AI killing all humans at greater than 1 in 10 'within the next decade.'
- Hubinger also acknowledged self-improving AI 'is happening faster than we thought' and conceded Anthropic does 'not yet have a plan' to ensure advanced AI safety and 'are not clearly on track to' develop one.
- Coxon previously trained AI systems at OpenAI before joining Anthropic, and his departure marks one of the most high-profile exits from a company founded by former OpenAI staff who left over safety concerns.
- The exchange comes as AI companies prepare for anticipated IPOs while managing fallout from 'rogue agent incidents' and warnings about the monitorability of frontier models.
Why it matters: Anthropic safety team lead Evan Hubinger publicly conceding the company 'not yet has a plan' to control superintelligent AI while the company pursues an IPO puts public-market investors in the position of pricing human extinction risk into a stock valuation — and Coxon's exit from a firm founded by OpenAI safety defectors shows the talent drain over these concerns is accelerating, not stabilizing.
Ask SkimNews
