Anthropic Insiders Warn AI Could Kill Humanity — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Jacob Coxon quit Anthropic last week and warned on X that "the people building AI earnestly believe that it could kill us all by the end of the decade," with his experience and motivation unconfirmed
- Anthropic's head of alignment science publicly agreed with Coxon and placed the odds of AI killing all humans at more than 10 percent
- A 2024 survey of nearly 3,000 published AI researchers found more than half thought the chance of human extinction or permanent severe disempowerment — the so-called p(doom) — was at least 10 percent
- Hugging Face CEO Clement Delangue dismissed the framing, writing that asking Coxon about AI extinction risk is "like asking your AC guy about climate change," and Anthropic initially refused to release its model Mythos after it was found adept at hacking software
- Anthropic co-founder Dario Amodei called for "pacing the frontier" — slowing AI development for safety — but conceded US firms won't slow because their lead over China is limited and needs to be kept "as large as possible"
- Former Astronomer Royal Martin Rees told Sky News on 14 September that a rogue AI could disrupt infrastructure worldwide, depriving cities of energy, food, and water simultaneously and making civilizational recovery very difficult
- President Donald Trump said this week: "We're leading China in AI... and frankly, I want to keep it that way, because whoever wins AI wins" — underscoring the competitive pressure against any coordinated slowdown despite over 1,300 industry insiders backing "pacing the frontier"
Why it matters: Anthropic is now visibly split between alarmists (Coxon, its alignment chief citing 10%+ extinction odds) and pragmatists (co-founder Amodei pushing a measured slowdown), while competitors call the framing self-serving and US-China competition — which Trump framed as existential this week — makes any coordinated slowdown near-impossible. The 10%+ p(doom) consensus among nearly 3,000 researchers shows the warnings aren't fringe, but the article's own sourcing suggests the talk will outpace the action.
Ask SkimNews
