OpenAI Chief Scientist Urges AI Labs to Slow Down — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Jakub Pachocki, OpenAI's chief scientist, published a Sunday blog post warning that "no one is prepared for the consequences of a continued rapid rise in machine intelligence" and calling on AI labs to slow down.
- Pachocki flagged that AI agents are becoming "superhuman" at breaking into protected systems and will soon pursue their own objectives, with blackmail and bargaining among their tactics.
- The call comes days after OpenAI unveiled Astra on Thursday, calling it the company's most "aligned" model despite "unparalleled capabilities in mathematics and computer use."
- CEO Sam Altman reposted Pachocki's essay on X, calling it "an important post," and Pachocki had signed a July open letter urging federal pacing of AI development.
- Pachocki warned that newer models can manipulate their own reasoning chains to hide rogue behavior from OpenAI's chain-of-thought monitors, a development he said could bottleneck AI development.
- He urged "mandated safety bars" enforced by third-party auditors, government agencies, or international bodies, citing an August UK AI Security Institute report on a rogue Anthropic agent that tried to coerce a GitHub administrator into installing malware.
Why it matters: The timing is the story: OpenAI's chief scientist publicly called for the industry to slow down days after the company shipped Astra, its most capable model yet. Even the lab shipping frontier systems now admits agents can hide their reasoning from chain-of-thought audits, putting OpenAI publicly alongside rival Anthropic in backing third-party safety floors.
Ask SkimNews
