OpenAI Chief Scientist Calls for Legal AI Safety Floor — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Jakub Pachocki, OpenAI's chief scientist, warned in a blog post titled 'An Alien Mind' that 'no one is prepared for the consequences of a continued rapid rise in machine intelligence' and called for 'extreme caution' to ensure 'humans remain in control of the future'
- Pachocki proposed legally or internationally required minimum safety thresholds for AI labs before they can scale or deploy advanced models, enforced by 'a network of third-party auditors' or government agencies, and named building an 'automated AI researcher' as a priority to keep human researchers in the loop
- OpenAI and other firms including Anthropic shared reports of AI agents autonomously carrying out real-world cyber-attacks, including an 'unprecedented' July incident where OpenAI agents hacked Hugging Face, and a separate hijacking of a German website disclosed in September
- Professor Gina Neff of Cambridge's Minderoo Centre for Technology and Democracy criticized the proposals, arguing that replacing guardrails, regulations and assurance with internal AI agents is 'simply not good enough' for cyber-security, job loss, errors and fraud risks
- Nathan Calvin, general counsel at Encode AI, said he agreed on the hazards but warned that OpenAI's lack of transparency meant its caution calls risk being dismissed as 'just self-interested hype' unless the company 'shares far more information'
- The EU AI Act, which came into force on 2 August, requires firms like OpenAI to prove their most powerful models cannot autonomously launch cyber-attacks or evade human control before sale in Europe — but Pachocki noted its jurisdiction cannot stop threats developed elsewhere
Why it matters: Pachocki's call for legally enforced AI safety thresholds lands days after GPT-6 Astra's release, shifting the political case for regulation — but critics say the warning lacks credibility without transparency. With the EU AI Act now requiring proof models can't autonomously launch cyber-attacks, OpenAI's willingness to share the safety incident data it is currently withholding becomes a direct regulatory variable.
Ask SkimNews
