Anthropic's AI Models Now Improve Twice as Fast — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- METR said after an 11-week investigation inside Anthropic that the lab's models are improving roughly twice as fast as a year ago — about two years of historical progress in 12 months — that researchers are confident this is not simply more training compute, and that they found no clear signs of diminishing returns or bottlenecks.
- Anthropic employees using the company's most capable internal model self-reported roughly 12 times their normal productivity, about double what staff had reported a few months earlier while using Mythos 5; METR attributed most of the acceleration as of October to AI, while noting that the gains currently come from AI assisting human researchers, with signs that fully autonomous contributions will play a larger role soon.
- Model 2 is one of three unreleased Anthropic models METR says finished training over the past six months — Model 2 in July, Model 3 in August–September and Model 4 in September–October — and METR also identified an unreleased model dating back to 2022.
- Sam McCandlish acknowledged that Anthropic redacted most of the evidence METR found most informative from a roughly 100-page full report, called that decision the "right call" because disclosing what drives the speedup is commercially sensitive and would hand rivals a roadmap to go faster, and said Anthropic will publish its own report on AI progress in the coming weeks.
- METR said Anthropic granted neither unrestricted nor the "employee-like" access promised to other safety researchers under a separate program, denied some data requests and kept the information pipeline narrow, so the public must rely on METR's judgment more than would be ideal.
- Apollo Research's Marius Hobbhahn said the report shows AI-enabled research and development running even faster than the AI 2027 scenario's forecast of a key breakthrough toward self-improvement in early 2027, adding that the pace is broadly in line with field expectations, perhaps a little faster.
- Ryan Greenblatt, a METR researcher, said he "was seriously wrong about RSI speed up; I previously expected much less in 2026," while GovAI's Alan Chan said the findings should prompt skeptics to shorten their timelines for systems that fully automate tasks like software engineering.
Why it matters: Anthropic shields commercially sensitive details while outsiders lose the evidence needed to check its safety claims: it redacted most of METR's evidence and denied some data requests, even as METR found models improving twice as fast and three unreleased models trained from July to October. Verification now rests on METR's judgment rather than the underlying record.
Ask SkimNews
