AI’s recursive self-improvement might not come so quickly after all
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- Princeton researchers Peter Kirgis and Sayash Kapoor developed a new "shadow evaluation" method testing AI agents on questions from unpublished NeurIPS 2026 papers, finding they lack the judgment and taste needed for open-ended research.
- Claude Opus 4.8 running on OpenClaw was given 6 days, $3,000 in Anthropic API credits, a GPU budget, and web access — both papers it produced were rejected by the original authors as nowhere close to top-conference quality.
- Agents excelled at engineering tasks (literature reviews, hundreds of experiments) but committed to unpromising approaches too quickly, rejected hypotheses on very limited data, and couldn't fundamentally rethink failing methods.
- Notably, agents did NOT engage in reward hacking — subagent hallucinations and misrepresentations were caught by the orchestrator AI, a positive finding the researchers flagged.
- Anthropic cofounder Jack Clark wrote in his Import AI newsletter that internal findings echo the study, calling AI's lack of "intuitive creativity" a "bearish signal on short recursive self-improvement timelines."
- Industry hype now contrasts with the results: Anthropic's June "When AI Builds Itself" blog post and OpenAI's July claim that GPT-5.6 Sol saved researchers weeks by helping post-train a smaller model.
- Kapoor's team is next testing Anthropic's Mythos model, which launched in April and is now restricted by Trump administration safety requirements to approved organizations only.
Why it matters: The study gives empirical weight to what Anthropic cofounder Jack Clark called a 'bearish signal' on short self-improvement timelines. With narrow-task gains failing to substitute for the creative leaps behind breakthroughs like transformers — Kapoor's stated 'trillion-dollar question' — labs betting on near-term recursive AI lose a key empirical anchor, and the gap between product announcements and open-ended capability becomes harder to wave away.
Ask SkimNews
