Most quantum computing papers fail to replicate — SkimNews
Get the Health newsletter
Daily health & science — research, biotech, public health, the studies worth knowing. Free.
- Mauerer led a two-part analysis at the Technical University of Applied Sciences Regensburg, manually evaluating 127 quantum computing papers from the past five years, then automating the evaluation across 4,966 papers using five reproducibility criteria.
- The findings showed only 24.4% of the 127 manually reviewed papers provided code that could be run, and 64.5% of that code failed to execute; the broader automated analysis found just 26.8% of 4,966 papers provided enough information to attempt replication.
- The research team attributed the problem partly to unusually variable, still-maturing quantum hardware, noting that cloud-accessible quantum machines differ fundamentally from conventional computers where code can run abstractly across machines.
- William Zeng at the Unitary Foundation said he wasn't surprised and expects AI coding agents and community maintenance efforts to improve reproducibility going forward.
- Fred Chong at the University of Chicago countered that the field is in its infancy and innovation may currently be more important than reproducible infrastructure.
- Ralf Ramsauer, a team member at Regensburg, said the research community has responded positively and the study offers a template for creating reproducibility packages.
Why it matters: Quantum computing researchers are pursuing commercial applications from molecule simulation to airline logistics, but most published results cannot be replicated — and the situation has not improved since a similar study four to five years ago. The credibility gap puts the field at odds with conventional computer science standards, where reproducibility is required for routine use.
Ask SkimNews
