Can AI help solve the COBOL code crisis? — SkimNews
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- OpenAI reported that GPT-6.1 Sol reached 87% behavior-preservation accuracy on legacy-code migration at low reasoning effort and 98% at higher effort; at low effort, GPT-6 Astra scored around 86%, GPT-6 Sol 21% and GPT-5.6 Sol 5%.
- COBOL was among the languages OpenAI’s models translated into C++, Java, Kotlin, Python and Rust while preserving software behavior; OpenAI considers a migration successful only if all tests pass.
- Anthropic published a step-by-step code-modernization playbook in February and promoted Claude Code for moving COBOL into Java or Python, but supplied no performance benchmarks; IBM shares nosedived after the announcement.
- IBM says COBOL underpins more than 40% of online banking systems, 80% of in-person credit-card transactions and 95% of ATM transactions, while one rough estimate puts the annual cost of its inefficiencies at roughly $105 billion to U.S. GDP.
- Vadim Zaytsev said AI is not playing a major role in replacing or migrating legacy COBOL systems; the largest system he had seen had 250 million lines of COBOL, and he illustrated the scale by noting that the remaining 1% can be a million lines.
- Arie van Deursen called OpenAI’s evaluation "superficial," saying its simple, well-defined language benchmark was "lightyears" from actual modernization, did not test breaking complex systems into individually replaceable units and fell far short of the harder work of modernization rather than maintenance.
- Jacopo Mauro said formal verification requires a "huge amount of work" to apply to legacy code; Alastair Donaldson said minor behavior differences are unacceptable in many critical systems, making robust old-versus-new workflow testing crucial and human verification essential.
Why it matters: OpenAI’s GPT-6.1 Sol reached 98% behavior preservation in migration tests, a result that matters to banks and government agencies still running COBOL. But translation is not system replacement: experts say integrations, data and behavior checks keep human verification essential, and IBM shares nosedived after Anthropic’s benchmark-free modernization pitch.
Ask SkimNews
