OpenAI GPT-6.1 Sol achieves high accuracy in legacy code migration tasks
New benchmarks show OpenAI's latest models significantly improve at handling COBOL and Fortran, aiding modernization efforts.
Key points
- GPT-6.1 Sol reaches 86.6% accuracy in low-effort COBOL migration tests, up from 5.4% for the previous model.
- The new model scored 87.4% on MultiLangBench, a significant increase from 26.8% for GPT-5.6 Sol.
- OpenAI highlights improvements in COBOL, Fortran, Visual Basic, and OCaml for critical infrastructure support.
OpenAI announced that its GPT-6.1 Sol model demonstrates substantial improvements in writing and migrating legacy code, according to a company blog post. The technology firm stated that newer models are significantly more capable of handling older programming languages that underpin essential global services.
The company measured this progress using MultiLangBench, a test requiring models to build a SQLite-compatible database engine from scratch across twenty programming languages. GPT-6 Astra achieved an average pass rate of 87.4%, a major increase from the 26.8% rate recorded for GPT-5.6 Sol in the same benchmark.
Performance improvements in legacy languages
OpenAI reported specific gains in languages with limited training data, such as COBOL and Fortran. In COBOL tasks, the pass rate jumped from 10.9% to 72.1%, while Fortran improved from 21.4% to 83.5%. Visual Basic scores rose from 27.5% to 89.4%, and OCaml increased from 25.6% to 85.8%.
These languages support critical infrastructure despite having less available code and documentation for models to learn from compared to modern languages like Python and JavaScript. OpenAI noted that COBOL supports over 40% of online banking systems and 95% of ATM transactions.
Migration accuracy and cost efficiency
The company also tested the ability to migrate COBOL programs to modern languages like C++, Java, Kotlin, Python, or Rust without altering behavior. GPT-6.1 Sol achieved 86.6% accuracy at low reasoning effort, compared to just 5.4% for GPT-5.6 Sol at the same setting.
This performance was achieved at only 61% of the API cost of the previous model. At higher reasoning efforts, the accuracy reached 97.9%, bringing the evaluation close to saturation. OpenAI emphasized that production migrations would still require expert review and testing.
What should you do?
Security and technology professionals should monitor these developments as legacy system modernization becomes more accessible. Organizations relying on older codebases may need to reassess their migration strategies in light of these improved AI capabilities.
What to do and how to stay safe: OpenAI
- Review internal policies regarding AI-assisted code generation and migration to ensure compliance with security standards.
- Maintain rigorous manual review processes for any code migrated by AI tools, as production use still requires expert validation.
- Audit legacy systems for vulnerabilities that might be exposed during automated modernization efforts.
- Monitor vendor announcements for updates on AI model capabilities relevant to your specific technology stack.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How much did COBOL migration accuracy improve in the new model?
GPT-6.1 Sol achieved 86.6% accuracy at low reasoning effort, up from 5.4% for GPT-5.6 Sol.
What is MultiLangBench and what were the results?
It is a test to build a SQLite engine in 20 languages; GPT-6 Astra scored 87.4%, up from 26.8% for GPT-5.6.
Which legacy languages saw the most improvement?
Significant improvements were noted in COBOL, Fortran, Visual Basic, and OCaml, with pass rates increasing substantially.
