What happened
OpenAI said GPT-6.1 Sol scored 86.6% on its COBOL migration benchmark at a low reasoning setting, versus 5.4% for GPT-5.6 Sol, at 61% of that model's API cost; higher reasoning reached 97.9%.
Why it matters
A near-perfect score means AI can now largely automate the mechanical translation of legacy COBOL, but OpenAI still requires expert review and business-rule testing before production use, so the tool speeds the work without removing human oversight.
WHO IT HITSIT teams maintaining COBOL-based banking and ATM systems stand to gain a faster first-pass translation tool, though the body notes that production deployment still requires expert review and testing against business rules.
Summaries like this, every morning.
OpenAI's announcement centers on two benchmarks it built to measure how well its models handle programming languages with sparse training resources. The first, MultiLangBench, asks models to build a SQLite-compatible database engine from scratch in 20 languages; there, GPT-6.1 Sol reached an average test pass rate of 87.4%, up from 26.8% for GPT-5.6 Sol, while COBOL remained the hardest language at 72.1%, up from 10.9%.
The second, the COBOL migration evaluation, tests whether a model can rewrite an existing COBOL program into another language while preserving behavior. Because COBOL is used in over 40% of online banking systems and 95% of ATM transactions, per OpenAI, the company set a strict bar: a migration counts as successful only if all outputs match byte-for-byte on a non-public test. The results show a sharp jump from GPT-5.6 Sol's 5.4% accuracy to GPT-6.1 Sol's 86.6% at low reasoning, with 97.9% at higher reasoning, nearing the benchmark's ceiling.
OpenAI frames the cost angle as well: GPT-6.1 Sol's API cost is 61% of GPT-5.6 Sol's. Even so, the company cautions that production use of AI-migrated code still requires expert review and testing against business rules and dependencies, suggesting the tool is positioned as an accelerator rather than a replacement for human verification.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Moonshot AI released Kimi K3.1 Pro, a post-trained upgrade to Kimi K3 that scores 55 points on the Artificial…
Induction AI's benchmark trials shared a prompt cache, so later trials read earlier trials' opening tokens at…
Magic Moment began providing an MCP server for its Magic Moment Playbook sales platform, so sales records tied…
Ahead of Oracle AI World on Oct
World, co-founded by Sam Altman, this week announced tools that let people link their proof of humanity to an…
In commentary on the unit-distance proof, Jacob Tsimerman, recently hired by OpenAI, said AI's edge is that it…
