Advancing coding in low‑resource programming languages
- ID
- 37165
- Status
- summarized
- Published
- 19 Oct 2026, 8:00 PM
- Fetched
- 20 Oct 2026, 7:57 AM
- Provider
- OpenAI News
- Category
- ai-labs
- Original URL
- https://openai.com/index/low-resource-programming-languages
- Source URL
- https://openai.com/news/rss.xml
Summary
- Score
- 6.0
- Created
- 20 Oct 2026, 7:57 AM
- Tags
- Audience
- developersai_ml_learners
What happened
OpenAI published two new evaluations for coding in low-resource languages: MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch across 20 languages, and a COBOL migration eval that checks whether a model can port existing programs while preserving behavior. It reports GPT-6 Astra hitting 87.4% average test pass rate on MultiLangBench versus 26.8% for GPT-5.6 Sol, and GPT-6.1 Sol reaching 86.6% COBOL migration accuracy at low reasoning effort (97.9% at high effort) versus 5.4% for GPT-5.6 Sol, at 39% lower cost. The post cites COBOL's footprint — over 40% of online banking systems, 80% of in-person card transactions, 95% of ATM transactions — and Fortran's use in weather forecasting and nuclear reactor safety analysis.
Why it matters
The COBOL migration number is the one to interrogate: going from 5.4% to 86.6% on a behavior-preservation eval is the difference between 'useless for legacy porting' and 'worth piloting', and that changes build-vs-buy decisions for anyone touching mainframe-backed banking or payment flows. But these are OpenAI's own evals on its own models with no third-party replication, so treat 97.9% as a ceiling on their test set, not a guarantee on your codebase — the useful move is to run a handful of your own programs through a migration and diff behavior before and after, not to quote the benchmark internally.
Discussion angle
The eval design is the interesting part, not the scores: 'preserve behavior' is the hard requirement in every legacy migration, so ask what a behavior-preserving test even looks like when the original business rules exist only in code and the original developers have left — and whether a pass rate on OpenAI's COBOL corpus tells you anything about the undocumented COBOL sitting in regional bank and telco billing systems.