AI Weekly Malaysia

OpenAI has made its Decisions API generally available to all customers, a fast, cheap 'super classifier' model that picks among options and returns a confidence rating rather than generating text. It is positioned against TypeSafe's Jev model, which created the category and helped make TypeSafe a $7.6 billion company. OpenAI's Nikunj Handa says the team reconfigured ChatGPT Luna rather than building a purpose-calibrated decision model, cutting inference from 1-3 seconds to 100-180 milliseconds; the API was announced at OpenAI's September 29 developer day, gained voice and visual modalities in the public version, and inference is now geographically targeted for lower latency, with Box, Datadog, Ramp and Speak as early users.

Why: If any of your LLM calls are really just routing, triage, moderation or 'pick one of these' choices, this is a concrete cost and latency swap: 100-180ms versus 1-3 seconds, on plain Luna rather than a specially trained classifier. Speak's CTO Andrew Hsu reports cutting time between actions from 1.4 seconds, so the honest test is to take your slowest classification prompt, run it against the Decisions API, and compare p95 latency and per-call cost against your current model. The article says inference is geographically targeted but does not name which regions, so measure latency from your own deployment location before committing to a migration.