AI Weekly Malaysia

Cloudflare launched Auto Router in public beta through AI Gateway: set your model to `cloudflare/auto` and each request is routed to a model judged 'capable enough' for the task instead of a manually chosen frontier model. Cloudflare reports up to 30% cost savings from its own internal use through its OpenCode harness and Cloudflare OS agent harness, versus using only frontier models it names as OpenAI Sol and Anthropic Claude Opus. The published post is truncated right where the results section begins, so the full measurement details are not in the text provided.

Why: If you already route LLM calls through Cloudflare AI Gateway, this is a one-line change (`cloudflare/auto`) you can A/B against your current model choice, which matters most for teams whose non-technical workflows are burning Opus-class tokens on tasks like email or thread summarisation. Treat the 30% as a vendor internal figure, not a benchmark: run it on your own traffic and compare quality on your hardest tasks before making it the default, because routing decisions you cannot see are also routing decisions you cannot easily debug.

At OpenAI's Dev Day, Sam Altman revealed a limited-preview "Decisions API" that gives the Luna model a predefined set of options to pick between — image categories, agent behaviors — and returns that choice fast. It looks like a clone of Jev, a model released earlier in September by TypeSafe AI that acts as an LLM-based classifier outputting probabilities over a fixed choice set cheaply and at high speed. TypeSafe CEO Diogo Almeida joked on X about "clone wars" and said OpenAI's interest could signal that building in a "System One" (fast, intuitive) way is the future; TechCrunch notes it's unclear how close the two products are, and hasn't yet spotted developers using Decisions API.

Why: If you're paying per-token for agent routing or classification steps, the pitch here is real: Jev-style endpoints replace an open-ended generation call with a probability over a fixed list of choices, which developers using Jev reportedly found faster and cheaper than augmenting an LLM. OpenAI's version is limited preview with no public developer reports, so don't re-architect on it yet — but it's worth benchmarking Jev on your own routing/classification workload now, since that one is already shipping.