AI Weekly Malaysia

Nvidia announced NeMo Switchyard, a software proxy that routes inference requests to different models based on cost, latency, or quality, claiming a 74% cost reduction versus using Claude Opus 4.8 alone with roughly a six-point accuracy tradeoff. Alongside it, Nvidia released Nemotron 3.5-30B-A3B-Lightning, a 30B-parameter MoE open-weights model for low-latency general use, and Nemotron Parse, a 1B-parameter model specialized in extracting context from PDFs including charts and tables.

Why: If you're paying for frontier-model API calls on every request, model routing lets you send trivial subtasks (title generation, summarization, PDF parsing) to cheaper or self-hosted models and reserve expensive models for the prompts that actually need them. The key insight is optimizing for completion cost, not per-token price—a model at 1/10th the token price that needs 10x tokens isn't cheaper. Evaluate whether a routing layer fits your stack before committing to a single provider.

NVIDIA announced Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts open model designed for high-volume specialized tasks within multi-agent systems, claiming up to 4x faster output speed and 30% faster agentic task completion versus peers in its class. They also released NeMo Switchyard, an open source routing library that directs each request to the most suitable model across a mix of open, proprietary, and NVIDIA models without requiring application rewrites.

Why: If you are building multi-agent systems, the NeMo Switchyard routing library is the practically actionable piece here — it lets you mix models (open, proprietary, NVIDIA) behind a single router without rewriting your app, which is a real architecture decision worth evaluating. The 30B MoE model itself is a vendor claim with no independent benchmarks yet, so treat the speed numbers as unverified until third-party testing appears.