AI Weekly Malaysia

Summaries

Short AI and tech summaries with source links, signal scores, and why each update matters for builders, founders, and Malaysian tech workers.

Reset

Showing 1-5 of 5 results

DateProviderScoreSummary
18 Nov 2026, 8:00 AMHugging Face Blog7.8 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

Tarek Ziadé's Hugging Face post experiments with running a 180B-parameter MoE model on a 32GB Mac by streaming experts from SSD, reporting that this approach gets within 6% of Strata on token generation. The post covers the author's setup—two RTX 4090s on Windows 11 plus an M5 MacBook with 32GB unified memory—and prior tests running Qwen 3.8 27B via llama.app and Qwen3.8-Flash-Next via Strata. It notes a 32GB M4 result of 13.5 tok/s, too slow for coding agents, and frames expert offload as a way to run MoE models that do not fit in unified memory.

Why: If you run local coding agents, use the reported numbers as a decision threshold: a 180B MoE can fit with SSD expert streaming, but 13.5 tok/s on a 32GB M4 is not interactive for agent loops, so benchmark your own model and engine before treating extra VRAM as the only fix. The 6% gap to Strata also makes offload engines worth testing before upgrading a 32GB Mac.

16 Nov 2026, 8:00 AMHugging Face Blog7.0 Hugging Face and OpenRouter join forces to make open-weight models easy to run

Hugging Face has replaced its own Inference Providers router with OpenRouter's, so requests to language models on the Hub are now routed across 80+ providers based on live price, speed and reliability, with automatic retry to another provider on failure — code stays the same, and the same Hugging Face token also covers Storage Buckets, Jobs and Inference Endpoints. The post cites OpenRouter data showing open-weight models went from 26.8% of tokens served in October 2025 to 71.7% by end-September 2026 (peaking at 77.5% in July), with weekly open-weight volume growing over 60x versus roughly 10x for closed models. Average prices quoted are $0.28/$1.62 per million input/output tokens for open-weight versus $1.27/$6.36 for closed, while the top open-weight model on the Artificial Analysis Intelligence Index (Kimi K3.1 Pro, 46) still trails the top closed one (Claude Fable 5.2, 65) at about a seventh of cost per task ($3.68 vs $26.54).

Why: A routing change happened underneath you: if you call models through Hugging Face Inference Providers, which upstream provider serves each request is now decided by OpenRouter on price/speed/reliability, with failover to a different provider — so pin down whether you need deterministic provider choice, consistent latency from your region, or a specific data-processing location before you ship. The pricing gap is the concrete decision: $0.28/$1.62 vs $1.27/$6.36 per million tokens, and $3.68 vs $26.54 per task on the hardest benchmark, means the 46-vs-65 intelligence gap is worth paying for only on tasks where you can measure the failure — route the cheap tier by default and escalate deliberately.

16 Nov 2026, 11:24 PMHacker News6.0 Nvidia was in trouble before the Jensen Huang mess

WIRED reports that Nvidia's Jensen Huang has been ousted as CEO, with CFO Colette Kress now running the company, at a moment when Nvidia's grip on AI accelerators is loosening: it still holds roughly 80% of AI accelerator sales per Silicon Analysis, but AMD's share has climbed from about 2% in 2022 to 7-10% in 2026, and Google, Amazon and Microsoft are all expanding custom silicon. The piece argues the shift toward inference for AI agents and assistants favours hyperscalers and startups over Nvidia, which has less of an edge there; Nvidia acquired inference-chip startup Groq in December 2025, while AMD bought Taalas, whose approach burns model weights into silicon. It also notes these alternative chips require substantial code rewrites to work with cutting-edge models, and the supplied excerpt cuts off mid-sentence, so later claims in the article are not visible here.

Why: The concrete decision this raises is code portability: the article states that using AMD, Cerebras or weight-burned-in-silicon chips 'requires rewriting a lot of code,' so if you are building agent or inference workloads, keeping a model-serving abstraction between your app and CUDA-specific kernels is what lets you chase cheaper inference later. Second, treat vendor market-share numbers (80% Nvidia, 7-10% AMD) as a signal to re-quote your GPU/inference costs rather than a reason to change architecture this week — and note the article gives no pricing or benchmark figures to act on.

16 Nov 2026, 9:00 PMTom's Hardware5.5 Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second

Tom's Hardware reports that a jailbroken PS4 can run a 9-billion-parameter AI model at up to 43 tokens per second using an open-source project called Garlic LLM, which bypasses standard drivers to communicate directly with the console's 2013-era GPU. The supplied page text is almost entirely site navigation and subscription boilerplate, so the model used, quantization, memory footprint, and setup steps are not stated. Treat this as a headline-level report only.

Why: The one usable signal is the ratio: 43 tok/s for a 9B model on a decade-old console GPU implies the software stack — drivers and kernels — is often the binding constraint, not raw silicon. That is relevant if you are sizing cheap or used hardware for local inference instead of buying new GPUs. But with no memory footprint, quantization, or context-length numbers in the text, you cannot yet judge whether the result is reproducible or comparable to a normal llama.cpp setup, so do not plan a build around it.

17 Nov 2026, 11:00 PMLatent Space5.0 Cloudflare Joins the Jevolution and Labels Itself a Neolab

At the AI Engineer Code Summit, Michelle Chen — who runs Cloudflare's AI Platform on the product side — pitched Cloudflare as a "neolab" and introduced two System One models: Clef, a decision model she said was directly inspired by TypeSafe AI's Jev model, and Clairic, an "extract" model. In a follow-up Latent Space interview she said Cloudflare has "entered this Pandora's box" and is experimenting with reasoning models and other LLMs beyond System One models. The excerpt cuts off mid-sentence and gives no benchmarks, pricing, model sizes, or availability dates.

Why: There is nothing here you can act on yet: no endpoint, price, context window, or eval number is given, so nobody should plan a migration or rewrite an extraction pipeline around Clef or Clairic. The one concrete decision signal is architectural — Cloudflare is splitting "System One" fast decision/extraction models from larger reasoning LLMs, which is the same split you'd make if you were routing cheap classification at the edge and reserving big-model calls for the hard cases.

Top