Summaries
Short AI and tech summaries with source links, signal scores, and why each update matters for builders, founders, and Malaysian tech workers.
Showing 1-2 of 2 results
| Date | Provider | Score | Summary |
|---|---|---|---|
| 18 Nov 2026, 8:00 AM | Hugging Face Blog | 7.8 | 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers
Tarek Ziadé's Hugging Face post experiments with running a 180B-parameter MoE model on a 32GB Mac by streaming experts from SSD, reporting that this approach gets within 6% of Strata on token generation. The post covers the author's setup—two RTX 4090s on Windows 11 plus an M5 MacBook with 32GB unified memory—and prior tests running Qwen 3.8 27B via llama.app and Qwen3.8-Flash-Next via Strata. It notes a 32GB M4 result of 13.5 tok/s, too slow for coding agents, and frames expert offload as a way to run MoE models that do not fit in unified memory. Why: If you run local coding agents, use the reported numbers as a decision threshold: a 180B MoE can fit with SSD expert streaming, but 13.5 tok/s on a 32GB M4 is not interactive for agent loops, so benchmark your own model and engine before treating extra VRAM as the only fix. The 6% gap to Strata also makes offload engines worth testing before upgrading a 32GB Mac. |
| 16 Nov 2026, 9:00 PM | Tom's Hardware | 5.5 | Jailbroken PS4 runs 9-billion-parameter AI model at up to 43 tokens per second
Tom's Hardware reports that a jailbroken PS4 can run a 9-billion-parameter AI model at up to 43 tokens per second using an open-source project called Garlic LLM, which bypasses standard drivers to communicate directly with the console's 2013-era GPU. The supplied page text is almost entirely site navigation and subscription boilerplate, so the model used, quantization, memory footprint, and setup steps are not stated. Treat this as a headline-level report only. Why: The one usable signal is the ratio: 43 tok/s for a 9B model on a decade-old console GPU implies the software stack — drivers and kernels — is often the binding constraint, not raw silicon. That is relevant if you are sizing cheap or used hardware for local inference instead of buying new GPUs. But with no memory footprint, quantization, or context-length numbers in the text, you cannot yet judge whether the result is reproducible or comparable to a normal llama.cpp setup, so do not plan a build around it. |