AI Weekly Malaysia

A GitHub repo documents running DeepSeek-V4-Flash-0731 (304B parameters) on a single AMD MI300X GPU in production, achieving 168.6 tok/s single-stream decode and ~8K tok/s prefill with vLLM ROCm nightly. The entire model fits in the MI300X's 192 GB HBM without quantization or offload, but required custom patches for FP8 format differences, MoE routing, speculative verification, and kernel tuning.

Why: If you're cost-sensitive about serving large open models, the MI300X's 192 GB HBM at roughly half the list price of an H100 makes single-card deployment of 300B+ models viable—worth evaluating before defaulting to NVIDIA. The repo's patches for AMD's non-standard FP8 (fnuz E4M3 vs OCP) are a concrete warning that porting NVIDIA-targeted vLLM recipes to MI300X is not drop-in.

Baseten's Philip Kiely and Ali Taha present a masterclass on inference engineering, covering autoregressive and diffusion model deployment. Ali recently published a viral deep-dive into Kimi K3's model code (tracing its full lineage across 8 papers), while Philip authored what's described as the definitive book on inference engineering — the discipline of turning trained weights into fast, reliable, affordable production systems.

Why: If you ship LLM or diffusion models to production, inference engineering is where your latency, cost, and reliability are actually won or lost. The Kimi K3 code breakdown and the inference engineering book are concrete references for understanding how modern open-weight models are structured and how to optimize serving — worth reviewing before your next model deployment or vendor evaluation.

This Interconnects post argues that AI lab consolidation hasn't materialized — more organizations are training strong open-weight models than ever, with Thinking Machines' Inkling (975B-A41B multimodal MoE, plus a 276B-A12B variant) and Tencent's Hy3 (295B-A21B MoE, now Apache 2.0 licensed) as key examples. The piece highlights that Thinking Machines' open model fine-tuning service is reportedly generating hundreds of millions in annual revenue, and that Chinese labs continue releasing competitive open models at a sustained pace.

Why: If you're choosing between API-based proprietary models and self-hosted open weights, the gap is narrowing fast — Tencent's switch to Apache 2.0 on Hy3 removes a real licensing blocker for commercial use, and Inkling's smaller 276B-A12B variant is positioned as a fine-tuning base worth evaluating for cost-sensitive deployments. Builders should benchmark these against their current API spend before assuming proprietary is cheaper.

Lumentum CEO Michael Hurlston warned at the RAISE Summit in Paris that indium phosphide—the compound semiconductor used in every AI data center laser—is entering a supply squeeze worse than the DRAM/NAND shortages. Lumentum runs five indium phosphide fabs but is already shipping 30% below customer demand, as hyperscalers order lasers in the hundreds of millions versus telecom's historical hundreds. Nvidia responded in March by investing $2 billion each into Lumentum and Coherent, which together hold most global indium phosphide capacity.

Why: If you build or procure AI infrastructure, expect silicon photonics and co-packaged optics components to face multi-quarter allocation constraints and price pressure—plan procurement timelines and vendor relationships now rather than assuming optics will scale freely with GPU demand.

Researchers from Singapore's Nanyang Technological University disclosed 84 security vulnerabilities across 4G and 5G core network implementations (Open5GS, OpenAirInterface, free5GC, SD-Core, eUPF) in signaling protocols GTP-C and PFCP. The root cause is a shared pattern of implicit trust between core network functions, made worse by cloud-native deployments exposing previously internal interfaces. They used an LLM-assisted multi-agent system called iFinder to categorize known flaws and discover new ones.

Why: If you build, operate, or audit telecom infrastructure using any of these open-source LTE/5G cores—several of which back commercial deployments, not just research testbeds—check whether your GTP-C and PFCP interfaces are reachable externally and whether message format, semantics, and resource availability are validated. Malaysian telcos and infrastructure providers moving to cloud-native 5G cores face the same expanded attack surface.

Simon Willison's `llm` CLI tool hits 0.32rc2, changing the default model from GPT-4o mini to GPT-5.6 Luna ($0.20/$1.20 per million input/output tokens vs $0.15/$0.60 for 4o mini). It also adds `llm openai endpoint`, a command for running prompts, chats, and model listings against any OpenAI-compatible endpoint without prior configuration, runnable via `uvx` without installing llm.

Why: If you use `llm` as your daily CLI, your default model just changed and your token costs went up slightly—run `llm models default gpt-4o-mini` or `gpt-5-nano` ($0.05/$0.40) if you want cheaper defaults. The new `llm openai endpoint` command is immediately useful for testing prompts against local models (e.g., LM Studio at 127.0.0.1:1234) or any OpenAI-compatible API without setup, which is handy for Malaysian developers experimenting with self-hosted or regional LLM endpoints.

AWS grew nearly 37% in Q2 2026 to $42.23B, beating analyst estimates of $40.54B and marking its fastest expansion since 2021. AWS's AI business and its custom chips each surpassed $25B in annualized revenue, more than doubling year-over-year, while Azure grew 43% and Google Cloud surged 82% to nearly $25B for the quarter.

Why: If you're deciding cloud provider for AI workloads, AWS's custom chip revenue doubling signals real production traction for Trainium/Inferentia as cost-competitive alternatives to NVIDIA; compare pricing and availability for your inference workloads across AWS, Azure, and GCP before committing, since all three are aggressively scaling AI capacity.

Internal Amazon AI usage metrics revealed the company accidentally spent $1.8 million using Claude for menial coding tasks, going 860% over budget in what was described as 'catastrophically expensive' coding blunders. The article body is mostly website boilerplate, so detailed context beyond the headline figures is unavailable from the provided text.

Why: If you're using Claude or similar LLMs for coding tasks at scale, set hard cost ceilings and monitor token usage per task — Amazon's 860% budget overrun shows that unbounded AI agent loops on trivial work can rack up seven-figure bills fast. For Malaysian startups using API-based AI coding tools, this is a concrete reason to implement per-task spend limits before scaling usage.

Simon Willison released llm-chat-completions-server 0.1a0, an LLM plugin that starts a localhost server exposing all installed LLM models through an OpenAI Chat Completions-compatible endpoint. It leverages the content-addressable log design in LLM 0.32rc1 to de-duplicate conversation messages via hashes, so repeated conversation prefixes don't re-send identical content. The plugin was written entirely by GPT-5.6 Sol.

Why: If you use the LLM CLI tool, you can now point any OpenAI-compatible client (agents, chat UIs, eval harnesses) at localhost:9001 and use your locally-installed models instead of paying for OpenAI API calls—useful for testing agent workflows locally before spending on hosted endpoints. The de-duplication via content-addressable hashes means long multi-turn conversations won't redundantly reprocess earlier messages.

British AI neocloud Nscale is acquiring Anyscale, the company behind the open-source Project Ray distributed programming framework, for $1.65 billion. The acquisition aims to vertically integrate Nscale's compute infrastructure with Anyscale's workload management and scaling software for AI training and inferencing. Anyscale will retain its branding and existing customers, with its 200 employees joining Nscale.

Why: If your team uses Ray or Anyscale for scaling AI workloads, the platform will continue operating independently under Nscale's ownership, but expect deeper integration with Nscale's hardware over time. For builders, this signals that neoclouds are increasingly bundling orchestration software with their own infrastructure, which could eventually impact pricing, portability, and vendor lock-in for AI compute stacks.

As of June 23, 2026, cdnjs—the open-source CDN used on ~12% of all websites, serving 108,000 requests/second (9B/day)—now runs entirely on Cloudflare's Developer Platform (Workers, Workflows, D1, Queues, R2, KV, Containers). The migration surfaced platform limits that Cloudflare then addressed. Notably, LLM coding assistants like ChatGPT, Claude, and Cursor keep generating cdnjs URLs because 15 years of training data reference them, making cdnjs still highly relevant despite the rise of bundlers and ESM.

Why: If you build on Cloudflare's Developer Platform (Workers, D1, R2, KV, Containers), this is a real-world stress test at 9B requests/day across 330+ data centers—useful for gauging production readiness. If you use AI coding assistants, expect them to keep emitting cdnjs <script> tags; plan around that rather than fighting it.

Google has launched Gemini Spark in Malaysia, an agentic AI powered by Gemini 3.6 Flash that runs continuously on Google cloud infrastructure and executes multi-step tasks across Gmail, Docs, Sheets, Keep, and Google Tasks even when your device is locked. It is available in Bahasa Malaysia and English for Google AI Pro and Google AI Ultra subscribers, with zero-setup Workspace integration and confirmation gates before high-stakes actions like sending emails or making purchases. Practical examples include auto-logging utility bills into a Ringgit-denominated spreadsheet, parsing travel bookings into calendar entries, and drafting client lead responses.

Why: If you already pay for Google AI Pro or Ultra in Malaysia, you now have a zero-setup agent that can automate inbox-to-spreadsheet-to-calendar workflows without writing any code—evaluate whether it replaces lightweight Zapier or custom automation you're currently paying for. SaaS founders whose products overlap with inbox parsing, expense tracking, or lead-response automation should assess Spark's Workspace-native advantage as a competitive threat.

Microsoft reported $90B quarterly revenue ($35.8B net income) and CEO Satya Nadella is openly pitching Microsoft's homegrown models and agent infrastructure as alternatives to OpenAI and Anthropic, telling enterprises to use multiple models and avoid relying on frontier labs for the agentic application layer. He explicitly advised keeping the 'harness separate from the model' so any model remains swappable, framing data leaks and vendor lock-in as risks of trusting model makers directly.

Why: Nadella's architectural prescription—decouple your agent harness from the model so models are swappable—is now being echoed by the largest enterprise cloud vendor, which means builders should design agent stacks with a model-agnostic abstraction layer rather than hard-coding to OpenAI or Anthropic APIs. If you're building AI agents on Azure, expect Microsoft to push its own models, security tooling, and agent infrastructure as the default, which affects vendor selection and cost planning.

Latent Space's roundup tracks AI adoption across financial services, summarizing talks from FactSet, Nubank, Intuit, Kepler, Morgan Stanley, Fidelity, and others. Key themes: AI agents in finance require evals, provenance, supply-chain vetting of AI skills, event-sourced audit trails, and prompt-injection defense—not just LLM calls.

Why: If you're building AI agents for any regulated vertical (fintech, insurance, accounting), the patterns here are concrete and worth copying: Nubank uses simulations to unblock agent evals as a release gate; FactSet treats AI skills as infrastructure needing ownership and governance; FlyersSoft uses event-sourced systems as the foundation for auditable agent decision loops; Fidelity flags group-chat and wearable agents as forcing new thinking on memory, permissions, and prompt-injection defense. Malaysian builders in the digital banking or payments space should treat these as a checklist for what enterprise-grade agent deployment actually requires.

OpenAI reports that GPT-5.6 Sol scored only 7.8% on the ARC-AGI-3 benchmark using the official harness, but solved all six puzzle levels when using their Responses API harness instead. The two settings that made the difference were retaining reasoning across steps and enabling compaction—essentially letting the agent remember what it has done and manage its context window more effectively.

Why: If you are building AI agents that solve multi-step problems, this is concrete evidence that enabling reasoning retention and context compaction in your harness can be the difference between total failure and full success. Check whether your agent framework preserves intermediate reasoning state between steps or discards it, and whether compaction is available—these are not cosmetic settings.

Cybersecurity researchers who hunt for vulnerabilities and build exploit tools report that AI guardrails from OpenAI and Anthropic are getting in the way of their legitimate offensive security work. The restrictions limit how researchers can use AI models for tasks like analyzing malware, fuzzing, and exploit development.

Why: For developers and AI/ML learners building security tooling or exploring AI-assisted vulnerability research, understanding where guardrails block legitimate work helps set realistic expectations for AI-assisted security workflows. Malaysian builders in fintech, payments, or govtech—where security testing is critical—should know these limitations when integrating LLMs into their security pipelines.

AMD is introducing Helios, a rack-scale AI system intended to compete directly with Nvidia's data center offerings, with shipments to customers expected later this year. The move signals AMD's continued push into the AI training and inference infrastructure market.

Why: For builders running AI workloads, a credible Nvidia alternative could eventually ease GPU supply constraints and put downward pressure on cloud compute pricing. Malaysian startups and ML teams relying on hyperscaler GPU access may benefit from increased competition, though real impact depends on software ecosystem maturity and actual availability.

Patreon is laying off 20% of its workforce to adjust its cost structure in response to market changes. In a staff memo, the company stated that while its core business remains strong, the restructuring is necessary for long-term stability.

Why: For SaaS and startup founders, this highlights the ongoing market pressure to optimize cost structures and maintain profitability, even when core business metrics appear healthy. It serves as a reminder for builders to proactively manage operational expenses during economic shifts.

AegisAI, founded by former Google security executives, raised $36M to build AI agents that detect AI-driven spear phishing by analyzing message anomalies the way a human reviewer would. The startup targets the growing problem of attackers using generative AI to craft highly personalized phishing messages at scale.

Why: For builders and founders, this signals a real market emerging around AI-vs-AI security: defensive agents that reason over messages rather than rely on static rules. Malaysian SaaS teams handling email, fintech, or enterprise communications should watch this space, as AI-generated phishing will likely pressure local compliance and security expectations.

Experts are pushing back on speculation that Kimi K3's strong performance came primarily from distilling Anthropic's Fable model, arguing the model's quality and speed of development suggest more sophisticated training methods were involved. The debate highlights ongoing scrutiny around how new AI labs achieve competitive results and the difficulty of proving distillation versus independent capability building.

Why: For AI/ML learners and builders tracking model provenance, this matters because the distillation debate affects how you evaluate which models to use, trust, or build on. Malaysian startups and developers selecting foundation models should understand that model lineage claims are contested and that strong performance alone doesn't confirm or rule out distillation.

Latent Space reports the release of Laguna S 2.1, a new model from neolab positioned as cheaper than Deepseek v4 Flash while outperforming V4 Pro. Details are sparse in the excerpt, but the headline frames it as a notable win in the competitive LLM landscape.

Why: For Malaysian builders and startups, a cheaper-yet-stronger model option directly impacts API costs and unit economics, especially for AI agent workloads and SaaS products. If benchmarks hold, it could be worth evaluating as an alternative to Deepseek or other budget models in production pipelines.

Google's cloud business is thriving as companies adopt its AI and AI infrastructure services, helping the tech giant report record profits. This validates the massive capital expenditure Google has been pouring into AI infrastructure.

Why: For Malaysian builders and startups, Google Cloud's AI-driven growth signals continued investment in AI infrastructure and services that many local companies already use. This likely means more competitive pricing, better regional availability, and expanded AI tooling that Malaysian developers and SaaS founders can leverage without building their own infrastructure.

Yope, a social app centered on private friend and family groups, has secured $12.3 million in seed funding. The startup differentiates itself by avoiding algorithmic feeds and ads, focusing instead on private communities enhanced by AI features.

Why: For SaaS founders and developers, this highlights a growing market demand for privacy-first, anti-algorithm social platforms and demonstrates how AI can be integrated into consumer apps to enhance real-world relationships rather than drive engagement metrics.

Celebrity entrepreneur and chef Khairul Aming has issued a Letter of Demand to Maxis and filed a police report after his private customer records were exposed. He announced the escalation via Threads, marking a formal legal response to the data breach involving the telco giant.

Why: For Malaysian builders and SaaS founders, this highlights the real legal and reputational consequences of data breaches involving customer records. It underscores the importance of data handling practices, vendor/telco accountability, and the growing willingness of individuals and businesses in Malaysia to pursue formal legal action over data privacy failures.

Glow has emerged from stealth with a $1.2B valuation, targeting endpoint security risks created by the rapid enterprise adoption of AI agents and developer tools. The startup is positioning itself around a new threat category that traditional endpoint security wasn't designed to handle.

Why: For builders deploying AI agents or integrating AI-powered developer tools into workflows, this signals that security tooling is starting to catch up to the new attack surfaces created by autonomous agents. Malaysian startups and enterprises adopting agentic AI should factor endpoint security into their roadmaps, and there may be opportunities for local players in the AI security space.