AI Weekly Malaysia

A new paper from Multiverse Computing introduces two systems-level optimizations for LLM knowledge distillation: caching the teacher's top-K logits offline so the teacher model never needs to co-reside in VRAM with the student, and a fused chunked KL-divergence loss that avoids materializing the full vocabulary×sequence-length probability matrix. Together these cut VRAM usage far below default PyTorch or NVIDIA Megatron-Bridge implementations, making long-context distillation feasible on a single GPU instead of requiring hundreds.

Why: If you're distilling large open-source models (e.g., gpt-oss-120b with its 201,088-token vocabulary) into smaller deployable students, this approach lets you skip keeping the teacher loaded during training—potentially dropping your GPU footprint from a cluster to a single card. Evaluate the offline top-K logits caching and fused KL loss before your next distillation run, especially if you've been blocked by VRAM costs.

Twitch has added a 'Training for Generative AI' opt-out toggle in channel settings, but it is enabled by default—meaning all channel content (livestreams, VODs, clips, highlights, text, images, and chat messages) is fed into Amazon's generative AI models unless a streamer manually disables it. Twitch CPO Mike Minton openly admitted the default-on choice was because 'if it was opt-in, nobody would opt in,' and confirmed Amazon has already been using Twitch data for AI training since at least 2024. Opting out only covers future model improvements, not data already ingested, and chat messages you post in another streamer's channel are governed by their setting, not yours.

Why: If you stream or build tools on Twitch, you should go into channel settings and disable 'Training for Generative AI' now if you don't want your content feeding Amazon's models—but understand this only stops future use and doesn't retroactively remove anything already trained. For Malaysian creators and builders using Twitch as a platform, this is a concrete data-rights decision point, not a theoretical one, and the chat-message cross-channel wrinkle means your audience's messages in your channel are your responsibility to protect.

Twitch will now use creators' livestream content to train Amazon's generative AI models by default, requiring streamers to manually opt out. During a stream to nearly 3,000 users, Twitch CPO Mike Minton explicitly admitted the policy is opt-out rather than opt-in because 'if this was opt-in, nobody would opt in.' Twitch framed the change as adding an opt-out setting rather than announcing new AI training, causing confusion over whether content had already been used.

Why: If you build platforms with UGC, this is a concrete example of the opt-out-vs-opt-in design choice becoming a public trust crisis—Twitch's own CPO admitted the default exists to harvest data creators wouldn't voluntarily give. For AI/ML practitioners, it signals that large-scale training data acquisition is increasingly shifting to owned-platform scraping (Amazon on Twitch, Meta on Instagram) rather than open-web crawling, which affects where data licensing and consent debates go next.

Nathan Lambert's post-training textbook 'Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs' is now published by Manning and freely available online, accompanied by a 12-hour video course, slides, a codebase with exercises, and model comparison examples. It covers topics like rejection sampling, outcome reward models, and character training at a foundational level, targeting readers with a CS background rather than beginners. The print edition is 50% off until August 19 with code PBLambert.

Why: If you're an AI/ML learner or developer moving from model usage to model fine-tuning, the free online book plus 12-hour course gives you a structured path into RLHF and post-training techniques that are otherwise thinly documented. The 50% discount code expires Aug 19, so decide before then if you want the print version.

Twitch announced it will share users' livestream content and other data with parent company Amazon to train AI models, defaulting users into data sharing with an opt-out available. Twitch's chief product officer acknowledged that opting in would be rare if it were voluntary, effectively admitting the default-on design is necessary for participation.

Why: If you stream or build tools on Twitch, your content is now training Amazon's AI unless you actively opt out—check your account settings now. For Malaysian creators and startups using Twitch as a platform, this means your proprietary streams, code demos, or product walkthroughs could feed Amazon's models without explicit consent, which is worth factoring into platform choice for sensitive content.

Gregory Kurtzer, founder of CentOS and Rocky Linux, has launched OpenWALDO—a project to build a shared, open-source AI training dataset with full provenance and a bill of materials. Funded by his AI infrastructure company CIQ, the effort currently targets 167 billion transparent tokens, a fraction of the trillions used by major AI labs. The project argues that even 'open-weight' models hide their training data, creating legal and compliance risk for downstream users.

Why: If you ship products using open-weight models, you currently have no auditable trail for training data lineage—OpenWALDO's 'bill of materials' concept could eventually let you point to a verified baseline corpus and reduce copyright/consent exposure. But at 167B tokens today versus trillions in proprietary datasets, this is not yet something you can train a competitive model on; treat it as a project to watch, not a dataset to use.

Independent bookstores across Europe are reportedly receiving suspicious bulk orders for thousands of books, with sellers suspecting the books are being acquired to be destroyed and used as AI training data. The orders are believed to be part of AI tech companies' broader push to acquire more training data as publicly available web sources become exhausted or legally contested.

Why: Signals that the era of scraping freely available web text for AI training is closing, and companies are turning to physical media acquisition as a data sourcing strategy. Founders building AI products should expect training data costs and legal scrutiny to rise, and should not assume their models can be trained on publicly available text indefinitely.

Amazon has used Twitch user content to train its AI models for years, and Twitch has now introduced an opt-out mechanism. The article body was not accessible—only cookie consent boilerplate was captured—so specific details on scope, retroactivity, and how to opt out are missing from the provided text.

Why: If you stream or build tools on Twitch, your content may already be in Amazon's training data. Check your Twitch privacy settings for the new opt-out, but don't assume it removes past data from already-trained models. The lack of article detail means you should read the source directly before acting.

The article title claims booksellers suspect AI firms are buying and then destroying rare books to train AI models, but the provided text contains only cookie consent boilerplate and no article body, so no concrete details can be extracted.

Why: No actionable takeaway can be derived from the provided text, which contains no substantive content beyond privacy consent notices. The headline suggests a potential data-sourcing and copyright story relevant to AI training datasets, but without the article body there is nothing to act on.