AI Weekly Malaysia

A new paper demonstrates a method to extract encrypted reasoning traces from frontier AI model APIs (Claude, GPT, Gemini) by replaying signed thinking blocks into weaker models from the same provider and prompting them to transcribe. The authors scanned ~7,000 public Claude Code/Codex sessions and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data—64 of which appeared exclusively inside reasoning blocks, not visible session output.

Why: If you've ever shared a Claude Code or Codex session publicly (e.g., in a GitHub repo, bug report, or forum post), your encrypted reasoning blobs may contain leaked API keys, passwords, or emails that can now be decoded. Audit any shared sessions immediately and rotate credentials. Builders using reasoning model APIs should also understand that obscured chain-of-thought is no longer a reliable security boundary against distillation or data leakage.

A new paper from Multiverse Computing introduces two systems-level optimizations for LLM knowledge distillation: caching the teacher's top-K logits offline so the teacher model never needs to co-reside in VRAM with the student, and a fused chunked KL-divergence loss that avoids materializing the full vocabulary×sequence-length probability matrix. Together these cut VRAM usage far below default PyTorch or NVIDIA Megatron-Bridge implementations, making long-context distillation feasible on a single GPU instead of requiring hundreds.

Why: If you're distilling large open-source models (e.g., gpt-oss-120b with its 201,088-token vocabulary) into smaller deployable students, this approach lets you skip keeping the teacher loaded during training—potentially dropping your GPU footprint from a cluster to a single card. Evaluate the offline top-K logits caching and fused KL loss before your next distillation run, especially if you've been blocked by VRAM costs.