AI and Tech Summaries - AI Weekly Malaysia

Summaries

Short AI and tech summaries with source links, signal scores, and why each update matters for builders, founders, and Malaysian tech workers.

Window

Today Yesterday This week Last week This month Last month All time

Showing 1-1 of 1 results

Date	Provider	Score	Summary
01 Sep 2026, 5:52 PM	Hacker News	7.5	I trained a small transformer in 1.5hrs and it beats many LLMs Mithil Vakde trained a small transformer from scratch in 1.5 hours on a single 5090 GPU for 67 cents, scoring 44% on ARC-AGI-1—matching TRM/HRM and beating many LLMs. Key upgrades from his previous model include SwiGlu instead of GELU, RMSnorm instead of layernorm, scaling to 8 layers, more data diversity, and better shuffling. The approach uses test-time training with 3D RoPE embeddings, color/dihedral permutations, and AAIVR augmentation, and the code is open source. Why: If you're an AI/ML learner or builder, this demonstrates that sample efficiency—not scale—is a tractable problem worth working on, and that meaningful ARC-AGI results are achievable on a single GPU for under a dollar. The specific architecture choices (SwiGlu, RMSnorm, 3D RoPE, test-time training per puzzle) are concrete techniques you can experiment with directly using the open-source code. ai-ml transformers arc-agi sample-efficiency open-source

Date

Provider

Score

Summary

01 Sep 2026, 5:52 PM

Hacker News

7.5

I trained a small transformer in 1.5hrs and it beats many LLMs

Mithil Vakde trained a small transformer from scratch in 1.5 hours on a single 5090 GPU for 67 cents, scoring 44% on ARC-AGI-1—matching TRM/HRM and beating many LLMs. Key upgrades from his previous model include SwiGlu instead of GELU, RMSnorm instead of layernorm, scaling to 8 layers, more data diversity, and better shuffling. The approach uses test-time training with 3D RoPE embeddings, color/dihedral permutations, and AAIVR augmentation, and the code is open source.

Why: If you're an AI/ML learner or builder, this demonstrates that sample efficiency—not scale—is a tractable problem worth working on, and that meaningful ARC-AGI results are achievable on a single GPU for under a dollar. The specific architecture choices (SwiGlu, RMSnorm, 3D RoPE, test-time training per puzzle) are concrete techniques you can experiment with directly using the open-source code.