AI Weekly Malaysia

A fully open recipe for fine-tuning LiquidAI's LFM2.5-350M model with GRPO via the TRL library to improve structured-output compliance, evaluated on the IFStruct benchmark. The training runs in ~500 samples and 100 steps on a free-tier Colab or Kaggle GPU, lifting IFStruct scores from 22.6% to 29.7%. Evaluation is done locally via llama.cpp on a MacBook Pro M5 Max with 36GB unified memory.

Why: If you ship LLM-powered pipelines that depend on schema-valid JSON or structured output, this shows you can cheaply fine-tune a 350M-parameter model on free-tier GPUs rather than paying for a large model API, with a concrete benchmark delta (22.6% to 29.7%) to set expectations. The entire pipeline is reproducible on GitHub and runnable without paid infrastructure.

Sergio Paniego reproduces Surya Narreddi's viral watercolour-painting LLM project using TRL and OpenEnv, training a Qwen3.5-35B-A3B model with LoRA and GRPO to write p5.brush JavaScript that generates watercolour art. The entire pipeline runs on Hugging Face infrastructure—Jobs for training, Spaces for the RL environment and scorer model, Inference Providers for the pairwise judge—and all artifacts, scripts, and a hand-rated reference pool are published openly.

Why: This is a concrete, reproducible GRPO recipe with exact hyperparameters (lr 5e-5, 110 steps, 240 episodes, 8 generations, max-completion-length 8192) and three compared reward mixes, runnable in a single command on HF Jobs with an H200. If you want to learn RL fine-tuning for code-generation models, you can clone the Spaces, swap the subject and references, and experiment with reward design immediately rather than building from scratch.