Training a coding model to paint watercolours with TRL and OpenEnv
- ID
- 20953
- Status
- summarized
- Published
- 03 Sep 2026, 8:00 AM
- Fetched
- 03 Sep 2026, 4:11 PM
- Provider
- Hugging Face Blog
- Category
- developer-ai
- Original URL
- https://huggingface.co/blog/train-to-paint-with-code
- Source URL
- https://huggingface.co/blog/feed.xml
Summary
- Score
- 7.0
- Created
- 03 Sep 2026, 4:11 PM
- Tags
- Audience
- developersai_ml_learnersvibe_coders
What happened
Sergio Paniego reproduces Surya Narreddi's viral watercolour-painting LLM project using TRL and OpenEnv, training a Qwen3.5-35B-A3B model with LoRA and GRPO to write p5.brush JavaScript that generates watercolour art. The entire pipeline runs on Hugging Face infrastructure—Jobs for training, Spaces for the RL environment and scorer model, Inference Providers for the pairwise judge—and all artifacts, scripts, and a hand-rated reference pool are published openly.
Why it matters
This is a concrete, reproducible GRPO recipe with exact hyperparameters (lr 5e-5, 110 steps, 240 episodes, 8 generations, max-completion-length 8192) and three compared reward mixes, runnable in a single command on HF Jobs with an H200. If you want to learn RL fine-tuning for code-generation models, you can clone the Spaces, swap the subject and references, and experiment with reward design immediately rather than building from scratch.
Discussion angle
Compare the three reward mixes the author trained and discuss which reward signals (aesthetic scorer vs. pairwise judge vs. reference matching) generalize best when adapting this recipe to non-art code-generation tasks.