LFM2.5 Q4\_0 Checkpoints from Quantization-Aware Distillation
- ID
- 15630
- Status
- summarized
- Published
- 19 Aug 2026, 9:48 PM
- Fetched
- 19 Aug 2026, 11:44 PM
- Provider
- Hugging Face Blog
- Category
- developer-ai
- Original URL
- https://huggingface.co/blog/LiquidAI/qad
- Source URL
- https://huggingface.co/blog/feed.xml
Summary
- Score
- 6.5
- Created
- 19 Aug 2026, 11:44 PM
- Tags
- Audience
- developersai_ml_learnersvibe_coders
What happened
LiquidAI released QAD (Quantization-Aware Distillation) Q4_0 GGUF checkpoints for four LFM2.5 models (230M, 350M, 1.2B-Instruct, 2.6B), recovering ~97% of the BF16 accuracy typically lost to 4-bit quantization while maintaining Q4_0 memory footprint and throughput. Benchmarks across GPQA Diamond, MMLU-Pro, IFEval, BFCLv4, and others show the QAD checkpoints match or exceed Q5_K_M and Q4_K_M quality at 3-33% higher decode throughput, tested on MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5.
Why it matters
If you're running LFM2.5 models on edge hardware via llama.cpp, swap your existing PTQ Q4_0 or Q4_K_M GGUFs for these QAD Q4_0 files to get measurably better reasoning, instruction-following, and tool-use accuracy at the same memory and speed — the 1.2B and 2.6B checkpoints are the most relevant for agentic workloads.
Discussion angle
Compare QAD's distillation approach against Unsloth's UD-Q4_K_XL post-training quantization — is training the quantized student model worth the complexity over better PTQ methods, and do these gains hold for non-LiquidAI architectures?