NeoMME: an efficient Multimodal-native and Multilingual Encoder
- ID
- 21057
- Status
- summarized
- Published
- 03 Sep 2026, 9:13 PM
- Fetched
- 03 Sep 2026, 10:33 PM
- Provider
- Hugging Face Blog
- Category
- developer-ai
- Original URL
- https://huggingface.co/blog/Hcompany/neomme
- Source URL
- https://huggingface.co/blog/feed.xml
Summary
- Score
- 6.5
- Created
- 03 Sep 2026, 10:34 PM
- Tags
- Audience
- developersai_ml_learners
What happened
Hcompany released NeoMME, a 260M and 800M multilingual multimodal encoder that processes text tokens and raw image patches in a single bidirectional Transformer trained from scratch with masked discrete-diffusion—no separate vision tower or causal LM. Fine-tuned for visual document retrieval, the 260M model encodes ~51 pages/sec on an L40S (about 2x ColModernVBERT), and hierarchical token pooling plus asymmetric quantization cut late-interaction index storage from ~1.5 MB to 6 kB per page (255x smaller) while retaining >95% of baseline nDCG@10. Checkpoints are Apache 2.0 on Hugging Face Transformers.
Why it matters
If you ship visual document RAG, the 255x index storage reduction at >95% retained nDCG@10 is a concrete cost win—re-evaluate your ColPali/ColModernVBERT pipeline against NeoMME-Retriever, especially if you're paying for vector index storage at scale. The 2x throughput on L40S also means fewer GPUs for the same ingestion rate.
Discussion angle
Compare NeoMME's single-tower from-scratch approach against the dominant pattern of bolting a pretrained vision tower onto a text encoder—is the storage/throughput win enough to justify retraining from scratch, or is this only relevant for teams whose retrieval cost is dominated by index storage?