Hot Chips 2026: High Bandwidth Flash promises massive bandwidth and capacity, but its usability is extremely limited — new memory format strikes a balance between HBM and NAND flash
- ID
- 18142
- Status
- summarized
- Published
- 26 Aug 2026, 9:00 PM
- Fetched
- 26 Aug 2026, 9:35 PM
- Provider
- Tom's Hardware
- Category
- technology
- Original URL
- https://www.tomshardware.com/pc-components/ssds/hot-chips-2026-high-bandwidth-flash-promises-massive-bandwidth-and-capacity-but-its-usability-is-extremely-limited-new-memory-format-strikes-a-balance-between-hbm-and-nand-flash
- Source URL
- https://www.tomshardware.com/feeds/all
Summary
- Score
- 4.5
- Created
- 26 Aug 2026, 9:36 PM
- Tags
- Audience
- developersai_ml_learnerssaas_founders
What happened
At Hot Chips 2026, OXMIQ Labs reported that High Bandwidth Flash (HBF), first proposed by SanDisk in early 2025, cannot replace HBM for most AI workloads and may even hurt performance for some. The spec defines three grades: Grade 1 offers 256GB at 384 GB/s via 8 GT/s UCIe, Grade 2 offers 512GB at 1.536 TB/s via 16 GT/s UCIe, and Grade 3 scales further. HBF only makes sense as a specialized tier for large, relatively cold datasets.
Why it matters
If you are designing or costing AI inference infrastructure, do not assume HBF will let you cheaply scale memory-bound LLM serving — the article explicitly says it cannot replace HBM for most workloads and can degrade performance. It may eventually serve as a cold-data tier alongside HBM, but no shipping product or timeline is given, so no infrastructure decision should hinge on it today.
Discussion angle
Whether the three-grade HBF spec (256GB/384 GB/s up to 512GB/1.536 TB/s) is enough bandwidth to matter for any real AI workload, or whether NAND-based memory tiers are fundamentally too slow for inference and useful only for checkpoint loading or large cold parameter storage.