Hot Chips 2026: d-Matrix stacks AI accelerator directly on custom DRAM for 100 TB/s per card — TSMC 4nm compute die bonded face-to-face at a 36-micron pitch on top of a custom-designed die
- ID
- 18145
- Status
- summarized
- Published
- 26 Aug 2026, 8:00 PM
- Fetched
- 26 Aug 2026, 9:35 PM
- Provider
- Tom's Hardware
- Category
- technology
- Original URL
- https://www.tomshardware.com/tech-industry/semiconductors/d-matrix-stacks-its-ai-accelerator-directly-on-custom-dram-for-100-tbs-per-card
- Source URL
- https://www.tomshardware.com/feeds/all
Summary
- Score
- 5.5
- Created
- 26 Aug 2026, 9:36 PM
- Tags
- Audience
- ai_ml_learnersai_agent_userssaas_founders
What happened
d-Matrix presented Raptor at Hot Chips 2026, a 3D DRAM accelerator for generative inference that bonds a TSMC 4nm compute die face-to-face onto a custom DRAM die at 36-micron pitch, delivering 100 TB/s bandwidth from 32GB per card. CTO Sudeep Bhoja cited a measured vertical interface energy cost of 0.37 pJ/bit versus ~2.4 pJ/bit for HBM4, and an accompanying ISCA 2026 paper projects ~4.7x higher throughput per card than HBM-based designs.
Why it matters
If d-Matrix ships Raptor at volume, inference cost per token for large models could drop materially versus HBM-based GPUs, which matters for anyone pricing AI inference or building agent workloads. But this is pre-production silicon entering the market during the tightest DRAM cycle in a decade, so builders should track availability and pricing rather than plan around it now.
Discussion angle
Does near-memory compute like Raptor's 3D DRAM stacking change the economics enough to shift inference workloads away from GPU-based serving, or will HBM4 and software optimization keep dominant share?