What Nvidia's first Groq 3 LPU benchmarks do and don't tell us about its $20B gamble
- ID
- 17306
- Status
- summarized
- Published
- 24 Aug 2026, 11:00 PM
- Fetched
- 25 Aug 2026, 1:01 AM
- Provider
- The Register
- Category
- technology
- Original URL
- https://www.theregister.com/systems/2026/08/24/what-nvidias-first-groq-3-lpu-benchmarks-do-and-dont-tell-us-about-its-20b-gamble/5291880
- Source URL
- https://www.theregister.com/headlines.atom
Summary
- Score
- 7.0
- Created
- 25 Aug 2026, 1:02 AM
- Tags
- Audience
- developersai_ml_learnersai_agent_userssaas_founders
What happened
Nvidia's first independent benchmarks for its Groq 3-based LPX racks show 3,400 tokens/second on Gemma 4 31B with 100K-token input, roughly 4x faster than Cerebras' 882 tok/s. The architecture trades capacity for speed: each LPU has only 500 MB of on-die SRAM (vs 288 GB on Rubin GPUs) but 150 TB/s of bandwidth, requiring models to be distributed across up to 256 LPUs per rack via Ethernet. Nebius will be among the first neoclouds to deploy the combined GPU-LPU systems.
Why it matters
If you're building AI agents, inference throughput directly constrains how long models can reason and how many agent turns are feasible within a time budget. The 3,400 tok/s figure is a best-case benchmark on a specific model, not a guarantee for your workload, but it signals that agentic inference economics are shifting toward speed-at-a-premium. Builders evaluating neocloud providers like Nebius for inference serving should track whether LPX-class throughput justifies the cost for their agent architectures rather than assuming GPU-only deployments.
Discussion angle
The 500 MB-per-chip SRAM constraint means model distribution across racks is mandatory, not optional — discuss whether this architecture actually helps real agentic workloads or just wins synthetic benchmarks, and what it means for inference pricing when you're paying for 256-chip racks to serve a 31B model.