Kog is going deeper to squeeze more inference out of GPUs
- ID
- 14210
- Status
- summarized
- Published
- 14 Aug 2026, 10:50 PM
- Fetched
- 14 Aug 2026, 11:28 PM
- Provider
- TechCrunch
- Category
- technology
- Original URL
- https://techcrunch.com/2026/08/14/kog-is-going-deeper-to-squeeze-more-inference-out-of-gpus/
- Source URL
- https://techcrunch.com/feed/
Summary
- Score
- 4.5
- Created
- 14 Aug 2026, 11:28 PM
- Tags
- Audience
- developersai_ml_learnerssaas_founders
What happened
French startup Kog, founded solo by Gaël Delalleau, claims 30x faster LLM inference on conventional datacenter GPUs (AMD MI300X, NVIDIA H200) via software optimization. Its demo hit 3,000 tokens/second but only with a 2B-parameter model (Laneformer 2B, now open-sourced), and Kog admits customers won't fine-tune small models, so it is pivoting to accelerate larger models — a claim still unproven. The startup generated 200 business leads and is targeting software engineering workflows where Claude Code users wait hours for results.
Why it matters
The 3,000 TPS demo is real but narrow — it runs on a 2B model, not the large models production teams actually use. Builders should treat the '30x faster' headline as aspirational until Kog shows results on production-scale models. The open-sourced Laneformer 2B is worth examining if you work on inference optimization, but don't change your serving stack based on this.
Discussion angle
The gap between a benchmark on a 2B model and the promise of 30x speedup on large models — what would you need to see before switching inference providers, and how does this compare to what Cerebras or Groq have actually shipped?