Why DuckDB 2.0 is faster
- ID
- 34164
- Status
- summarized
- Published
- 11 Oct 2026, 2:08 AM
- Fetched
- 11 Oct 2026, 8:19 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://motherduck.com/blog/why-duckdb-20-is-faster/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 8.0
- Created
- 11 Oct 2026, 8:19 PM
- Tags
- Audience
- developersdatabase_learnersai_ml_learners
What happened
MotherDuck's Mehdi Ouazza benchmarks the DuckDB 2.0 alpha (due this fall) and reports that a 2.2 GB Parquet file on S3 — Stack Overflow votes, 228M rows, 2268 row groups, reading one of four columns (~230 MB) — drops from 18.8s on DuckDB 1.5.5 to 7.7s on 2.0 alpha on an M5 laptop over home internet. The cause is async I/O: 1.5.5 makes each of 18 workers download-then-decode in turn, while 2.0 uses a separate download thread pool that keeps dozens of row groups in flight, controlled by read_ahead_depth (default -1 = automatic, set 0 to restore 1.5 behavior). Other numbers: 23 large Parquet files totalling 13.6 GB go 11.8s to 3.9s, a 1.7 GB plain CSV goes 116s to 55s, but 30 tiny ~1 MB Parquet files barely move (3.7s to 3.3s). The HN thread has 200 points and 62 comments.
Why it matters
If you query Parquet on S3 with DuckDB, the upgrade path is now concrete: expect roughly a 2.4x speedup on one large file and ~3x across many large files, but almost nothing on many tiny files, so batching small files or keeping them local is still the fix. Because read_ahead_depth defaults to auto, upgrading silently changes memory and network behavior — set it to 0 if you need the old 1.5 profile. All figures come from one M5 laptop over home internet to us-east-1 and the author explicitly says to re-run your own before quoting them; there is no Malaysia- or SEA-specific angle in this text, the impact is purely tooling.
Discussion angle
Where does the async I/O win break down in real pipelines — the tiny-file result suggests per-file overhead dominates, so what does that mean for table layouts and compaction, and does read_ahead_depth's automatic default quietly raise S3 request or egress costs for anyone running DuckDB against object storage?