The next major version of the self-contained analytics database will offer async I/O, a new SQL parser, and much more.
DuckDB, the analytics database with high-end features in a single easily deployed binary, is due to release a major update, version 2.0, this fall.
In a blog post on the DuckDB site, the project’s maintainers outlined several key features that motivated a major version bump. The single biggest is a client/server mode, allowing DuckDB to function both as a compact, in-process database for a single client (its original use case), and as a service for multiple clients. The previously released quack extension for DuckDB made this possible on an experimental basis; now quack is being promoted to a stable release with DuckDB 2.0. The quack extension also makes it easier to remotely query other supported databases such as PostgreSQL and MySQL.
Many of the other major changes in DuckDB 2.0 are internal, but with big implications for end users. The data engine beneath DuckDB now works asynchronously, allowing the I/O layer and the query processing layer to scale separately and make better use of parallelism. This, plus other internal changes, means faster queries overall, with the speedups ranging up to 40 times faster depending on the kind of workoad involved. A new storage format for the database means databases with large indexes and “wide” tables (many columns) can be opened and read faster. A completely written SQL parser allows extensions to extend SQL syntax and delivers more detailed error messages.
DuckDB already has a fair number of third-party extensions written for it, but extensions have had to be written against an unstable C++ API. Each new DuckDB release required extensions to be recompiled for that release. With DuckDB 2.0, a stable C API now supports a broad feature set for extensions to use. It also allows extensions to be compiled from a single header file, and they won’t need recompiling to work with new releases. A work-in-progress effort allows you to register your own DuckDB extension repository, rather than require an extension to be distributed from the community repo.
Previews of DuckDB 2.0 are now available in an unstable, nightly release version. No date has yet been set for a release candidate or final release, although the goal is to release it sometime this coming fall.
