ducklake-sdk — Native SDKs for DuckLake

Read and write DuckLake tables from Rust and Python - no DuckDB required.

CI Build GitHub Release

DuckLake is an integrated data lake and catalog format that stores metadata in a SQL catalog database and writes data as Parquet files. This repository provides standalone Rust and Python SDKs that talk to DuckLakes directly, with no dependency on DuckDB or its DuckLake extension.

All language SDKs are built on the same Rust core, which bundles the implementation of the DuckLake specification.

Python (ducklake-sdk)

pypi-version conda-forge python-version readthedocs codecov

Rust (ducklake)

crates.io docs.rs codecov

[!WARNING] This is not an official SDK released by the DuckDB Foundation.

Getting Started

Python

pip install ducklake-sdk            # core
pip install "ducklake-sdk[polars]"  # Polars integration
pip install "ducklake-sdk[arrow]"   # Arrow + DuckDB integration

Rust

cargo add ducklake

Quick Example

import ducklake as dl
import polars as pl

# Create a new DuckLake backed by SQLite metadata and local Parquet storage
ducklake = dl.create("sqlite:///metadata.sqlite", data_path="data_files/")

# Define a table.
table = ducklake.create_table(
    "events",
    schema={"id": dl.Int64(), "message": dl.Varchar()},
)

# Write data using Polars
lf = pl.LazyFrame({"id": [1, 2, 3], "message": ["hello", "ducklake", "sdk"]})
table.sink_polars(lf)

# Read it back as a Polars LazyFrame
df = table.scan_polars().collect()

For the full API, see the Python documentation or the Rust API docs.

Features

The Rust core — and therefore every SDK built on top of it — supports:

The Python SDK additionally provides:

  • Reading and writing data through Polars
  • Reading, writing, and deleting data through DuckDB
  • Maintenance operations — compaction, snapshot expiration, and more — via DuckDB

Compatibility Matrix

Catalog Databases

Database Status
SQLite ✅
Postgres ✅
MySQL 🟧 (no data inlining*)

*Data inlining for MySQL is not defined in the DuckLake specification.

Storage Backends

Backend Status
Local / NFS ✅
AWS S3-compatible ✅
Google Cloud Storage ✅
Azure Blob Storage ✅

DuckLake Specification Versions

Version Status
1.0 ✅ (actively supported)
0.4 ⬆️ (requires migration)
0.3 ⬆️ (requires migration)
0.2 ⬆️ (requires migration)
0.1 ⬆️ (requires migration)

See the DuckLake release calendar for upcoming versions.

Project Status

[!NOTE] This project is in alpha. It will move to beta once the full specification is implemented, and to stable once all relevant limitations have been addressed. Expect occasional breaking changes until then.

Not yet implemented from the specification

  • GEOMETRY and VARIANT data types
  • Mapping columns by name (Parquet files must currently carry field IDs)
  • Views, macros, sort info, and encrypted files

Known limitations

Rust SDK (may impact efficiency):

  • Tables partitioned with a non-identity transform do not benefit from file pruning yet.
  • Filters are not pushed down into the metadata query. Statistics are still loaded eagerly and used by readers to prune files, but the metadata query may transmit more data than necessary.

Python SDK:

Contributing

Contributions, bug reports, and feature requests are very welcome. See the contribution guidelines to get started.

License

Licensed under the MIT License.

Project links

Data verified by PyPI on Aug 11, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.0.15)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for borchero from gravatar.com borchero

Credits

Author: Oliver Borchert

GitHub Statistics

Data verified by PyPI on Aug 11, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.10

Provides Extra

arrow polars sqlalchemy

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ducklake_sdk-0.0.12.tar.gz (209.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ducklake_sdk-0.0.12-cp310-abi3-win_amd64.whl (10.3 MB view details)

Uploaded CPython 3.10+Windows x86-64

ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ducklake_sdk-0.0.12-cp310-abi3-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ducklake_sdk-0.0.12-cp310-abi3-macosx_10_12_x86_64.whl (9.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ducklake_sdk-0.0.12.tar.gz.

File metadata

  • Download URL: ducklake_sdk-0.0.12.tar.gz
  • Upload date:
  • Size: 209.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ducklake_sdk-0.0.12.tar.gz
Algorithm Hash digest
SHA256 3b93e582b60622d2306cdc6b0b636d6406745699f960554c4cf8e89ec0ad1eb8
MD5 4d74807cf1056eb868c636469e191b22
BLAKE2b-256 5690d1ac1279308aee22e1bc4c96ff907660b1a2d8c34653a985ac2f5e3a8f97

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12.tar.gz:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.12-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.12-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 9ecc66cfd7e4a7d45f8bb636ce878d49b7d9fe4123488eea766c9f55048d8056
MD5 697228754210e48ae1e8923e59f0e8a0
BLAKE2b-256 07cf1d63430e9b971adfdce51c65069c79fee59d2803063965500cd3b4df3b6c

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12-cp310-abi3-win_amd64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d135b30c978114e0ab118bbfa15986ab2b67b1c0a09ef2cdb2329999823e828b
MD5 bfd7a6c5d3ce56bf6a2e4ca151827f03
BLAKE2b-256 b02826d718feaca9681b7157dc00e01dbbfb180af732d357b7ba293e51597b41

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 7758ffbc4f7dfdba7adde30d2f7129ec45959ebd83f5554310dc86aa8b80faab
MD5 004af583aa917d90172cd4a95007dfe3
BLAKE2b-256 e57c13a22575c670f973d56c6484bc965fa977d273a9912f53e1ee9b7130275e

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.12-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.12-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7602572a07ec17448df6853bb7982e85e27335dc0d2cb3f716428bef5e7c295c
MD5 07841785d2a41670b1900310197075bd
BLAKE2b-256 a06177bcc16c69a7780b1b07c8d7ef669e8586ba7febf3510da33ac46392e3d1

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.12-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.12-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 5ca6625dd9e5aee9513383efa6c3128017b45eaa10653a00e7da1a36ccfbe1ea
MD5 4f3104e80a789b4ef85a4ec7138e525b
BLAKE2b-256 d24d27986c2ba126b1261d699490f342325d51fdbadecc777692fbec1b9e057b

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.12-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.0.15

6 files

0.0.14

6 files

0.0.13

6 files

This release

0.0.12 This release

6 files

0.0.11

6 files

0.0.10

6 files

0.0.9

6 files

0.0.8

6 files

0.0.7

6 files

0.0.6

6 files

0.0.5

6 files

0.0.4

6 files

0.0.3

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page