ducklake-sdk — Native SDKs for DuckLake

Read and write DuckLake tables from Rust and Python - no DuckDB required.

CI Build GitHub Release

DuckLake is an integrated data lake and catalog format that stores metadata in a SQL catalog database and writes data as Parquet files. This repository provides standalone Rust and Python SDKs that talk to DuckLakes directly, with no dependency on DuckDB or its DuckLake extension.

All language SDKs are built on the same Rust core, which bundles the implementation of the DuckLake specification.

Python (ducklake-sdk)

pypi-version conda-forge python-version readthedocs codecov

Rust (ducklake)

crates.io docs.rs codecov

[!WARNING] This is not an official SDK released by the DuckDB Foundation.

Getting Started

Python

pip install ducklake-sdk            # core
pip install "ducklake-sdk[polars]"  # Polars integration
pip install "ducklake-sdk[arrow]"   # Arrow + DuckDB integration

Rust

cargo add ducklake

Quick Example

import ducklake as dl
import polars as pl

# Create a new DuckLake backed by SQLite metadata and local Parquet storage
ducklake = dl.create("sqlite:///metadata.sqlite", data_path="data_files/")

# Define a table.
table = ducklake.create_table(
    "events",
    schema={"id": dl.Int64(), "message": dl.Varchar()},
)

# Write data using Polars
lf = pl.LazyFrame({"id": [1, 2, 3], "message": ["hello", "ducklake", "sdk"]})
table.sink_polars(lf)

# Read it back as a Polars LazyFrame
df = table.scan_polars().collect()

For the full API, see the Python documentation or the Rust API docs.

Features

The Rust core — and therefore every SDK built on top of it — supports:

The Python SDK additionally provides:

  • Reading and writing data through Polars
  • Reading, writing, and deleting data through DuckDB
  • Maintenance operations — compaction, snapshot expiration, and more — via DuckDB

Compatibility Matrix

Catalog Databases

Database Status
SQLite ✅
Postgres ✅
MySQL 🟧 (no data inlining*)

*Data inlining for MySQL is not defined in the DuckLake specification.

Storage Backends

Backend Status
Local / NFS ✅
AWS S3-compatible ✅
Google Cloud Storage ✅
Azure Blob Storage ✅

DuckLake Specification Versions

Version Status
1.0 ✅ (actively supported)
0.4 ⬆️ (requires migration)
0.3 ⬆️ (requires migration)
0.2 ⬆️ (requires migration)
0.1 ⬆️ (requires migration)

See the DuckLake release calendar for upcoming versions.

Project Status

[!NOTE] This project is in alpha. It will move to beta once the full specification is implemented, and to stable once all relevant limitations have been addressed. Expect occasional breaking changes until then.

Not yet implemented from the specification

  • GEOMETRY and VARIANT data types
  • Mapping columns by name (Parquet files must currently carry field IDs)
  • Views, macros, sort info, and encrypted files

Known limitations

Rust SDK (may impact efficiency):

  • Tables partitioned with a non-identity transform do not benefit from file pruning yet.
  • Filters are not pushed down into the metadata query. Statistics are still loaded eagerly and used by readers to prune files, but the metadata query may transmit more data than necessary.

Python SDK:

Contributing

Contributions, bug reports, and feature requests are very welcome. See the contribution guidelines to get started.

License

Licensed under the MIT License.

Project links

Data verified by PyPI on Aug 19, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.0.15)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for borchero from gravatar.com borchero

Credits

Author: Oliver Borchert

GitHub Statistics

Data verified by PyPI on Aug 19, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.10

Provides Extra

arrow polars sqlalchemy

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ducklake_sdk-0.0.14.tar.gz (212.1 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ducklake_sdk-0.0.14-cp310-abi3-win_amd64.whl (10.3 MB view details)

Uploaded CPython 3.10+Windows x86-64

ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ducklake_sdk-0.0.14-cp310-abi3-macosx_11_0_arm64.whl (8.8 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ducklake_sdk-0.0.14-cp310-abi3-macosx_10_12_x86_64.whl (9.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ducklake_sdk-0.0.14.tar.gz.

File metadata

  • Download URL: ducklake_sdk-0.0.14.tar.gz
  • Upload date:
  • Size: 212.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ducklake_sdk-0.0.14.tar.gz
Algorithm Hash digest
SHA256 fea39112048fa494f5790bab435b4b54d0744c27ebe71d13f8011495e7aee6bd
MD5 3eae7eac8689ee42472c78fc067f8150
BLAKE2b-256 e1a7d45eae55d155cad746a40883e637fe0a9d95ec5dec5415d66ba5dfaa1d22

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14.tar.gz:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.14-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.14-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 47b90c5918b46ce6f61d1fc25472daf0a86d99b512a5d1e5967b02f351efb039
MD5 a44fc228815ef5821e82f2853ab69191
BLAKE2b-256 65daace75218568496c0ea239ce54472d0bfcb130fbe335bf1cdd5aafe6e88d8

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14-cp310-abi3-win_amd64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 63f381d0a9b66049019e0a5b09068e13c9c74027e9de9cb0cf6e24b62f76ee86
MD5 d98df056339833aee66041eb33b0fb5f
BLAKE2b-256 ee9079d06ac5fa8e7132673ad8adf54b83e00505bc440a6f6b961005afadf9cc

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 53980679d90d1a03ba55fa70abbad154cd5be5dffc4d06ee4510563f3e927b77
MD5 47066c844efc06d1f57aa792331e1f9f
BLAKE2b-256 ed76243a5005ea0c52f6958c50ce9d308861804ccd22fbc5f2f9850ae7ab7787

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.14-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.14-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9dc438dd51db858645007486cd8afd8e12490b44aebf021f9849d23245c7185f
MD5 e00064f199fa96a39d61f8ceee149399
BLAKE2b-256 d22ac0afd5a2f151af6b9614ee9556818620115fb62f64582328648016fd799d

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.14-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.14-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 eafe9cb5b4d2ea800db6e6db485902bae581173eaa6a48951b72145beb44b4ad
MD5 c50b03a1ac677c0cbf2f32d00384099d
BLAKE2b-256 a7057679ffc314218faa0bb4296009223c974ddfba0b92fec20cf2f6a96c9d26

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.14-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.0.15

6 files

This release

0.0.14 This release

6 files

0.0.13

6 files

0.0.12

6 files

0.0.11

6 files

0.0.10

6 files

0.0.9

6 files

0.0.8

6 files

0.0.7

6 files

0.0.6

6 files

0.0.5

6 files

0.0.4

6 files

0.0.3

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page