ducklake-sdk — Native SDKs for DuckLake

Read and write DuckLake tables from Rust and Python - no DuckDB required.

CI Build GitHub Release

DuckLake is an integrated data lake and catalog format that stores metadata in a SQL catalog database and writes data as Parquet files. This repository provides standalone Rust and Python SDKs that talk to DuckLakes directly, with no dependency on DuckDB or its DuckLake extension.

All language SDKs are built on the same Rust core, which bundles the implementation of the DuckLake specification.

Python (ducklake-sdk)

pypi-version conda-forge python-version readthedocs codecov

Rust (ducklake)

crates.io docs.rs codecov

[!WARNING] This is not an official SDK released by the DuckDB Foundation.

Getting Started

Python

pip install ducklake-sdk            # core
pip install "ducklake-sdk[polars]"  # Polars integration
pip install "ducklake-sdk[arrow]"   # Arrow + DuckDB integration

Rust

cargo add ducklake

Quick Example

import ducklake as dl
import polars as pl

# Create a new DuckLake backed by SQLite metadata and local Parquet storage
ducklake = dl.create("sqlite:///metadata.sqlite", data_path="data_files/")

# Define a table.
table = ducklake.create_table(
    "events",
    schema={"id": dl.Int64(), "message": dl.Varchar()},
)

# Write data using Polars
lf = pl.LazyFrame({"id": [1, 2, 3], "message": ["hello", "ducklake", "sdk"]})
table.sink_polars(lf)

# Read it back as a Polars LazyFrame
df = table.scan_polars().collect()

For the full API, see the Python documentation or the Rust API docs.

Features

The Rust core — and therefore every SDK built on top of it — supports:

The Python SDK additionally provides:

  • Reading and writing data through Polars
  • Reading, writing, and deleting data through DuckDB
  • Maintenance operations — compaction, snapshot expiration, and more — via DuckDB

Compatibility Matrix

Catalog Databases

Database Status
SQLite ✅
Postgres ✅
MySQL 🟧 (no data inlining*)

*Data inlining for MySQL is not defined in the DuckLake specification.

Storage Backends

Backend Status
Local / NFS ✅
AWS S3-compatible ✅
Google Cloud Storage ✅
Azure Blob Storage ✅

DuckLake Specification Versions

Version Status
1.0 ✅ (actively supported)
0.4 ⬆️ (requires migration)
0.3 ⬆️ (requires migration)
0.2 ⬆️ (requires migration)
0.1 ⬆️ (requires migration)

See the DuckLake release calendar for upcoming versions.

Project Status

[!NOTE] This project is in alpha. It will move to beta once the full specification is implemented, and to stable once all relevant limitations have been addressed. Expect occasional breaking changes until then.

Not yet implemented from the specification

  • GEOMETRY and VARIANT data types
  • Mapping columns by name (Parquet files must currently carry field IDs)
  • Views, macros, sort info, and encrypted files

Known limitations

Rust SDK (may impact efficiency):

  • Tables partitioned with a non-identity transform do not benefit from file pruning yet.
  • Filters are not pushed down into the metadata query. Statistics are still loaded eagerly and used by readers to prune files, but the metadata query may transmit more data than necessary.

Python SDK:

Contributing

Contributions, bug reports, and feature requests are very welcome. See the contribution guidelines to get started.

License

Licensed under the MIT License.

Project links

Data verified by PyPI on Aug 19, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.0.15)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for borchero from gravatar.com borchero

Credits

Author: Oliver Borchert

GitHub Statistics

Data verified by PyPI on Aug 19, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.10

Provides Extra

arrow polars sqlalchemy

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ducklake_sdk-0.0.13.tar.gz (210.9 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ducklake_sdk-0.0.13-cp310-abi3-win_amd64.whl (10.3 MB view details)

Uploaded CPython 3.10+Windows x86-64

ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ducklake_sdk-0.0.13-cp310-abi3-macosx_11_0_arm64.whl (8.8 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ducklake_sdk-0.0.13-cp310-abi3-macosx_10_12_x86_64.whl (9.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ducklake_sdk-0.0.13.tar.gz.

File metadata

  • Download URL: ducklake_sdk-0.0.13.tar.gz
  • Upload date:
  • Size: 210.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ducklake_sdk-0.0.13.tar.gz
Algorithm Hash digest
SHA256 b6fc309283be0736821252d90bb9fdf7ad822513f9033817023db7a2fbb076a3
MD5 fb2b310f53a45e3a57327e656edc4c05
BLAKE2b-256 7a6bbe45d4cc1348f4690a204e42f030023fd3d4cdacae49141e0e28c5b0aed7

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13.tar.gz:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.13-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.13-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 ff7806351ecc58fb949868ad45998dd64679ba7138d6c9f1f6c1f8d176d9602c
MD5 175deff1474c76b997ecbf3785623ee5
BLAKE2b-256 14d4fe7d98979a6c7dbe20c8ec179a12754af857091867a54e76b5c432b09d82

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13-cp310-abi3-win_amd64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6421c505ee2678699da90c8ebf8c684f429c594bce96d96dedefa7b9c225e7c4
MD5 8c2614a3e0a5be8a69d66985f02442d1
BLAKE2b-256 167b61269f7abb13ea386bb7e7635272ba70caa32a970b26642b7ab807022c43

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 8d7ef8b56027b58649ee132368f8ffa386d0a6e455ac194cf94a9d9094b9f51d
MD5 2e978a832d2cf40fac89c77ff4423388
BLAKE2b-256 917ae2c757c3a40d007ecf1c8f440b1e431a74c65313683fabe94ed0e4295c5b

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.13-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.13-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b134d204f5b36ca17beae643ce4193d78f16ba9eec54a2443d3cf34007a8e089
MD5 bca7fb1b92e5d6a86b58339986372ee6
BLAKE2b-256 052cee66261a80f6fac23649237ee633ab46af6e44bb8b9aed0a59b833f58f0a

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.13-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.13-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 76890a99973c3143a7a758b08d0657d603a0a79325a900c4cfd14cb8f30b4453
MD5 8faba02b7f9fd69dad630cb01f3a8cd5
BLAKE2b-256 e72397932daf568e2b087228f52e31f27b34f69739c4ef0ad4a68ec5a6bcdc23

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.13-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.0.15

6 files

0.0.14

6 files

This release

0.0.13 This release

6 files

0.0.12

6 files

0.0.11

6 files

0.0.10

6 files

0.0.9

6 files

0.0.8

6 files

0.0.7

6 files

0.0.6

6 files

0.0.5

6 files

0.0.4

6 files

0.0.3

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page