ducklake-sdk — Native SDKs for DuckLake

Read and write DuckLake tables from Rust and Python - no DuckDB required.

CI Build GitHub Release

DuckLake is an integrated data lake and catalog format that stores metadata in a SQL catalog database and writes data as Parquet files. This repository provides standalone Rust and Python SDKs that talk to DuckLakes directly, with no dependency on DuckDB or its DuckLake extension.

All language SDKs are built on the same Rust core, which bundles the implementation of the DuckLake specification.

Python (ducklake-sdk)

pypi-version conda-forge python-version readthedocs codecov

Rust (ducklake)

crates.io docs.rs codecov

[!WARNING] This is not an official SDK released by the DuckDB Foundation.

Getting Started

Python

pip install ducklake-sdk            # core
pip install "ducklake-sdk[polars]"  # Polars integration
pip install "ducklake-sdk[arrow]"   # Arrow + DuckDB integration

Rust

cargo add ducklake

Quick Example

import ducklake as dl
import polars as pl

# Create a new DuckLake backed by SQLite metadata and local Parquet storage
ducklake = dl.create("sqlite:///metadata.sqlite", data_path="data_files/")

# Define a table.
table = ducklake.create_table(
    "events",
    schema={"id": dl.Int64(), "message": dl.Varchar()},
)

# Write data using Polars
lf = pl.LazyFrame({"id": [1, 2, 3], "message": ["hello", "ducklake", "sdk"]})
table.sink_polars(lf)

# Read it back as a Polars LazyFrame
df = table.scan_polars().collect()

For the full API, see the Python documentation or the Rust API docs.

Features

The Rust core — and therefore every SDK built on top of it — supports:

The Python SDK additionally provides:

  • Reading and writing data through Polars
  • Reading, writing, and deleting data through DuckDB
  • Maintenance operations — compaction, snapshot expiration, and more — via DuckDB

Compatibility Matrix

Catalog Databases

Database Status
SQLite ✅
Postgres ✅
MySQL 🟧 (no data inlining*)

*Data inlining for MySQL is not defined in the DuckLake specification.

Storage Backends

Backend Status
Local / NFS ✅
AWS S3-compatible ✅
Google Cloud Storage ✅
Azure Blob Storage ✅

DuckLake Specification Versions

Version Status
1.0 ✅ (actively supported)
0.4 ⬆️ (requires migration)
0.3 ⬆️ (requires migration)
0.2 ⬆️ (requires migration)
0.1 ⬆️ (requires migration)

See the DuckLake release calendar for upcoming versions.

Project Status

[!NOTE] This project is in alpha. It will move to beta once the full specification is implemented, and to stable once all relevant limitations have been addressed. Expect occasional breaking changes until then.

Not yet implemented from the specification

  • GEOMETRY and VARIANT data types
  • Mapping columns by name (Parquet files must currently carry field IDs)
  • Views, macros, sort info, and encrypted files

Known limitations

Rust SDK (may impact efficiency):

  • Tables partitioned with a non-identity transform do not benefit from file pruning yet.
  • Filters are not pushed down into the metadata query. Statistics are still loaded eagerly and used by readers to prune files, but the metadata query may transmit more data than necessary.
  • Not tested on Windows.

Python SDK:

Contributing

Contributions, bug reports, and feature requests are very welcome. See the contribution guidelines to get started.

License

Licensed under the MIT License.

Project links

Data verified by PyPI on Jul 28, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.0.15)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for borchero from gravatar.com borchero

Credits

Author: Oliver Borchert

GitHub Statistics

Data verified by PyPI on Jul 28, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.10

Provides Extra

arrow polars sqlalchemy

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ducklake_sdk-0.0.10.tar.gz (205.1 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ducklake_sdk-0.0.10-cp310-abi3-win_amd64.whl (10.2 MB view details)

Uploaded CPython 3.10+Windows x86-64

ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ducklake_sdk-0.0.10-cp310-abi3-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ducklake_sdk-0.0.10-cp310-abi3-macosx_10_12_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ducklake_sdk-0.0.10.tar.gz.

File metadata

  • Download URL: ducklake_sdk-0.0.10.tar.gz
  • Upload date:
  • Size: 205.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for ducklake_sdk-0.0.10.tar.gz
Algorithm Hash digest
SHA256 f4c2adac8f83d95130329c07b91a90f3d99723e9730106fd2b7fe99e76cba7c7
MD5 dc40cf5cc759dbf664afd526455c0a0c
BLAKE2b-256 ed17b23b141afd7497c8ab833c62a171f81ea4d9ad1510ddb4f5a18f48253220

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10.tar.gz:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.10-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.10-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 a838bb242cad8a7074a710e27f6448f1a41ff44ce4ea1ccb1ba49efcba8363cc
MD5 834dc9a1696ef00effd59d60ade9d46a
BLAKE2b-256 df783ca4a690dc4a07e34d5d89593b05c24f792f0d5ed46bc069de1d9c88d376

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10-cp310-abi3-win_amd64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5c1f8488ade39f5eb76733089837a5f36cd65a19cae0efe14503d96769f09b33
MD5 17570422d3b1f415778c36658ea006f4
BLAKE2b-256 d82d79781d55dd5b7e84aeb35f1eaf5b60efde04481419b580db4dbe8c0e4dc8

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f5649d8b7366d80a5bd894b03bde277d14cee2a9374dbbbfad60a79e526ce47b
MD5 af7033f7e9ddc06eb5a2e86ab02534be
BLAKE2b-256 cec347d2ea50213345e4b5814d1f1c4c1423b8e54497e9e18ec381b09d19815c

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.10-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.10-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 88dfb49d426efc84b1a05bf37fe848522bfa27fd83d4890a56f6f311f3f15008
MD5 3164de9bf05187a55765d64d8fd38b18
BLAKE2b-256 ca4ef1763f81734e34af12abefda461ae84b94b22e8a4d777c53e862a4ce967e

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.10-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.10-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 8bacff0a32ed7b369a8e9a1a77a2305d3160c578f0f50d6ad1349786d3469f2d
MD5 23372dfca9143d3692165c794f1ef9d4
BLAKE2b-256 59c035b3e8f0363fba02bba60c11b5db5fca5312df843724b6de89360e724aec

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.10-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.0.15

6 files

0.0.14

6 files

0.0.13

6 files

0.0.12

6 files

0.0.11

6 files

This release

0.0.10 This release

6 files

0.0.9

6 files

0.0.8

6 files

0.0.7

6 files

0.0.6

6 files

0.0.5

6 files

0.0.4

6 files

0.0.3

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page