ducklake-sdk — Native SDKs for DuckLake

Read and write DuckLake tables from Rust and Python - no DuckDB required.

CI Build GitHub Release

DuckLake is an integrated data lake and catalog format that stores metadata in a SQL catalog database and writes data as Parquet files. This repository provides standalone Rust and Python SDKs that talk to DuckLakes directly, with no dependency on DuckDB or its DuckLake extension.

All language SDKs are built on the same Rust core, which bundles the implementation of the DuckLake specification.

Python (ducklake-sdk)

pypi-version conda-forge python-version readthedocs codecov

Rust (ducklake)

crates.io docs.rs codecov

[!WARNING] This is not an official SDK released by the DuckDB Foundation.

Getting Started

Python

pip install ducklake-sdk            # core
pip install "ducklake-sdk[polars]"  # Polars integration
pip install "ducklake-sdk[arrow]"   # Arrow + DuckDB integration

Rust

cargo add ducklake

Quick Example

import ducklake as dl
import polars as pl

# Create a new DuckLake backed by SQLite metadata and local Parquet storage
ducklake = dl.create("sqlite:///metadata.sqlite", data_path="data_files/")

# Define a table.
table = ducklake.create_table(
    "events",
    schema={"id": dl.Int64(), "message": dl.Varchar()},
)

# Write data using Polars
lf = pl.LazyFrame({"id": [1, 2, 3], "message": ["hello", "ducklake", "sdk"]})
table.sink_polars(lf)

# Read it back as a Polars LazyFrame
df = table.scan_polars().collect()

For the full API, see the Python documentation or the Rust API docs.

Features

The Rust core — and therefore every SDK built on top of it — supports:

The Python SDK additionally provides:

  • Reading and writing data through Polars
  • Reading, writing, and deleting data through DuckDB
  • Maintenance operations — compaction, snapshot expiration, and more — via DuckDB

Compatibility Matrix

Catalog Databases

Database Status
SQLite ✅
Postgres ✅
MySQL 🟧 (no data inlining*)

*Data inlining for MySQL is not defined in the DuckLake specification.

Storage Backends

Backend Status
Local / NFS ✅
AWS S3-compatible ✅
Google Cloud Storage ✅
Azure Blob Storage ✅

DuckLake Specification Versions

Version Status
1.0 ✅ (actively supported)
0.4 ⬆️ (requires migration)
0.3 ⬆️ (requires migration)
0.2 ⬆️ (requires migration)
0.1 ⬆️ (requires migration)

See the DuckLake release calendar for upcoming versions.

Project Status

[!NOTE] This project is in alpha. It will move to beta once the full specification is implemented, and to stable once all relevant limitations have been addressed. Expect occasional breaking changes until then.

Not yet implemented from the specification

  • GEOMETRY and VARIANT data types
  • Mapping columns by name (Parquet files must currently carry field IDs)
  • Views, macros, sort info, and encrypted files

Known limitations

Rust SDK (may impact efficiency):

  • Tables partitioned with a non-identity transform do not benefit from file pruning yet.
  • Filters are not pushed down into the metadata query. Statistics are still loaded eagerly and used by readers to prune files, but the metadata query may transmit more data than necessary.
  • Not tested on Windows.

Python SDK:

Contributing

Contributions, bug reports, and feature requests are very welcome. See the contribution guidelines to get started.

License

Licensed under the MIT License.

Project links

Data verified by PyPI on Aug 2, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.0.15)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for borchero from gravatar.com borchero

Credits

Author: Oliver Borchert

GitHub Statistics

Data verified by PyPI on Aug 2, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.10

Provides Extra

arrow polars sqlalchemy

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ducklake_sdk-0.0.11.tar.gz (207.0 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ducklake_sdk-0.0.11-cp310-abi3-win_amd64.whl (10.3 MB view details)

Uploaded CPython 3.10+Windows x86-64

ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ducklake_sdk-0.0.11-cp310-abi3-macosx_11_0_arm64.whl (8.7 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ducklake_sdk-0.0.11-cp310-abi3-macosx_10_12_x86_64.whl (9.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ducklake_sdk-0.0.11.tar.gz.

File metadata

  • Download URL: ducklake_sdk-0.0.11.tar.gz
  • Upload date:
  • Size: 207.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ducklake_sdk-0.0.11.tar.gz
Algorithm Hash digest
SHA256 e3f9e5e3c0aa369772a81d018efba0a654f60eeeee793de36e0daaf571947756
MD5 f91a4c15b2c3bec4c03cb5dfbb1315a8
BLAKE2b-256 60d1340da3147aea35d5705cb94e62cc91bc3c609ceb64a077094a12e0786ca3

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11.tar.gz:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.11-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.11-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 2fa362444f74b91d5d90f1fabe3585d57d34e33b5fa3aa660fb10f85aa448576
MD5 230d3076d2b6788f6e77c87395340b9b
BLAKE2b-256 2a0b3aa0d468f7233f3faf4ac312a42ee09a1c740aa937caae7a9d893c736984

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11-cp310-abi3-win_amd64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8f6074dd1ec08b736c67986adeb79d88871cd28bea61a08baa44e972a485ea44
MD5 1be5974d4e162a9ce76e73eda5b8cc1b
BLAKE2b-256 6500c9479e3509c67ccbcca9b8b6caf8fb88ab0b5940ce4e591cfe06463b3c80

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 938eccc786aa50d8bcf9b5c2bd490d25d8129c66708770c381b31b9ea375a4a3
MD5 ea3c3f20bbc8da861cb68221f6838572
BLAKE2b-256 7aa8a11b72d50bd112db177430d2ddfef609ead3b0413bbd61e3483288252ca7

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.11-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.11-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b249d7769cb48863d9bcc3524e26d1186b15e6b405d7217de4aa04089ecbea9c
MD5 0197556863663c4897044de3b09ad856
BLAKE2b-256 7b14552a3443475a17bd56c5c8890a3e72b294df6a42888dec2b0035a49723f5

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ducklake_sdk-0.0.11-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ducklake_sdk-0.0.11-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ae5fdf8c39f9a427169f6e09b215a815f8d47d52f37696707daca4ca92e49ddf
MD5 752ad13f53e41b28793d36b623d06568
BLAKE2b-256 c417a5781dacab1e8041f4a566867a38679ffe59889b1f4b3d3f020350717960

See more details on using hashes here.

Provenance

The following attestation bundles were made for ducklake_sdk-0.0.11-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: build.yml on borchero/ducklake-sdk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.0.15

6 files

0.0.14

6 files

0.0.13

6 files

0.0.12

6 files

This release

0.0.11 This release

6 files

0.0.10

6 files

0.0.9

6 files

0.0.8

6 files

0.0.7

6 files

0.0.6

6 files

0.0.5

6 files

0.0.4

6 files

0.0.3

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page