Deltalake-python
Native Delta Lake Python binding based on delta-rs with Pandas integration.
Example
from deltalake import DeltaTable
dt = DeltaTable("../rust/tests/data/delta-0.2.0")
dt.version()
3
dt.file_uris()
['s3://bucket/table/part-00000-cb6b150b-30b8-4662-ad28-ff32ddab96d2-c000.snappy.parquet',
's3://bucket/table/part-00000-7c2deba3-1994-4fb8-bc07-d46c948aa415-c000.snappy.parquet',
's3://bucket/table/part-00001-c373a5bd-85f0-4758-815e-7eb62007a15c-c000.snappy.parquet']
See the user guide for more examples.
Installation
# with pip
pip install deltalake
# with uv
uv add deltalake
# with poetry
poetry add deltalake
NOTE: official binary wheels are linked against openssl statically for remote object store communication. Please file a Github issue to request a critical openssl upgrade.
Tracing and Observability
Delta-rs supports OpenTelemetry tracing for performance analysis and debugging.
Basic Example
import os
import deltalake
from deltalake import write_deltalake, DeltaTable
# Enable logging to see trace output in stdout
os.environ["RUST_LOG"] = "deltalake=debug"
# Initialize tracing (uses default HTTP endpoint or OTEL_EXPORTER_OTLP_ENDPOINT env var)
# For authentication, set OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
# The HTTP exporter automatically reads OTEL_EXPORTER_OTLP_HEADERS for API keys
deltalake.init_tracing()
# All Delta operations are now traced
write_deltalake("my_table", data)
dt = DeltaTable("my_table")
df = dt.to_pandas()
When you run this code, you'll see trace information in stdout showing operation timings and execution flow.
Build custom wheels
Sometimes you may wish to build custom wheels. Maybe you want to try out some unreleased features. Or maybe you want to tweak the optimization of the Rust code.
To compile the package, you will need the Rust compiler and maturin:
curl https://sh.rustup.rs -sSf | sh -s
Then you can build wheels for your own platform like so:
uvx maturin build --release --out wheels
Note:
uvxinvokes a tool without installing it.- if you plan to often use
maturin, you can install the "tool" withuv tool install maturin.
For a build that is optimized for the system you are on (but sacrificing portability):
RUSTFLAGS="-C target-cpu=native" uvx maturin build --release --out wheels
Cross compilation
The above command only works for your current platform. To create wheels for other
platforms, you'll need to cross compile. Cross compilation requires installing
two additional components: to cross compile Rust code, you will need to install
the target with rustup; to cross compile the Python bindings, you will need
to install ziglang.
The following example is for manylinux2014. Other targets will require different
Rust target and Python compatibility tags.
rustup target add x86_64-unknown-linux-gnu
Then you can build wheels for the target platform like so:
uvx --from 'maturin[zig]' maturin build --release --zig \
--target x86_64-unknown-linux-gnu \
--compatibility manylinux2014 \
--out wheels
If you expect to only run on more modern system, you can set a newer target-cpu
flag to Rust and use a newer compatibility tag for Linux. For example, here
we set compatibility with CPUs newer than Haswell (2013) and Linux OS with
glibc version of at least 2.24:
RUSTFLAGS="-C target-cpu=haswell" uvx --from 'maturin[zig]' maturin build --release --zig \
--target x86_64-unknown-linux-gnu \
--compatibility manylinux_2_24 \
--out wheels
See note about RUSTFLAGS from the arrow-rs readme.
5 maintainers
Metadata
5 maintainers
Release files for deltalake 1.6.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| deltalake-1.6.3.tar.gz | 5.6 MB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| deltalake-1.6.3-cp310-abi3-win_amd64.whl | CPython 3.10 | abi3 | Windows x86-64 | Details |
| deltalake-1.6.3-cp310-abi3-musllinux_1_2_x86_64.whl | CPython 3.10 | abi3 | Linux musl 1.2+ x86-64 | Details |
| deltalake-1.6.3-cp310-abi3-musllinux_1_2_aarch64.whl | CPython 3.10 | abi3 | Linux musl 1.2+ ARM64 | Details |
| deltalake-1.6.3-cp310-abi3-manylinux_2_28_aarch64.whl | CPython 3.10 | abi3 | Linux glibc 2.28+ ARM64 | Details |
| deltalake-1.6.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.10 | abi3 | Linux glibc 2.17+ x86-64 | Details |
| deltalake-1.6.3-cp310-abi3-macosx_11_0_arm64.whl | CPython 3.10 | abi3 | macOS 11.0+ ARM64 | Details |
| deltalake-1.6.3-cp310-abi3-macosx_10_12_x86_64.whl | CPython 3.10 | abi3 | macOS 10.12+ x86-64 | Details |
Total release size: 364.0 MB
Release files / deltalake-1.6.3.tar.gz
| Download URL | deltalake-1.6.3.tar.gz |
|---|---|
| Size | 5.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b7176c9d2a48b3ad339902e6216cdde813857b9628ef53e972889c71d2836fc4
|
|
BLAKE2b-256 checksum How to use checksums |
08ca62354ebb384806b3496b25708ac605af54c5ed11cb2a8111ea1892aec9c9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-win_amd64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-win_amd64.whl |
|---|---|
| Size | 55.4 MB |
| Tags | CPython 3.10 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
7920a60510376f3c90feb62d182063b0202f117411e73ff9f23ffe969b800dce
|
|
BLAKE2b-256 checksum How to use checksums |
639128cb7e0edf10f05f1025e2b4bb0085a1b23aa11472c06e43f1c5897c183e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-musllinux_1_2_x86_64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 51.9 MB |
| Tags | CPython 3.10 Linux musl 1.2+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
b51d0109b0a86b336c7254a01495536d69c72593663d8de0bc320aa4523c0502
|
|
BLAKE2b-256 checksum How to use checksums |
5ec25cf8b65d6ddb2a3b0b0b1f4359ddbfba1afa93764bb6e19fc4d31dd8dba8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-musllinux_1_2_aarch64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-musllinux_1_2_aarch64.whl |
|---|---|
| Size | 50.8 MB |
| Tags | CPython 3.10 Linux musl 1.2+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
8f2de230b5cd979b3b984db3b9a24bf14722d4b46a4d062b41fe3fd2e5bfbd68
|
|
BLAKE2b-256 checksum How to use checksums |
7ae0038b81370988e47e80e8e243355a49c3f58c20624375df6c5140f922dfd1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-manylinux_2_28_aarch64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 50.8 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
d5aa5f89367deca941ed5e8496b6f6521b42ff0380609ff7a02f771ae9ae7abe
|
|
BLAKE2b-256 checksum How to use checksums |
6a5a18a21ca1b75228e2a5655616e2493f7dfe7c75397c5b7a616299342feeae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 51.9 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
1a5163ed5550f5b83ba6f9a95b5e3dff09934a3165fb80ee0ab8e5919782ac6c
|
|
BLAKE2b-256 checksum How to use checksums |
9bd868505832d7fdf3cae5f4bcf5bb1d7fb31a8e07f909515bf845b09b3ec58c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-macosx_11_0_arm64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 47.0 MB |
| Tags | CPython 3.10 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8cec745b5dff6581c0b26c5136e3f9eea7ac715a684fe4459b5c37c66407ae02
|
|
BLAKE2b-256 checksum How to use checksums |
505456c21b2c7dcee35ad84fcd4aeb73043440320e7ed0114185610a4dd96e14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
Release files / deltalake-1.6.3-cp310-abi3-macosx_10_12_x86_64.whl
| Download URL | deltalake-1.6.3-cp310-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 50.7 MB |
| Tags | CPython 3.10 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
c25928df2a30a19dec3f86ffd931fd59b7f175b5781ba998b34001eb4d49e1f7
|
|
BLAKE2b-256 checksum How to use checksums |
e826048812e37921a88935fc75811905dab15a97fc76570a7f6716e68b855cce
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.14.1
|
