Skip to main content

Native Delta Lake Python binding based on delta-rs with Pandas integration

Project description

Deltalake-python

PyPI userdoc apidoc

Native Delta Lake Python binding based on delta-rs with Pandas integration.

Example

from deltalake import DeltaTable
dt = DeltaTable("../rust/tests/data/delta-0.2.0")
dt.version()
3
dt.files()
['part-00000-cb6b150b-30b8-4662-ad28-ff32ddab96d2-c000.snappy.parquet',
 'part-00000-7c2deba3-1994-4fb8-bc07-d46c948aa415-c000.snappy.parquet',
 'part-00001-c373a5bd-85f0-4758-815e-7eb62007a15c-c000.snappy.parquet']

See the user guide for more examples.

Installation

# with pip
pip install deltalake
# with uv
uv add deltalake
# with poetry
poetry add deltalake

NOTE: official binary wheels are linked against openssl statically for remote objection store communication. Please file Github issue to request for critical openssl upgrade.

Build custom wheels

Sometimes you may wish to build custom wheels. Maybe you want to try out some unreleased features. Or maybe you want to tweak the optimization of the Rust code.

To compile the package, you will need the Rust compiler and maturin:

curl https://sh.rustup.rs -sSf | sh -s

Then you can build wheels for your own platform like so:

```sh
uvx maturin build --release --out wheels

Note:

  • uvx invokes a tool without installing it.
  • if you plan to often use maturin, you can install the "tool" with uv tool install maturin.

For a build that is optimized for the system you are on (but sacrificing portability):

RUSTFLAGS="-C target-cpu=native" uvx maturin build --release --out wheels

Cross compilation

The above command only works for your current platform. To create wheels for other platforms, you'll need to cross compile. Cross compilation requires installing two additional components: to cross compile Rust code, you will need to install the target with rustup; to cross compile the Python bindings, you will need to install ziglang.

The following example is for manylinux2014. Other targets will require different Rust target and Python compatibility tags.

rustup target add x86_64-unknown-linux-gnu

Then you can build wheels for the target platform like so:

uvx --from 'maturin[zig]' maturin build --release --zig \
    --target x86_64-unknown-linux-gnu \
    --compatibility manylinux2014 \
    --out wheels

If you expect to only run on more modern system, you can set a newer target-cpu flag to Rust and use a newer compatibility tag for Linux. For example, here we set compatibility with CPUs newer than Haswell (2013) and Linux OS with glibc version of at least 2.24:

RUSTFLAGS="-C target-cpu=haswell" uvx --from 'maturin[zig]' maturin build --release --zig \
    --target x86_64-unknown-linux-gnu \
    --compatibility manylinux_2_24 \
    --out wheels

See note about RUSTFLAGS from the arrow-rs readme.

Project details


Release history Release notifications | RSS feed

This version

1.0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

deltalake-1.0.2.tar.gz (5.1 MB view details)

Uploaded Source

Built Distributions

deltalake-1.0.2-cp39-abi3-win_amd64.whl (42.6 MB view details)

Uploaded CPython 3.9+Windows x86-64

deltalake-1.0.2-cp39-abi3-manylinux_2_28_aarch64.whl (40.3 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

deltalake-1.0.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (51.3 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

deltalake-1.0.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.3 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

deltalake-1.0.2-cp39-abi3-macosx_11_0_arm64.whl (38.6 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

deltalake-1.0.2-cp39-abi3-macosx_10_12_x86_64.whl (41.6 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file deltalake-1.0.2.tar.gz.

File metadata

  • Download URL: deltalake-1.0.2.tar.gz
  • Upload date:
  • Size: 5.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.6.0

File hashes

Hashes for deltalake-1.0.2.tar.gz
Algorithm Hash digest
SHA256 fbe4cccde0af14c6e30b62cc3dd09e9a46777e8fd8e375ec809a6bf4edea756c
MD5 8988f86f0240ae289c9394c57c3d584d
BLAKE2b-256 1ec319cd8457243c41aa60562d28b66271ff958d896e3fd9373816d8fd781f1a

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: deltalake-1.0.2-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 42.6 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.8.6

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 7a1606f535416d4a38ce554019f9fcad194aaec33d638328662b2de46af03059
MD5 8f630e5c1ca6973b284c29d48c37c544
BLAKE2b-256 78a99014b804f947a505c21a6c0cbc87e2673cacb6cd82ac70be9a60f26a836b

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 67d3224ce7e569bbb6d5181f9ed2530b237a1cdc87f413e5ff0bc1227aab50d5
MD5 e3951f5ecd7600f6fd817e80055c20e9
BLAKE2b-256 5c2a1dfc1f337f85d62141b4e70923b923d5faccbe666d4253b670c6d506d1bb

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 59a3b403e5871d12920798d27f2b1e4b70f4e975381841066cb6733ccbc80071
MD5 abfe0587898d9e5cf969e0766bd469bb
BLAKE2b-256 dc6f4707d7511bd172f6c6504e87ea0bc43cdf7b5a4c85340ff61cee83170e37

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 c921b47e4810a346650141dae30abc69564e57f26e00cce256f1837dd9c4b5fd
MD5 62bc67d72d0a6b0c45307c07e6ec2042
BLAKE2b-256 6ff1feee0df833eed13a27aafeedfac313c0b6bf7b0d712fa5892b1099a7a752

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 43731c48657c16c1728c90270e5e7ae1f3fa1a5b6fb0cb0b55c88c5c8f23cc3f
MD5 e6e8b73d0fda00998a56417f1854c4f6
BLAKE2b-256 f999ced0f538deacdf0f1e78e28a14c30420d8df1c7d9ca30ff8f71a03a008a7

See more details on using hashes here.

File details

Details for the file deltalake-1.0.2-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for deltalake-1.0.2-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e4f24cdbadaf8a4c32ae535a44b89d8bcafd5cb97897de33a4ec8609058a7d50
MD5 50095605c12da3782e67fafdd99132aa
BLAKE2b-256 7874043f52f50cbda7f651d39465fb7c5a9e8880e9a332abbb4f64c4d0522306

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page