Skip to main content

Native Delta Lake Python binding based on delta-rs with Pandas integration

Project description

Deltalake-python

PyPI userdoc apidoc

Native Delta Lake Python binding based on delta-rs with Pandas integration.

Example

from deltalake import DeltaTable
dt = DeltaTable("../rust/tests/data/delta-0.2.0")
dt.version()
3
dt.files()
['part-00000-cb6b150b-30b8-4662-ad28-ff32ddab96d2-c000.snappy.parquet',
 'part-00000-7c2deba3-1994-4fb8-bc07-d46c948aa415-c000.snappy.parquet',
 'part-00001-c373a5bd-85f0-4758-815e-7eb62007a15c-c000.snappy.parquet']

See the user guide for more examples.

Installation

# with pip
pip install deltalake
# with uv
uv add deltalake
# with poetry
poetry add deltalake

NOTE: official binary wheels are linked against openssl statically for remote objection store communication. Please file Github issue to request for critical openssl upgrade.

Build custom wheels

Sometimes you may wish to build custom wheels. Maybe you want to try out some unreleased features. Or maybe you want to tweak the optimization of the Rust code.

To compile the package, you will need the Rust compiler and maturin:

curl https://sh.rustup.rs -sSf | sh -s

Then you can build wheels for your own platform like so:

```sh
uvx maturin build --release --out wheels

Note:

  • uvx invokes a tool without installing it.
  • if you plan to often use maturin, you can install the "tool" with uv tool install maturin.

For a build that is optimized for the system you are on (but sacrificing portability):

RUSTFLAGS="-C target-cpu=native" uvx maturin build --release --out wheels

Cross compilation

The above command only works for your current platform. To create wheels for other platforms, you'll need to cross compile. Cross compilation requires installing two additional components: to cross compile Rust code, you will need to install the target with rustup; to cross compile the Python bindings, you will need to install ziglang.

The following example is for manylinux2014. Other targets will require different Rust target and Python compatibility tags.

rustup target add x86_64-unknown-linux-gnu

Then you can build wheels for the target platform like so:

uvx --from 'maturin[zig]' maturin build --release --zig \
    --target x86_64-unknown-linux-gnu \
    --compatibility manylinux2014 \
    --out wheels

If you expect to only run on more modern system, you can set a newer target-cpu flag to Rust and use a newer compatibility tag for Linux. For example, here we set compatibility with CPUs newer than Haswell (2013) and Linux OS with glibc version of at least 2.24:

RUSTFLAGS="-C target-cpu=haswell" uvx --from 'maturin[zig]' maturin build --release --zig \
    --target x86_64-unknown-linux-gnu \
    --compatibility manylinux_2_24 \
    --out wheels

See note about RUSTFLAGS from the arrow-rs readme.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

deltalake-1.1.0.tar.gz (5.1 MB view details)

Uploaded Source

Built Distributions

deltalake-1.1.0-cp39-abi3-win_amd64.whl (42.7 MB view details)

Uploaded CPython 3.9+Windows x86-64

deltalake-1.1.0-cp39-abi3-manylinux_2_28_aarch64.whl (40.7 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.28+ ARM64

deltalake-1.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (51.8 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

deltalake-1.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (40.7 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

deltalake-1.1.0-cp39-abi3-macosx_11_0_arm64.whl (39.1 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

deltalake-1.1.0-cp39-abi3-macosx_10_12_x86_64.whl (41.9 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file deltalake-1.1.0.tar.gz.

File metadata

  • Download URL: deltalake-1.1.0.tar.gz
  • Upload date:
  • Size: 5.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.6.0

File hashes

Hashes for deltalake-1.1.0.tar.gz
Algorithm Hash digest
SHA256 b3b5d29e8e854c6480c4618fb11101259f6cf4294ae1fe88073c0d18e5c8e95f
MD5 c513dc972fe50d742223404100d5eaab
BLAKE2b-256 f9ee2fb4d65acf2703ecf91e7ec704a11b72bb6fd000eecae3590ef4bb3c48a6

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: deltalake-1.1.0-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 42.7 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: maturin/1.9.1

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 dc515d641f6d3c7e88483e02c2619268e6cdfc428515a9d3ea0457671afc69c6
MD5 5b824ead90339670c7bfc7037412d79e
BLAKE2b-256 33835cf3fc7eb433c5f97cae9db0defb11940ffdddcdafa6c5ce2d7ebb34f452

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 da44072892218ef148a4d756698acd2c13790ead618227937122f9f49013c690
MD5 c1bf8ed055ef7e8aabf045c0b98fac88
BLAKE2b-256 538b83c976a91cdb1c8116164a2be541855a8e44367f92858bcefcbe692fa5b3

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 cb5024270c87a0cbc708d73f42994cc1942fee882f573bba7ad4b5fa5f2a84d3
MD5 4aab4c2e4b18e003610134d232d34bf0
BLAKE2b-256 8079ba4e2a83ce8b368606795fb3ee168fa5b3e0db438ac62268b373b9a1c3fa

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ce14441f5ae62bf542d61d41e1c8f6668cffca1432912accb6dadaa425896664
MD5 01615fbd69a0c98f81a7121a77c5f946
BLAKE2b-256 16b643ccc93e1f5cbf152cf4bd02ff3ce154e5724488b7d2f41fd18d471cd20c

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4d587f407e03d1dad5df75d55938cb2d74a02e743557158423dfd03b0b3f49d7
MD5 7939a57e74c7d759d802e4403ed7149a
BLAKE2b-256 a34010f177c6fbf1e14c1222fbdb0308c13953f5201e9b21a52f5f0862a9cc76

See more details on using hashes here.

File details

Details for the file deltalake-1.1.0-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for deltalake-1.1.0-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 323aa5024e3d857b5770466c211b37009891c5ee8266feed4ff95279f49ac657
MD5 b68842907405f7fe09a57981ddb8e5a2
BLAKE2b-256 c681daebfa2455937102967f057cf3b1bad00251cb2c3c0df994ca1f64152693

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page