Skip to main content

Data lake operations toolkit

Project description

LakeOps

PyPI version Python Versions Tests codecov

A modern data lake operations toolkit working with multiple table formats (Delta, Iceberg, Parquet) and engines (Spark, Polars) via the same APIs.

Features

  • Multi-format support: Delta, Iceberg, Parquet
  • Multiple engine backends: Apache Spark, Polars (default)
  • Storage operations: read, write

To learn more, read the user guide.

Quick Start

Installation

pip install lakeops

Sample Usage

from pyspark.sql import SparkSession
from lakeops import LakeOps
from lakeops.core.engine import SparkEngine

# Init Spark session and create LakeOps instance
spark = SparkSession.builder.getOrCreate()
engine = SparkEngine(spark)
ops = LakeOps(engine)

# Read data from table name
df = ops.read("s3://local/test/table", format="parquet")

# Write data to table name
ops.write(df, "s3://local/test/table", format="parquet")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lakeops-0.2.0-cp312-cp312-win_amd64.whl (257.7 kB view details)

Uploaded CPython 3.12Windows x86-64

lakeops-0.2.0-cp312-cp312-manylinux_2_34_x86_64.whl (409.9 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

lakeops-0.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (403.8 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

lakeops-0.2.0-cp312-cp312-macosx_11_0_arm64.whl (365.1 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

lakeops-0.2.0-cp311-cp311-win_amd64.whl (257.7 kB view details)

Uploaded CPython 3.11Windows x86-64

lakeops-0.2.0-cp311-cp311-manylinux_2_34_x86_64.whl (410.4 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

lakeops-0.2.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (404.9 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ ARM64

lakeops-0.2.0-cp311-cp311-macosx_11_0_arm64.whl (367.4 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

lakeops-0.2.0-cp310-cp310-win_amd64.whl (257.8 kB view details)

Uploaded CPython 3.10Windows x86-64

lakeops-0.2.0-cp310-cp310-manylinux_2_34_x86_64.whl (410.8 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

lakeops-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (405.0 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ ARM64

lakeops-0.2.0-cp310-cp310-macosx_11_0_arm64.whl (367.7 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file lakeops-0.2.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: lakeops-0.2.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 257.7 kB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for lakeops-0.2.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 0252137b53e67188acbb35dfb942a3f4a338a4e1bf5652bbb71b038cd31e2cbf
MD5 ce2845c7308f6ac3e18c71e88742b873
BLAKE2b-256 264d50635efe62990dccd3347bcb3deb76b369499220f39ccb73178bcc6f56a2

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp312-cp312-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 f1068086b7f04fc5e3dd75065044b24d04a4f8cd2225108384b45f6de0ed3fee
MD5 7c7e94beb776bc605684b06dd0fdddc3
BLAKE2b-256 28e90d221ac2e1a9a82e449aef75a5b389c3c4273fa17907439c19a39efff5c4

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp312-cp312-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 742a2b160eb14eefa6dd57b9072f72c601faa1c4c296093a80c17c8a81a3cad4
MD5 2f406abd29216795b255083304ea73b3
BLAKE2b-256 ca6532eeaf5f5dc3e8bb67af1a348682fa3c78ad267aad4a62d50d523b6ddc8d

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0577ef550e24cfaa8b94c901b782e3531814f941b48c5d10f575169bd40a043e
MD5 465be482de5203ae18dbe25e7590020c
BLAKE2b-256 a6792fe1774459f968cd15bddd142d259cf76ac240cf40e069356b9e99f6cf07

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: lakeops-0.2.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 257.7 kB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for lakeops-0.2.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 1d0c2a8cafc76e3cf1742cd67c886dac02479870b059a45e5b76bc4585e12ad8
MD5 ca75f4442c98760db0ea328a53e951f1
BLAKE2b-256 ae8f69af7b78e9ebacef6bec885224dec2b4712f5ef4666b6f11f92dda7e95e6

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp311-cp311-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 a8b95f4abfcbc17ed68461c721b63d327b5c4fc02708efa7bc563ba92ef156e3
MD5 44b987770b45bd7b0cb3181fa94d3b3e
BLAKE2b-256 c48d55aa063f92da10fd231097d1541a49bc9f58bba9b98502b3249de90404b7

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp311-cp311-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f32cf13df799e2ca74f6ede8a8adfc25014da876cc8adae0e66f0c866dabf7c9
MD5 b45cd16d6197824a03a7242706278e0c
BLAKE2b-256 1b58bcabf4b0c79dad94d7a74114780275ec5ed72bda57d8a3dd680c3c41bbc8

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 aeed89a6e93a177bc2032d174c26ae335be9bfb617c65a1269676cb7e6f5e864
MD5 a3ab10166ccd4740ee2408bc38ae95db
BLAKE2b-256 d47f2273965f3420bf2fec2123f10354fd8a981f29d2549e2b98e2d1133d5d55

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: lakeops-0.2.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 257.8 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for lakeops-0.2.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 66600fdf78ebaf668bf98504709f58a4276e914f7e7becc6a081c475092dc8cc
MD5 22e2a91388254693add84b0e5a60cb26
BLAKE2b-256 3a6949b5d0699947d1dca4ab682a187c627fd453fafc328a0f327589b8dc4ebf

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp310-cp310-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 488f27f0e1cdffc1766bad36b6286f53cd9102dc33009840b4df03915d4cd60d
MD5 8b92811fe82ffabe9b9621f0850141b7
BLAKE2b-256 2f6c763fd12bccc85b3da184f632e609afe6e2e5634822ca41b3c9a31eb55251

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp310-cp310-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 09adcf65a18b10d413d0f36b6d022565be19e1383c52c51a1520a0c8ebd52927
MD5 b9517e25665ae09b8204d2cc04ea3399
BLAKE2b-256 e6eeda16a853e8625a0a171113cb78fa37945e83bea96f8e59b55d841cd50013

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-0.2.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-0.2.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0b3bccbe790d850eaabb84660443fa412b51286858dce862f48e51aef6a1d6aa
MD5 2ff35f1413a5d401320e86a918dc894a
BLAKE2b-256 45e07cc33f2dadad9caf264897231f8b9ad61606fc75b95f8350c44bdec3b9ae

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-0.2.0-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page