Skip to main content

Data lake operations toolkit

Project description

LakeOps

PyPI version Python Versions Tests codecov

A modern data lake operations toolkit working with multiple table formats (Delta, Iceberg, Parquet) and engines (Spark, Polars) via the same APIs.

Features

  • Multi-format support: Delta, Iceberg, Parquet
  • Multiple engine backends: Apache Spark, Polars (default)
  • Storage operations: read, write

To learn more, read the user guide.

Quick Start

Installation

pip install lakeops

Sample Usage

from pyspark.sql import SparkSession
from lakeops import LakeOps
from lakeops.core.engine import SparkEngine

# Init Spark session and create LakeOps instance
spark = SparkSession.builder.getOrCreate()
engine = SparkEngine(spark)
ops = LakeOps(engine)

# Read data from table name
df = ops.read("s3://local/test/table", format="parquet")

# Write data to table name
ops.write(df, "s3://local/test/table", format="parquet")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lakeops-1.1.0-cp312-cp312-win_amd64.whl (267.1 kB view details)

Uploaded CPython 3.12Windows x86-64

lakeops-1.1.0-cp312-cp312-manylinux_2_34_x86_64.whl (425.2 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

lakeops-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (415.8 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

lakeops-1.1.0-cp312-cp312-macosx_11_0_arm64.whl (371.5 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

lakeops-1.1.0-cp311-cp311-win_amd64.whl (269.1 kB view details)

Uploaded CPython 3.11Windows x86-64

lakeops-1.1.0-cp311-cp311-manylinux_2_34_x86_64.whl (425.3 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

lakeops-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (417.0 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ ARM64

lakeops-1.1.0-cp311-cp311-macosx_11_0_arm64.whl (373.2 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

lakeops-1.1.0-cp310-cp310-win_amd64.whl (269.2 kB view details)

Uploaded CPython 3.10Windows x86-64

lakeops-1.1.0-cp310-cp310-manylinux_2_34_x86_64.whl (425.5 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

lakeops-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (417.2 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ ARM64

lakeops-1.1.0-cp310-cp310-macosx_11_0_arm64.whl (373.6 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file lakeops-1.1.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.1.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 267.1 kB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.1.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 6c3fad5e337def54f52df4ff27f1d809a53ab1e6375ecdc199b1fedb15d5a4c8
MD5 683cd64314ab92db58d20fae61557ada
BLAKE2b-256 b29ccb811cadfbe2bc5c779bfb4c3acfb75cf436a9f2a54c4dfc2b1adb5cf8f5

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp312-cp312-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 6a0597bf97f738437ca27506d5b2847d223cf99c2272b83fed21701253eda9a9
MD5 48e471197ab3be7be428b6e6ea4d1f0a
BLAKE2b-256 4957c1587c018ce9b0283fed98baad01d0b6edd08cf5854c211a839ffca78140

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp312-cp312-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 458043a02c4952838eeb90d83844b31082ae2f7c91857b099ac7282d3f468bc2
MD5 cef3e43324ae0a41c748f86bcde4a762
BLAKE2b-256 a5078251f7eba474ca805d3417841786f8c680b77c432b7c047917c08de74f27

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7695d6ce410e76a51123764e87dda792e51a35d70428c2f2fa5fba19845b2c0b
MD5 5cd41ae9218009b9709e11201691ef9a
BLAKE2b-256 d561a8cabd8bdc2d41edbc98e085a85d6d37906541f2a794ef6cb20cb1fedc07

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.1.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 269.1 kB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.1.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 6ab412ab49f17ce31824c682c7604dfd5f3bab34a300c6db5d1997ef5f3ea15f
MD5 8d316bda82cee687f3246a153fa5c4d1
BLAKE2b-256 31b580e89f67606cb1b2fd92b1ca7eee2105d65898bbde34773200d36d0e182b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp311-cp311-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 7f2b1114e877ad366b9f4396282f74fc90b4b05b49d529608a1f862a247c1af3
MD5 ea693a2288487866ea8a3541007574ff
BLAKE2b-256 09ede6dd2a614adbac31bad153e886d958e95a171957ce0fd842df60bd806ec6

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp311-cp311-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 2e5ccf05549e93f2078b8d0a64ec2506451ad4d213ec22c169389b22fb7c9a36
MD5 7f352a4da1584173a60299e8dbf857a4
BLAKE2b-256 c6ede15b0f692ca3971760f10324bfd84fb2f8766fcc10f95cb03b25e6801e2a

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a4862ed7de2abfb7c05d133e5158a3a4f014a42033f855dee01fba6d2e1c67da
MD5 3298293fedc4d90ad269bb3ef2629c30
BLAKE2b-256 738289e1f391187440ff89fe55bd32f1a4c3b0c148180735da8d4675a42cd26b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.1.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 269.2 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.1.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 5e65a2f77cf95e4d50f5e367106a7fd286d7c7dea6185bf17a95a51663e750a4
MD5 69f2d1098ee468119fe825b1b065c53b
BLAKE2b-256 b45271442c87b4511e406e57e582aaff6fa447465d416bcd2d4f7366aff2319b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp310-cp310-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 db4c76fbee6357455aa4fb8c56b64023c09fa610214684ed7f36c5a7d4cacc81
MD5 a118f812672900cd7ed2fcd19b3bdce1
BLAKE2b-256 8e673a6a8c38e3753cca63d25e85582698d91d1d3e76a3a20640c1fb13df57b8

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp310-cp310-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 345fcd06ef2b28d154f9316d581cb15f0c3f4217524c90925fa9723f4a5be929
MD5 a0b57758df8525cf4bcbe57593410a7e
BLAKE2b-256 07690e40ce72d216d77d19d5b33ddd06313a7d86747de79dcc2bce741347b4ab

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.1.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.1.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8b0b1f3ce456238a8a8370e803a73ffacd7a61b82b89cad1502b6511f3d6799c
MD5 3295b8432f8c2fadc8578eaa8d4c1398
BLAKE2b-256 b4ecd715cef4dce6058e398c4402c482ca4f56a801c6c8e5ce03114fb69dec7d

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.1.0-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page