Skip to main content

Data lake operations toolkit

Project description

LakeOps

PyPI version Python Versions Tests codecov

A modern data lake operations toolkit working with multiple table formats (Delta, Iceberg, Parquet) and engines (Spark, Polars) via the same APIs.

Features

  • Multi-format support: Delta, Iceberg, Parquet
  • Multiple engine backends: Apache Spark, Polars (default)
  • Storage operations: read, write

To learn more, read the user guide.

Quick Start

Installation

pip install lakeops

Sample Usage

from pyspark.sql import SparkSession
from lakeops import LakeOps
from lakeops.core.engine import SparkEngine

# Init Spark session and create LakeOps instance
spark = SparkSession.builder.getOrCreate()
engine = SparkEngine(spark)
ops = LakeOps(engine)

# Read data from table name
df = ops.read("s3://local/test/table", format="parquet")

# Write data to table name
ops.write(df, "s3://local/test/table", format="parquet")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lakeops-1.0.0-cp312-cp312-win_amd64.whl (266.6 kB view details)

Uploaded CPython 3.12Windows x86-64

lakeops-1.0.0-cp312-cp312-manylinux_2_34_x86_64.whl (424.7 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

lakeops-1.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (415.4 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

lakeops-1.0.0-cp312-cp312-macosx_11_0_arm64.whl (371.0 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

lakeops-1.0.0-cp311-cp311-win_amd64.whl (268.7 kB view details)

Uploaded CPython 3.11Windows x86-64

lakeops-1.0.0-cp311-cp311-manylinux_2_34_x86_64.whl (424.9 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

lakeops-1.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (416.5 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ ARM64

lakeops-1.0.0-cp311-cp311-macosx_11_0_arm64.whl (372.7 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

lakeops-1.0.0-cp310-cp310-win_amd64.whl (268.8 kB view details)

Uploaded CPython 3.10Windows x86-64

lakeops-1.0.0-cp310-cp310-manylinux_2_34_x86_64.whl (425.0 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

lakeops-1.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (416.7 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ ARM64

lakeops-1.0.0-cp310-cp310-macosx_11_0_arm64.whl (373.1 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file lakeops-1.0.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 266.6 kB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 42f7d47271e04274467fb304cfbde60aeb6dabb3758f578deebb3dd21bb4080e
MD5 3bac13b4afbc2b8bed2a019b7d99f7db
BLAKE2b-256 429cf83c2b90d964f96480f8b22f221be6aa9aef3e18858149525c74c501efb5

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp312-cp312-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 bdec51af8c51ce1d5a773352138f4b1cc6a656a70f26dadb11fab789fbe7b0ab
MD5 f8c1ada9eaf64d90565508048d688447
BLAKE2b-256 51554c5e850a95c3652285af0d99febfaea74dfa2e109b464fa4fe78ee256e63

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp312-cp312-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 91cbb2d628c424c92dec0b2d06c35c17b684a42835a791f4bb878ca6610ff3e7
MD5 6f3d9dc2d6e1c41136b0c08486973257
BLAKE2b-256 84bc85e9c3aa87dd7935fdbcbe6bcc92ec9cec62b335c9af76f791f4f56bd58b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 09eab504e074662a9dbfa4e52893d87414422795ac57f44ec5a6b12aa7c76966
MD5 8f2ef40655eaae6be488bc394a51d30d
BLAKE2b-256 4689af51ea8292eaa9b72954129e000f9b6bb3413e1cf44c3bf1b530d1a16fae

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 268.7 kB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 8202ce38783dab7d82c104cc45bd01138e84f8ad0bbc0e8e56827ff3aa81bb58
MD5 05c51e2b491b16abe3e2cd92871331e6
BLAKE2b-256 596be8229f94c2f4afe76a42b9d1f58bf0745736409de60497ccd2581175d5ab

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp311-cp311-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 bcad996f792038d28ef634c2c938f8189251ccd4697ba7cf60117c908fb9aeeb
MD5 d0ab1805fa444d006b3abd4729f8d256
BLAKE2b-256 d564d1a22e8cec3388963871b553ba31e9833df662911e4bf44839fdca6bab90

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp311-cp311-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cb3f4e4716d032e1d21d382ed0ade436b316938a1271ddb99e2f2cc883320957
MD5 e503a09b72574797af92126be0306055
BLAKE2b-256 4e610c379a7b2c69ef7967196fb3efd034c458d853082ad76b70b000fb32c146

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 01cdab830a4bd5201fd8cc0dd1ffa32c02164285854ad35d4df4abefd271db44
MD5 57a95b66d3ca62f3d3cbd30bdb3013f9
BLAKE2b-256 d21e734ce701bf0cfab684efb4001512990f24281558c06ccd93d1be24e12c97

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 268.8 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 3223a2a3bdf4eae9ac640d8649d705e7be5a604034935774032dca198288159f
MD5 140c66de3a51cca189ed7a8b2d42d65b
BLAKE2b-256 2e047b3d5aed1762bc8d999ff92159a6d959449840a00642fc6735a06b189c9b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp310-cp310-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 396fbee36fde9a0934a88613c673db7880485b903a638cab2e317b189e93ac15
MD5 d33688aefeb992fdc3a50b6172f4a3e4
BLAKE2b-256 14fa692ad86878fb9c3bff2a9402f45c70acb7bc40b6f4c6ec6026bc8258df69

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp310-cp310-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 8341706f7a7acf5d8990ced4b7553eadc75f910cdd1b4b6e6e710ccf43e3c451
MD5 43da5a130f46b4a15b572a32f680e058
BLAKE2b-256 8da8616b3f0a01c3790d87dedf63ba647aac3fcde56afbadd22578a2ad91fe3b

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 029e8eb636642956d0d42a47a09c40a3316e3a06a743d00bf255c798a81cee22
MD5 99b848b905252880335616c2965177c2
BLAKE2b-256 9990a7aeafdb895c146410341cea7f1f26e146a4083fc311e9bbcc3813cdfdb6

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.0-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page