Skip to main content

Data lake operations toolkit

Project description

LakeOps

PyPI version Python Versions Tests codecov

A modern data lake operations toolkit working with multiple table formats (Delta, Iceberg, Parquet) and engines (Spark, Polars) via the same APIs.

Features

  • Multi-format support: Delta, Iceberg, Parquet
  • Multiple engine backends: Apache Spark, Polars (default)
  • Storage operations: read, write

To learn more, read the user guide.

Quick Start

Installation

pip install lakeops

Sample Usage

from pyspark.sql import SparkSession
from lakeops import LakeOps
from lakeops.core.engine import SparkEngine

# Init Spark session and create LakeOps instance
spark = SparkSession.builder.getOrCreate()
engine = SparkEngine(spark)
ops = LakeOps(engine)

# Read data from table name
df = ops.read("s3://local/test/table", format="parquet")

# Write data to table name
ops.write(df, "s3://local/test/table", format="parquet")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

lakeops-1.0.1-cp312-cp312-win_amd64.whl (266.6 kB view details)

Uploaded CPython 3.12Windows x86-64

lakeops-1.0.1-cp312-cp312-manylinux_2_34_x86_64.whl (424.7 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.34+ x86-64

lakeops-1.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (415.4 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

lakeops-1.0.1-cp312-cp312-macosx_11_0_arm64.whl (371.0 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

lakeops-1.0.1-cp311-cp311-win_amd64.whl (268.7 kB view details)

Uploaded CPython 3.11Windows x86-64

lakeops-1.0.1-cp311-cp311-manylinux_2_34_x86_64.whl (424.9 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.34+ x86-64

lakeops-1.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (416.5 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ ARM64

lakeops-1.0.1-cp311-cp311-macosx_11_0_arm64.whl (372.7 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

lakeops-1.0.1-cp310-cp310-win_amd64.whl (268.8 kB view details)

Uploaded CPython 3.10Windows x86-64

lakeops-1.0.1-cp310-cp310-manylinux_2_34_x86_64.whl (425.0 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.34+ x86-64

lakeops-1.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (416.7 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ ARM64

lakeops-1.0.1-cp310-cp310-macosx_11_0_arm64.whl (373.1 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file lakeops-1.0.1-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.1-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 266.6 kB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 3c6f513e026232349fdf51cb9caef3424922d0551acf739c613b156ff337b3b9
MD5 be6323dbaa689c94d3612a2fcca13249
BLAKE2b-256 b9b4a7a7f24745e9bde62a02503fa037f1fcab2392c1693d677faef45c1e0843

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp312-cp312-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp312-cp312-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp312-cp312-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 54201b4bef94343e843e569ac9b5069d99d7b76204910cf9831dbb907e3c34cf
MD5 239964c4aeca4ca1aaa1cdd296cc62b8
BLAKE2b-256 955f245624136b7f0c15cd5877abe8becdcef2eea44d307973d4ba17dbf9839f

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp312-cp312-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 33b37fc95b62f676da7205671a975dcdbddc0dd374a96a278fd0285e5443f16f
MD5 71b36ab474524f4662f5f9dc78cebc75
BLAKE2b-256 8224688f313e9879b219699e08c0a7f2c8c4ef07512908bf75542f944e25f89c

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 65b216c19e2e8ea94abe3316157d5a3727578a2facd0f95552b078663157b29b
MD5 20a765a7a78b7d19b560a6332399b265
BLAKE2b-256 dd2694b5c3a2eb1a4c505e61bcb292afa7624c8231a2496b2a5a12b557a3da59

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 268.7 kB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 162bbb1d4a15d3d8f6b994e98c123d3d19ad89237d1cde4515bd61f5abea21a7
MD5 39d737b0cea5991818e6f786270d642e
BLAKE2b-256 d0286cc46ea41bc737ae21838bd70f55e10d3075bc3993c8f9c2c2c9caa0bd4d

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp311-cp311-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp311-cp311-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 4103d041ed96be41aa0d6b06706905ea56540e6d6200793ba20e165310dbb724
MD5 da29d4ae0f2b0d7829657803d545d9e0
BLAKE2b-256 3b4017123d91fcdf7df703fca4a0256e81a615e67e43ade10caed23aefb90156

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp311-cp311-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 6f953b98cc45368924d07879b088681b412934314c36a4e34516328dd40a1d36
MD5 1a46a8854ca7b8344fbf412e1842c924
BLAKE2b-256 96a4903c1703acc128050bb9a8d1ddba139c5ed89e46e6b3f07f753121a05a53

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4a9d3cd8aaab063d86790d09d7289a58753e46474a63c0f215581672e41216d4
MD5 dc86d8aa5c904a7060d44bdee025e8a7
BLAKE2b-256 5dcf37dab850eeb472f2f8c868711b81346ad95179729a2ddfe2894ebf21495f

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: lakeops-1.0.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 268.8 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for lakeops-1.0.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 04b9e3e818216f5c2bffbb670e092cd147b20889cdd9a2f697d84d60c0cc3807
MD5 6745590306761176cf6b52f0daa8da39
BLAKE2b-256 2248a5681ec20e2d8a234763edf144fe2e9126fa9323126e9d724c8a0fe7adf0

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp310-cp310-win_amd64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp310-cp310-manylinux_2_34_x86_64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm Hash digest
SHA256 e064b8b616e46aecee42dffaa12a7d444761f1150e16608e49c65bba21260d5b
MD5 b2752da71a400f7bb9c1f480fda1f24c
BLAKE2b-256 b3490608514296342b861afb57c8490e93ebdb21109996370a9eec42066fb5b7

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp310-cp310-manylinux_2_34_x86_64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 036049fba4427a0a43b953431227cac5a3299e61ca4e57ac4c22e2e2539e1e18
MD5 faa967c1704b7d074ea23bbae7127bae
BLAKE2b-256 5bd2999ca856fd8f4e5930342850b2fc2b1a887d5d5e73c7d5899f052517eb0f

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file lakeops-1.0.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for lakeops-1.0.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ae147b2bfd6bf082058385bcf38cf7e7ba20ae0a3a2e4c96ee080c274f2ea9c1
MD5 edaf6ff047b2fb10fa4791bfbf76f450
BLAKE2b-256 bda93e0569a693d506357c9b16eb2da5144224fc213ea3966ac720cd2028ad9c

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakeops-1.0.1-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: publish.yml on hoaihuongbk/lakeops

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page