Skip to main content

LakeSoul Python Support

Native Writer

The native writer accepts PyArrow batches, tables, and record batch readers. It writes data files and returns their metadata; publishing those files to the LakeSoul metadata service is a separate operation.

import pyarrow as pa

from lakesoul.io import IOConfig, Writer

table = pa.table({"id": [1, 2], "value": ["a", "b"]})
config = IOConfig(
    path="./output",
    schema=table.schema,
    format="parquet",
)

with Writer(config) as writer:
    writer.write(table)

print(writer.result.files)

Ray Writer

Importing lakesoul.ray registers a Pythonic write method on Ray datasets. The target LakeSoul table must already exist. Data files are written by Ray tasks, then committed to LakeSoul metadata by the driver after every task succeeds.

import ray
import lakesoul.ray

dataset = ray.data.from_items(
    [
        {"id": 1, "value": "a"},
        {"id": 2, "value": "b"},
    ]
)
dataset.write_lakesoul(
    "target_table",
    namespace="default",
    format="parquet",
)

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

lakesoul-2.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (55.9 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

File details

Details for the file lakesoul-2.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for lakesoul-2.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c578d95c3f452036b21eea0ca6c8e4b91b7464cc58b2d77879298f1c8872aebc
MD5 8fe4470366f7f12ac2f1de90699a0cbd
BLAKE2b-256 1046e06514934b4f08ded9bb91e49a80b949360514cf05c352b7da7c4c74d5a2

See more details on using hashes here.

Provenance

The following attestation bundles were made for lakesoul-2.0.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: python-ci.yml on lakesoul-io/LakeSoul

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

2.0.0 This release

1 file

1.0.2

3 files

1.0.1

3 files

1.0.0

3 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page