lakeFS High-Level Python SDK
lakeFS High Level SDK for Python, provides developers with the following features:
- Simpler programming interface with less configuration
- Inferring identity from environment
- Better abstractions for common, more complex operations (I/O, transactions, imports)
Requirements
Python 3.10+
Installation & Usage
pip install
pip install lakefs
Import the package
import lakefs
Getting Started
Please follow the installation procedure and afterward refer to the following example snippet for a quick start:
import lakefs
from lakefs.client import Client
# Using default client will attempt to authenticate with lakeFS server using configured credentials
# If environment variables or .lakectl.yaml file exist
repo = lakefs.repository(repository_id="my-repo")
# (The positional argument is simpler!)
repo = lakefs.repository("my-repo")
# Or explicitly initialize and provide a Client object
clt = Client(username="<lakefs_access_key_id>", password="<lakefs_secret_access_key>", host="<lakefs_endpoint>")
repo = lakefs.Repository(repository_id="my-repo", client=clt)
# From this point, proceed using the package according to documentation
main_branch = repo.create(storage_namespace="<storage_namespace>").branch(branch_id="main")
# Prefer to use "repo" rather than pass in the same repository_id each time.
...
Examples
Print sizes of all objects in lakefs://repo/main~2
ref = lakefs.Repository("repo").ref("main~2")
for obj in ref.objects():
print(f"{o.path}: {o.size_bytes}")
Difference between two branches
for i in lakefs.Repository("repo").ref("main").diff("twig"):
print(i)
You can also use the ref expressions here, for instance
.diff("main~2") also works. Ref expressions are the lakeFS analogues of
how Git specifies revisions.
Search a stored object for a string
with lakefs.Repository("repo").ref("main").object("path/to/data").reader(mode="r") as f:
for l in f:
if "quick" in l:
print(l)
Upload and commit some data
with lakefs.Repository("golden").branch("main").object("path/to/new").writer(mode="wb") as f:
f.write(b"my data")
# Returns a Reference
lakefs.Repository("golden").branch("main").commit("added my data using lakeFS high-level SDK")
# Prints "my data"
with lakefs.Repository("golden").branch("main").object("path/to/new").reader(mode="r") as f:
for l in f:
print(l)
Unlike references, branches are readable. This example couldn't work if we used a ref.
Tests
To run the tests using pytest, first clone the lakeFS git repository
git clone https://github.com/treeverse/lakeFS.git
cd lakefs/clients/python-wrapper
Unit Tests
Inside the tests folder, execute pytest utests to run the unit tests.
Integration Tests
See testing documentation for more information
Documentation
Author
Metadata
Release files for lakefs 0.17.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| lakefs-0.17.0.tar.gz | 36.5 kB | Without provenance | Details for lakefs-0.17.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| lakefs-0.17.0-py3-none-any.whl | Python 3 | none | any | Details for lakefs-0.17.0-py3-none-any.whl |
Total release size: 78.7 kB
Release files / lakefs-0.17.0.tar.gz
| Download URL | lakefs-0.17.0.tar.gz |
|---|---|
| Size | 36.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0c2c1daff9b4e60717abaf0c834b3e9872c2038c77bf0cf891773c7e1cc8f310
|
|
BLAKE2b-256 checksum How to use checksums |
0c6e414f2c6d30468418a56276b8c257c05062c739bcc41ffb59dee4209601ad
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Release files / lakefs-0.17.0-py3-none-any.whl
| Download URL | lakefs-0.17.0-py3-none-any.whl |
|---|---|
| Size | 42.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
aaff11b74c8cbf51980b3bac034ce9035911dcf03f43c86b35c4c5a93f5d5d4b
|
|
BLAKE2b-256 checksum How to use checksums |
d17bf0183851c4b82afef38df5eb4f05b79d52ae1bee4dd7a6fcc9941f5a78f5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Security for lakefs 0.17.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.