Lakebench
Evaluate lakehouse architectures through the workloads they execute.
Lakebench runs defined data workloads against configurable lakehouse architectures on Kubernetes. It measures how the assembled system behaves, where time and resources go, and how results change when the architecture or operating conditions change. It produces measurements and supporting context, not a verdict on the best architecture or its production suitability.
How Lakebench works
An experiment combines four elements:
- Workload: the data, processing, queries and outcomes that exercise the architecture.
- Architecture: the composition of processing engines, table formats, catalogs and query engines.
- System: the Kubernetes, compute, network and storage environment.
- Execution conditions: scale, resource allocations, execution mode and other controls.
Workload x Architecture x System x Execution conditions -> Evidence
Lakebench deploys the components, generates data, runs the workload, collects measurements and writes reports.
Workloads
- Customer 360: a retail analytics workload. Customer interactions are ingested, validated, transformed, aggregated and queried.
- Financial Crime / AML: transaction-monitoring over synthetic financial transactions: ingestion, transformation, detection and analytical queries. It evaluates how the architecture executes the workload, not the effectiveness of a production AML system.
Running an experiment
Install
pip install lakebench-k8s
Binaries that need no Python are on GitHub Releases.
Run
lakebench init --endpoint http://your-s3-endpoint:80 # writes lakebench.yaml
export LAKEBENCH_S3_ACCESS_KEY=...
export LAKEBENCH_S3_SECRET_KEY=...
lakebench admin install --component all lakebench.yaml # once per cluster
lakebench plan lakebench.yaml # what the run needs (read-only)
lakebench run lakebench.yaml --generate --yes # deploy, generate, run workload
lakebench report lakebench.yaml # print results
lakebench destroy lakebench.yaml # remove what this deployment created
initwrites a minimal configuration.--recipe <name>selects the architecture;--workload financialselects the AML workload.lakebench config recipeslists the 11 supported recipes.admin installplaces shared cluster components (Spark Operator, Stackable operators for Hive recipes, scratch StorageClass). Run once per cluster.planchecks whether the cluster has room for the workload. It changes nothing.--generatefills the bronze bucket with synthetic data. Without it,runrefuses (exit 4).--yesletsrundeploy the namespace and components. Without it, runlakebench deployfirst.destroyremoves only what this deployment created. It asks for confirmation unless--yesis passed.
The full walkthrough is Getting Started.
Results
Depending on the workload, a run records outcomes and correctness, stage timings, throughput, query or detection performance, storage I/O, and compute and memory use. Results describe behaviour under the conditions of that run. Compare runs only when their execution conditions and workload semantics match.
lakebench report prints a summary and the path to the HTML report.
--render writes a fresh copy; --format json or csv is for scripts.
Prerequisites
Lakebench does not provision the Kubernetes cluster or the storage.
Host tooling:
kubectlandhelmon PATH.
Kubernetes:
- Kubernetes 1.26+.
- Permission to create a namespace.
Storage:
- S3-compatible object storage with an endpoint URL, access key and secret
key. Validated on FlashBlade and Garage; expected to work on AWS S3,
MinIO and other S3-compatible stores.
lakebench config storagechecks yours.
Cluster capacity:
- Room for the workload. Scale 1 batch needs about 41 cores and 544 GB;
requirements grow with scale and differ by workload and mode.
lakebench planchecks the cluster and Sizing lists every combination.
Prerequisites lists every check and its fix.
Documentation
- Getting Started: install, first run, teardown
- Documentation index: full navigation
- Configuration: every YAML field
- CLI Reference: every command and flag
- Workloads: Customer 360 and AML
- Recipes: supported architectural compositions
- Results & Comparison: measurements and interpretation
License
Apache 2.0
1 maintainer
Metadata
1 maintainer
Release files for lakebench-k8s 1.7.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| lakebench_k8s-1.7.4.tar.gz | 2.0 MB | With provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| lakebench_k8s-1.7.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 3.7 MB
Release files / lakebench_k8s-1.7.4.tar.gz
| Download URL | lakebench_k8s-1.7.4.tar.gz |
|---|---|
| Size | 2.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c360d1d06df60bbd9fa6e976a18e75e518ae96d4286e6c648d3d2a34761c6de2
|
|
BLAKE2b-256 checksum How to use checksums |
9a217366e3096f3444adad0639714462e372444993f6900688c57ecb32978b79
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 13, 2026.
Transparency logRelease files / lakebench_k8s-1.7.4-py3-none-any.whl
| Download URL | lakebench_k8s-1.7.4-py3-none-any.whl |
|---|---|
| Size | 1.7 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
62726e600e2c9d25a1465b7da6f94571cdd5567a476cca3304e7e8d9e0fc7acc
|
|
BLAKE2b-256 checksum How to use checksums |
79da6b318782fe8fb057639559b604cf89d89ba76d6ef25147705b3792776106
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 13, 2026.
Transparency logSecurity for lakebench-k8s 1.7.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for lakebench_k8s-1.7.4.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 13, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
