Skip to main content

Docs ftm-lakehouse on pypi PyPI Downloads PyPI - Python Version Python test and package pre-commit Coverage Status AGPLv3+ License Pydantic v2

ftm-lakehouse

ftm-lakehouse provides a data standard and archive storage for leaked data, private and public document collections and structured FollowTheMoney data. The concepts and implementations are originally inspired by mmmeta, Aleph's servicelayer archive and OpenSanctions work on dataset catalog metadata.

ftm-lakehouse acts as a multi-tenant storage and retrieval mechanism for structured entity data, documents and their metadata. It can be used by tenants to produce and/or consume data such as investigraph, memorious and the full suites of various search and analysis platforms, such as OpenALeph, ICIJ Datashare or Liquid Investigations

Installation

Requires python 3.11 or later.

pip install ftm-lakehouse

Remote storage backends are optional extras – install the one matching your archive/lake URI:

pip install "ftm-lakehouse[s3]"     # S3-compatible object storage (s3fs)
pip install "ftm-lakehouse[gcs]"    # Google Cloud Storage (gcsfs)
pip install "ftm-lakehouse[azure]"  # Azure Blob Storage (adlfs)
pip install "ftm-lakehouse[http]"   # HTTP(S)-backed api stores (aiohttp)

Extras combine, e.g. pip install "ftm-lakehouse[s3,gcs]".

Documentation

openaleph.org/docs/lib/ftm-lakehouse

Development

This package is using poetry for packaging and dependencies management, so first install it.

Clone this repository to a local destination.

Within the repo directory, run

poetry install --with dev

This installs a few development dependencies, including pre-commit which needs to be registered:

poetry run pre-commit install

Before creating a commit, this checks for correct code formatting (isort, black) and some other useful stuff (see: .pre-commit-config.yaml)

Testing

ftm-lakehouse uses pytest as the testing framework.

make test

Background

The design grew out of the FollowTheMoney data lake RFC discussion and prior art in mmmeta, Aleph's servicelayer archive, OpenSanctions dataset metadata and nomenklatura statements.

For contributing, development setup and testing see the repository README.

License and Copyright

leakrfc (predecessor), (c) 2024 investigativedata.io

ftm-lakehouse, (c) 2024 investigativedata.io

ftm-lakehouse, (c) 2025-2026 Data and Research Center – DARC

ftm-lakehouse is licensed under the AGPLv3 or later license.

Metadata

Release files for ftm-lakehouse 0.7.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ftm-lakehouse 0.7.0
File Size Uploaded Provenance Actions
ftm_lakehouse-0.7.0.tar.gz 129.9 kB Without provenance Details for ftm_lakehouse-0.7.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ftm-lakehouse 0.7.0
File Interpreter ABI Platform Actions
ftm_lakehouse-0.7.0-py3-none-any.whl Python 3 none any Details for ftm_lakehouse-0.7.0-py3-none-any.whl

Total release size: 293.3 kB

Release files / ftm_lakehouse-0.7.0.tar.gz

Download URL ftm_lakehouse-0.7.0.tar.gz
Size 129.9 kB
Tags Source
SHA-256 checksum
How to use checksums
461d938ca231ab813ee4a1c8258b5a103cc8a3e6b5194b2d1c5f0802bc16faa9
BLAKE2b-256 checksum
How to use checksums
212a0f2f0731dd32071b3589d062aaaac5f52ea3f2a4f0504fc0a019a6600122
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.4.1 CPython/3.14.4 Linux/7.0.0-30-generic

Release files / ftm_lakehouse-0.7.0-py3-none-any.whl

Download URL ftm_lakehouse-0.7.0-py3-none-any.whl
Size 163.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
66510d69e17f55be223fd5d52b71ab0c68ca4907e31a0183b1fea32c00dff3a9
BLAKE2b-256 checksum
How to use checksums
28898dde5d8d7336c4f3b6c4f719cc2aca4659958ef711327c178325743cafb2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.4.1 CPython/3.14.4 Linux/7.0.0-30-generic

Security for ftm-lakehouse 0.7.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page