ftm-lakehouse
ftm-lakehouse provides a data standard and archive storage for leaked data, private and public document collections and structured FollowTheMoney data. The concepts and implementations are originally inspired by mmmeta, Aleph's servicelayer archive and OpenSanctions work on dataset catalog metadata.
ftm-lakehouse acts as a multi-tenant storage and retrieval mechanism for structured entity data, documents and their metadata. It can be used by tenants to produce and/or consume data such as investigraph, memorious and the full suites of various search and analysis platforms, such as OpenALeph, ICIJ Datashare or Liquid Investigations
Installation
Requires python 3.12 or later.
pip install ftm-lakehouse
Remote storage backends are optional extras – install the one matching your archive/lake URI:
pip install "ftm-lakehouse[s3]" # S3-compatible object storage (s3fs)
pip install "ftm-lakehouse[gcs]" # Google Cloud Storage (gcsfs)
pip install "ftm-lakehouse[azure]" # Azure Blob Storage (adlfs)
pip install "ftm-lakehouse[http]" # HTTP(S)-backed api stores (aiohttp)
Extras combine, e.g. pip install "ftm-lakehouse[s3,gcs]".
Documentation
openaleph.org/docs/lib/ftm-lakehouse
Development
This package is using poetry for packaging and dependencies management, so first install it.
Clone this repository to a local destination.
Within the repo directory, run
poetry install --with dev
This installs a few development dependencies, including pre-commit which needs to be registered:
poetry run pre-commit install
Before creating a commit, this checks for correct code formatting (isort, black) and some other useful stuff (see: .pre-commit-config.yaml)
Testing
ftm-lakehouse uses pytest as the testing framework.
make test
Background
The design grew out of the FollowTheMoney data lake RFC discussion and prior art in mmmeta, Aleph's servicelayer archive, OpenSanctions dataset metadata and nomenklatura statements.
For contributing, development setup and testing see the repository README.
License and Copyright
leakrfc (predecessor), (c) 2024 investigativedata.io
ftm-lakehouse, (c) 2024 investigativedata.io
ftm-lakehouse, (c) 2025-2026 Data and Research Center – DARC
ftm-lakehouse is licensed under the AGPLv3 or later license.
1 maintainer
Metadata
1 maintainer
Release files for ftm-lakehouse 0.9.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| ftm_lakehouse-0.9.0.tar.gz | 124.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| ftm_lakehouse-0.9.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 281.0 kB
Release files / ftm_lakehouse-0.9.0.tar.gz
| Download URL | ftm_lakehouse-0.9.0.tar.gz |
|---|---|
| Size | 124.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
76b7aad493334bb6f109041d9f4774af88d08b6c7dbb7db327946b3ab50ae0a7
|
|
BLAKE2b-256 checksum How to use checksums |
935237cfbd68e7c08cad8b388a9775864a22d2e8bea420b1ae3536c9ac207c15
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-31-generic
|
Release files / ftm_lakehouse-0.9.0-py3-none-any.whl
| Download URL | ftm_lakehouse-0.9.0-py3-none-any.whl |
|---|---|
| Size | 156.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
104759d466e9025f91c646e9b6c0b97f5484c0f8fb0a71fe495d38a29415f1fa
|
|
BLAKE2b-256 checksum How to use checksums |
7b306bdacef4483fa656128edd54e2defab21c34cc093509bbcda1a4509da49a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-31-generic
|
