Culvert
A cloud-agnostic, polyglot data-pipeline framework. One contract set —
Source, Sink, Transform, Pipeline, RuntimeContext, BlobStore,
Warehouse, JobControlRepository, and friends — realised in Java and
Python, with cloud specifics behind adapters. Define the pipeline once
against contracts; point it at an emulator on your laptop, a dev project, or
production by wiring, not rewriting.
- Contract-driven — the language-neutral spec is the portability boundary. Business logic depends only on contracts.
- Two clouds today — GCP is the first full implementation; AWS is a real (Java) adapter family with the same pipeline proven on both. Azure is on the roadmap.
- No application framework in the core — plain Python Protocols + entry-point auto-discovery; composes into Beam pipelines, Airflow DAGs, or a script on your laptop.
- Local-first — the whole stack runs against emulators with zero cloud account; cloud is where you prove and ship, not where you develop.
Install
pip install culvert # core contracts only (no cloud SDKs)
pip install culvert[gcp] # + BigQuery, GCS, Pub/Sub, Secret Manager, observability
pip install culvert[orchestration] # + Airflow-side DAG factory, operators, sensors
pip install culvert[transform] # + dbt integration
pip install culvert[all]
Python ≥ 3.10. For 0.1.0 the import packages keep their library names — the
contracts live in data_pipeline_core:
from data_pipeline_core import autoconfig
config = autoconfig.discover() # entry-point adapter discovery
blob_store = config.blob_store() # GcsBlobStore if culvert[gcp] installed
The Java twin ships as com.enrichmeai.culvert:* on Maven Central — same
contracts, Java owns the Beam/Dataflow execution layer; Python owns the
Airflow runtime and dbt packaging.
Status
0.1.0 — first public release. Built and validated against a real GCP
project (Cloud Run, BigQuery, Pub/Sub, event-driven end-to-end) before
publishing. GCP adapters are production-shaped; the AWS family is Java-side;
Azure is a roadmap skeleton. Honest limitations are documented per adapter in
the source.
Docs, architecture, worked example deployments, and the engineering story: https://github.com/enrichmeai/culvert
MIT licensed.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file culvert-0.1.1.tar.gz.
File metadata
- Download URL: culvert-0.1.1.tar.gz
- Upload date:
- Size: 124.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ea90d1a000b4a8ad29d1877ccf17890966e2b07b766cae8df88d6694624c918c
|
|
| MD5 |
7923a855f9c76c2ed7e2b4a3c634d23f
|
|
| BLAKE2b-256 |
a6945dd3d9e2830b5e0c2c29915dfcfa0e51263cfc8b4785d5f80a3ac1c15096
|
Provenance
The following attestation bundles were made for culvert-0.1.1.tar.gz:
Publisher:
publish-pypi.yml on enrichmeai/culvert
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
culvert-0.1.1.tar.gz -
Subject digest:
ea90d1a000b4a8ad29d1877ccf17890966e2b07b766cae8df88d6694624c918c - Sigstore transparency entry: 2173622215
- Sigstore integration time:
-
Permalink:
enrichmeai/culvert@ab01cfe8e0c75cb1cbb4ba4477e0765d46ef8d37 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/enrichmeai
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-pypi.yml@ab01cfe8e0c75cb1cbb4ba4477e0765d46ef8d37 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file culvert-0.1.1-py3-none-any.whl.
File metadata
- Download URL: culvert-0.1.1-py3-none-any.whl
- Upload date:
- Size: 184.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
04642d84a389ff7eb31fcac5b31146bf45fb538f9678339cbb5dd6cf90081b08
|
|
| MD5 |
a9100a7cfbdc5a5c93822b8858d8cc4f
|
|
| BLAKE2b-256 |
6365b545e4d589e58c57e58c51dd17c432e03cb35023047d52681c3a3851b66d
|
Provenance
The following attestation bundles were made for culvert-0.1.1-py3-none-any.whl:
Publisher:
publish-pypi.yml on enrichmeai/culvert
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
culvert-0.1.1-py3-none-any.whl -
Subject digest:
04642d84a389ff7eb31fcac5b31146bf45fb538f9678339cbb5dd6cf90081b08 - Sigstore transparency entry: 2173622739
- Sigstore integration time:
-
Permalink:
enrichmeai/culvert@ab01cfe8e0c75cb1cbb4ba4477e0765d46ef8d37 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/enrichmeai
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish-pypi.yml@ab01cfe8e0c75cb1cbb4ba4477e0765d46ef8d37 -
Trigger Event:
workflow_dispatch
-
Statement type:
