This project has been archived by its maintainers, and is no longer receiving any updates.
Pipelantic
Catch incompatible data-pipeline wiring before you process data.
Define datasets, transformations, and pipelines as typed Python classes. Validate and plan them once. Run locally today; swap Polars, Pandas, or SQL backends without rewriting the logical pipeline.
Status: Alpha 0.6.0 — local runtime + optional Polars/Pandas/SQL plugins. Spark and Airflow compilation are not shipped.
Install
Requires Python 3.11 or newer.
pip install pipelantic
# optional engines
pip install pipelantic-polars
pip install pipelantic-pandas
pip install pipelantic-sql
Verify:
python -c "import pipelantic; print(pipelantic.__version__)"
Development
Requires uv.
uv sync
uv run pytest
uv run ruff check .
uv run ruff format .
uv sync creates .venv, installs the package in editable mode, and installs
the dev dependency group (pytest, ruff, mkdocs) by default.
Release
Tag a version that matches src/pipelantic/_version.py, then push the tag:
git tag v0.6.0
git push origin v0.6.0
GitHub Actions runs checks and publishes to PyPI using the PYPI_API_TOKEN
repository secret.
Quick example
from pipelantic import (
Data,
Input,
Output,
Pipeline,
PipelineRuntime,
Sink,
Source,
Transformation,
)
class RawCustomer(Data):
customer_id: int
first_name: str
last_name: str
class Customer(Data):
customer_id: int
full_name: str
class NormalizeCustomers(Transformation):
customers: Input[RawCustomer]
result: Output[Customer]
class CustomerPipeline(Pipeline):
raw: Source[RawCustomer] = Source(binding="customer_source")
normalized = NormalizeCustomers.step(customers=raw)
curated: Sink[Customer] = Sink(
input=normalized.result,
binding="customer_sink",
)
@NormalizeCustomers.implementation("local")
def normalize(customers: list[RawCustomer]) -> list[Customer]:
return [
Customer(
customer_id=row.customer_id,
full_name=f"{row.first_name} {row.last_name}",
)
for row in customers
]
CustomerPipeline.validate(profile="development").raise_for_errors()
runtime = PipelineRuntime()
runtime.memory.seed(
"customer_source",
[RawCustomer(customer_id=1, first_name="Ada", last_name="Lovelace")],
)
run_report = CustomerPipeline.run(profile="development", runtime=runtime)
print(runtime.memory.get("customer_sink"))
Run the complete tested version at examples/quickstart.py.
Current capability boundary
| Capability | 0.6 |
|---|---|
| Typed modeling, validation, contracts, and planning | Available |
| Local Python execution and run reports | Available |
| Memory, callable, JSON, CSV, and no-write storage | Available |
| Polars and Pandas dataframe plugins | Available (pipelantic-polars / pipelantic-pandas) |
| SQL plugin | Available (pipelantic-sql) |
| Spark and Airflow plugins | Not yet available |
Documentation
- Documentation site
- Getting Started
- Capabilities and Limitations
- Evaluator brief
- Core Concepts
- Architecture
- Roadmap
License
MIT
Metadata
Release files for pipelantic 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pipelantic-0.6.0.tar.gz | 159.3 kB | Without provenance | Details for pipelantic-0.6.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pipelantic-0.6.0-py3-none-any.whl | Python 3 | none | any | Details for pipelantic-0.6.0-py3-none-any.whl |
Total release size: 318.4 kB
Release files / pipelantic-0.6.0.tar.gz
| Download URL | pipelantic-0.6.0.tar.gz |
|---|---|
| Size | 159.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6ffc563d9e5bf1cd0b989ea6d68f7d2e9099f94eab0ff806838ce0e38da8ca0c
|
|
BLAKE2b-256 checksum How to use checksums |
7091a95c0515f7aa1196a2778dcfc088f8e198b3b5c8abebc68b91738de7f9b9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / pipelantic-0.6.0-py3-none-any.whl
| Download URL | pipelantic-0.6.0-py3-none-any.whl |
|---|---|
| Size | 159.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5657a67e3b3a4d3231adcb7d349eaa84a5ee1cfb52e0da0e2fcfeff16676b5a8
|
|
BLAKE2b-256 checksum How to use checksums |
33473080bfcf067d0f093d2df8cba2965d427ddcf9d462143ad6d0098c6c5e16
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.3 {"installer":{"name":"uv","version":"0.11.3","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for pipelantic 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.