ETLantic
One typed pipeline model. Many execution backends.
Typed contracts. Deterministic plans. Pluggable execution.
Quickstart · Documentation · Is ETLantic for me? · Python API · CLI
ETLantic is a Python library for defining data pipelines as typed contracts and graphs, validating them before they run, and producing a deterministic plan that a plugin executes (local Python, Polars, Pandas, SQL, PySpark, DataFusion, or DuckDB) or an orchestrator compiles (Airflow). It is not dbt, not a dataframe engine, and not a hosted scheduler.
Quickstart
ETLantic requires Python 3.11 or newer. In an activated virtual environment:
python -m pip install etlantic
python -m etlantic --version
mkdir my-pipeline
cd my-pipeline
python -m etlantic init --with-toml
python -m etlantic validate pipeline.py:SamplePipeline --profile development
python -m etlantic run pipeline.py:SamplePipeline --profile development
ETLantic 0.58.0 is the current Beta release candidate. The latest published
and supported line remains 0.57.x. The run should succeed and write Ada and
Grace to data/out.json. Until publication, pip install etlantic installs
the latest published 0.57.x package. See the
full Quickstart
for setup details and expected output.
If init refuses the directory, use an empty folder (or --force only after
you have reviewed what it overwrites). Pin every official plugin to the same
version as core (etlantic-polars with etlantic). Mixed
plugin versions fail closed — see
Troubleshooting.
What it provides
- Typed
Data,Transformation,Extract,Load, andPipelinebuilding blocks. - Validation before execution or publication, with human, JSON, and SARIF diagnostics.
- Deterministic plans that can be inspected, fingerprinted, diffed, run, or compiled.
- Contract generation for ODCS, DTCS, and DPCS artifacts.
- Pluggable execution across local Python, Polars, Pandas, SQL, and PySpark, DataFusion, and DuckDB, plus orchestration integrations.
Define transformation logic once with @Transformation.portable. ETLantic's
qualified relational baseline runs that definition across all seven engines.
Use @Transformation.implementation(...) only as an explicit escape hatch for
behavior outside the portable surface: native bodies are tied to one engine
and will not be eligible for adaptive execution.
Application code should use the curated public facade:
import etlantic as etl
The public CLI (see the CLI reference):
- Authoring:
init,doctor,validate,inspect,plan,profile,run,compile,generate,diff - Ops:
plugin,schema,reliability,erasure,viz,report,watch,stream,schedule,scheduler,worker - Agents:
context,proposal
Execution options
Core has no dataframe, database, Spark, or orchestrator dependency. Install only the integrations a pipeline needs:
| Capability | 0.58 |
|---|---|
| Local Python + JSON/CSV | etlantic |
| Polars or Pandas | etlantic[polars] or etlantic[pandas] |
| SQL or PySpark | etlantic[sql] or etlantic[pyspark] |
| Airflow or Prefect | etlantic[airflow] or etlantic[prefect] |
Prefer portable transformations and set portable_transform_policy="require"
so unsupported semantics fail during validation or planning. In controlled
deployments, pin ETLantic and official plugins to the same release. See
engine selection
and
compatibility.
Security and production posture
ETLantic is Beta, community-supported, with no SLA. Use it for documented
single-tenant pilots. You can embed an HTTP control plane
(etlantic-fastapi) with Supported isolation profiles
(isolated-deployment, dedicated-schema). There is no hosted multi-tenant
SaaS. See
Capabilities
and the
control-plane program.
Plans and reports contain secret references, never resolved secret values.
Production profiles require an explicit non-empty plugin_allowlist; an
allowlist controls selection but is not a sandbox. Schema history stores
fingerprints and metadata, never source rows.
Review the production readiness and security model before a pilot.
Learn more
These links use the Read the Docs stable alias, which remains on the latest published release. During candidate review, that is 0.57.2. The immutable patch-release tree is at v0.57.2.
- Quickstart and first pipeline
- Is ETLantic for me? and capabilities
- CLI reference and Python API
- Plugin SDK and optional packages
- Human-governed AI and scheduler tutorial
Contributing
See CONTRIBUTING.md for development setup and pull-request guidance. Use GitHub Issues for bugs and questions, and follow the private instructions in SECURITY.md to report vulnerabilities.
ETLantic is available under the MIT License.
1 maintainer
Metadata
1 maintainer
Release files for etlantic 0.58.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| etlantic-0.58.0.tar.gz | 2.1 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| etlantic-0.58.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 3.4 MB
Release files / etlantic-0.58.0.tar.gz
| Download URL | etlantic-0.58.0.tar.gz |
|---|---|
| Size | 2.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5cb982df46cd9cdb43b53a056b45166bf28e035d62cdd9c248aba07cbfd5889b
|
|
BLAKE2b-256 checksum How to use checksums |
51a6cbb442e3fbfd21bd9775ed3e99fe1c6cf063261b5e43fb226f0b391967ae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / etlantic-0.58.0-py3-none-any.whl
| Download URL | etlantic-0.58.0-py3-none-any.whl |
|---|---|
| Size | 1.2 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7f21c9da40e9565e9881825597a1e71a35abb1a12d6105830e7b4245d2d5cc4e
|
|
BLAKE2b-256 checksum How to use checksums |
a6c6bbc03423fa166b29918455a07880b31e3662b3d9fd6c1e78e073489e2eaf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
