ETLantic
One typed pipeline model. Many execution backends.
Typed contracts. Deterministic plans. Pluggable execution.
Quickstart · Documentation · Is ETLantic for me? · Python API · CLI
ETLantic is a Python library for defining data pipelines as typed contracts and graphs, validating them before they run, and producing a deterministic plan that a plugin executes (local Python, Polars, Pandas, SQL, PySpark, DataFusion, or DuckDB) or an orchestrator compiles (Airflow). It is not dbt, not a dataframe engine, and not a hosted scheduler.
Quickstart
ETLantic requires Python 3.11 or newer. In an activated virtual environment:
python -m pip install 'etlantic==0.51.0'
python -m etlantic --version
mkdir my-pipeline
cd my-pipeline
python -m etlantic init --with-toml
python -m etlantic validate pipeline.py:SamplePipeline --profile development
python -m etlantic run pipeline.py:SamplePipeline --profile development
The run should succeed and write Ada and Grace to data/out.json. See the
full Quickstart
for setup details and expected output.
If init refuses the directory, use an empty folder (or --force only after
you have reviewed what it overwrites). Pin every official plugin to the same
version as core (etlantic-polars==0.51.0 with etlantic==0.51.0). Mixed
plugin versions fail closed — see
Troubleshooting.
What it provides
- Typed
Data,Transformation,Extract,Load, andPipelinebuilding blocks. - Validation before execution or publication, with human, JSON, and SARIF diagnostics.
- Deterministic plans that can be inspected, fingerprinted, diffed, run, or compiled.
- Contract generation for ODCS, DTCS, and DPCS artifacts.
- Pluggable execution across local Python, Polars, Pandas, SQL, and PySpark, DataFusion, and DuckDB, plus orchestration integrations.
Define transformation logic once with @Transformation.portable. ETLantic's
qualified relational baseline runs that definition across all seven engines.
Use @Transformation.implementation(...) only as an explicit escape hatch for
behavior outside the portable surface: native bodies are tied to one engine
and will not be eligible for adaptive execution.
Application code should use the curated public facade:
import etlantic as etl
The public CLI (see the CLI reference):
- Authoring:
init,doctor,validate,inspect,plan,profile,run,compile,generate,diff - Ops:
plugin,schema,reliability,erasure,viz,report,watch,stream,schedule,scheduler,worker - Agents:
context,proposal
Execution options
Core has no dataframe, database, Spark, or orchestrator dependency. Install only the integrations a pipeline needs:
| Capability | 0.51 |
|---|---|
| Local Python + JSON/CSV | etlantic |
| Polars or Pandas | etlantic[polars] or etlantic[pandas] |
| SQL or PySpark | etlantic[sql] or etlantic[pyspark] |
| Airflow or Prefect | etlantic[airflow] or etlantic[prefect] |
Prefer portable transformations and set portable_transform_policy="require"
so unsupported semantics fail during validation or planning. In controlled
deployments, pin ETLantic and official plugins to the same release. See
engine selection
and
compatibility.
Security and production posture
ETLantic is Beta, community-supported, with no SLA. Use it for documented
single-tenant pilots. You can embed an HTTP control plane
(etlantic-fastapi) with Supported isolation profiles
(isolated-deployment, dedicated-schema). There is no hosted multi-tenant
SaaS. See
Capabilities
and the
control-plane program.
Plans and reports contain secret references, never resolved secret values.
Production profiles require an explicit non-empty plugin_allowlist; an
allowlist controls selection but is not a sandbox. Schema history stores
fingerprints and metadata, never source rows.
Review the production readiness and security model before a pilot.
Learn more
These links use the Read the Docs stable alias, which remains on the latest published release until 0.51.0 is tagged. The immutable 0.51.0 tree will be at v0.51.0.
- Quickstart and first pipeline
- Is ETLantic for me? and capabilities
- CLI reference and Python API
- Plugin SDK and optional packages
- Human-governed AI and scheduler tutorial
Contributing
See CONTRIBUTING.md for development setup and pull-request guidance. Use GitHub Issues for bugs and questions, and follow the private instructions in SECURITY.md to report vulnerabilities.
ETLantic is available under the MIT License.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file etlantic-0.51.0.tar.gz.
File metadata
- Download URL: etlantic-0.51.0.tar.gz
- Upload date:
- Size: 1.2 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
64b88db617c1fa827577b098a6f64db26d4a5d9327dbdb149cfae0ccf4ec85cd
|
|
| MD5 |
6a3862f16c22bfe8585ea80df2c7b300
|
|
| BLAKE2b-256 |
d89f1f552438d219c7d4883957de807054b63c81d41dd1fa3aed582b50571e87
|
File details
Details for the file etlantic-0.51.0-py3-none-any.whl.
File metadata
- Download URL: etlantic-0.51.0-py3-none-any.whl
- Upload date:
- Size: 818.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.0 {"installer":{"name":"uv","version":"0.12.0","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
339915b63fb639d628122fed78b594efe2ca217af185ace2893299449a833f62
|
|
| MD5 |
a4e5fbab8d8c7eb119cb0430311d5931
|
|
| BLAKE2b-256 |
b17b1408270f3db8e593736ba21338233c9471e34f52c2337beedeedfd715a6b
|
