NoETL
NoETL orchestrates APIs, databases, and scripts through a declarative Playbook DSL. This package is the NoETL command-line tool, installable with pip.
pip install noetl
It installs two equivalent commands, noetl and ntl, and an importable
Python API. There is no separate binary to download and nothing to put on your
PATH yourself.
noetl run pipeline.yaml --runtime local --set region=west
import noetl
outcome = noetl.run("pipeline.yaml", variables={"region": "west"})
print(outcome["status"], outcome["final_result"])
noetl.cli(["catalog", "list"]) # any CLI subcommand
What it is
The CLI is written in Rust and lives at
noetl/cli. This package compiles that crate
into a native extension module, so noetl.run(...) and the noetl command
both drive the same execution engine — nothing here reimplements playbook
execution or shells out to a separate process.
Every subcommand is available: run, catalog, context, auth, register,
status, query, ehdb, provider, subscribe, and the rest. Run
noetl --help for the full tree.
Python API
| Function | Purpose |
|---|---|
noetl.run(playbook, **options) |
Execute a playbook locally; returns the run outcome as a dict. |
noetl.cli(args) |
Run any CLI subcommand; returns its exit code. |
noetl.__version__ |
This package's version. |
noetl.run() returns the outcome envelope: status, playbook_name,
playbook_path, started_at, completed_at, duration_seconds,
executed_steps, step_results, final_result, and error when the run
failed. It executes locally; to submit to a distributed runtime, use
noetl.cli(["run", playbook, "--runtime", "distributed"]).
Versioning
This package's version and the CLI's version are separate. noetl --version
reports the version of the Rust CLI compiled into this release — that is the
number to quote in a bug report against
noetl/cli. noetl.__version__ and
pip show noetl report the package version.
Cutting a release
The published version comes from this directory, not from the git tag. Nothing
in pypi.yml derives it from the tag, and semantic-release's prepareCmd rewrites
the root pyproject.toml, which is a different file. So:
- Bump
packaging/pypi/pyproject.tomlandpackaging/pypi/Cargo.tomlto the same new version. They are bumped by hand and must move together. - Merge that, then push a
pypi-v<version>tag matching it exactly.
ci/verify_release.py gates the publish job and refuses the two ways this goes
wrong quietly — a tag ahead of the source tree, and a re-tag of a version already
on PyPI. The second one matters because twine upload --skip-existing exits 0
when it skips, so without the check that release would report success and ship
nothing. Run it yourself before tagging:
python packaging/pypi/ci/verify_release.py --self-test
python packaging/pypi/ci/verify_release.py --dist dist --tag pypi-v<version> --require-new
Known limitation: DuckDB
These wheels are built without the embedded DuckDB engine. Compiling DuckDB from its C++ source takes tens of minutes to an hour per platform, which a five-platform wheel matrix cannot absorb.
Affected surfaces report a rebuild hint rather than running:
kind: duckdbandkind: ducklakeplaybook steps- the
sink: duckdb:target - the
noetl iapcommand family, whose state ledger is a localstate.duckdb
Everything else — HTTP, shell, Postgres, playbook composition, providers, secrets, and the catalog/server/worker verbs — works normally. If you need DuckDB, install the CLI from source with the feature enabled:
cargo install --bins noetl --features duckdb-integration
or use a pre-built release binary, Homebrew, or the APT package — all of which ship with DuckDB included.
Requirements
Python 3.9 or newer, on Linux or macOS. The wheels are abi3, so one wheel
per platform serves every supported Python version.
Windows is not supported: the CLI uses POSIX process and signal control, which does not build on Windows (the native binaries are Linux/macOS-only for the same reason). On Windows, use WSL.
Documentation
- noetl.dev — user-facing documentation
- CLI reference
- Platform wiki
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for noetl 5.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| noetl-5.2.0.tar.gz | 50.4 kB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| noetl-5.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.9 | abi3 | Linux glibc 2.17+ x86-64 | Details |
| noetl-5.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | CPython 3.9 | abi3 | Linux glibc 2.17+ ARM64 | Details |
| noetl-5.2.0-cp39-abi3-macosx_11_0_arm64.whl | CPython 3.9 | abi3 | macOS 11.0+ ARM64 | Details |
Total release size: 23.3 MB
Release files / noetl-5.2.0.tar.gz
| Download URL | noetl-5.2.0.tar.gz |
|---|---|
| Size | 50.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2defc7b37712c645a2b7f957019f019bb4d47a3a34e281e68681a3853bbf4e8c
|
|
BLAKE2b-256 checksum How to use checksums |
04cb945bc3088021c9160bda0236afc1a1d3db21f486d883d2fa5bccccd0956b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / noetl-5.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | noetl-5.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 8.1 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
38308caa14869cbf22e50bc27b71c620bbfe0ea1f9de018e84758c3988451c81
|
|
BLAKE2b-256 checksum How to use checksums |
c28e48213040e296a3ef2d2aa5da9aab6fd3cfbed04b43006143f5b3ccc443c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / noetl-5.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | noetl-5.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 7.7 MB |
| Tags | CPython 3.9 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
4e80520c18d2d33cc2405f9b7ca6d67b19d06f67b037c63948a51565a21980c9
|
|
BLAKE2b-256 checksum How to use checksums |
ef41cb251507b741df7261c3d2f3b3abe8eca300b2dd4aab3f000052a114c3e9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
Release files / noetl-5.2.0-cp39-abi3-macosx_11_0_arm64.whl
| Download URL | noetl-5.2.0-cp39-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 7.4 MB |
| Tags | CPython 3.9 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
739e26eb8789cb259dd0d204e11d2f44e0641c563666d50cb802dec540c06c1d
|
|
BLAKE2b-256 checksum How to use checksums |
7802bc52503d91edd56b479cd3de042ab1ecac949a3baade9a292507efc877e9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.3
|
