A framework to build and run data pipelines and workflows.
Project description
NoETL
NoETL is an automation framework for orchestrating APIs, databases, and scripts using a declarative Playbook DSL.
Execution is standardized around an MCP-style tool model: consistent tool contracts, structured input/output, and a predictable lifecycle. From an MCP perspective, tools include API endpoints, database operations, and scripts/utilities NoETL orchestrates and optimizes them via playbooks.
With NoETL Gateway, playbooks can be deployed as a distributed backend: developers ship business logic as playbooks, and UIs/clients call stable endpoints without deploying dedicated microservices for each workflow.
Documentation
https://noetl.dev — user-facing site.
NoETL wiki — operator and
developer reference. Pages mirror the code tree under noetl/noetl.
Async batch acceptance and recovery references (on the wiki):
- Batch Events API —
POST /api/events/batchasync acceptance flow - Recovery: Auto-Resume — readiness-gated parent-execution restart at startup
- Command Reaper — runtime re-publish for orphaned / stranded commands
Distributed runtime components
Reference docs for the event-sourced, projection-backed runtime
(implements v2 distributed-runtime spec phases 0–2; spec lives in
noetl/docs at docs/features/noetl_distributed_runtime_spec.md):
- Event Store —
durable append-only event log (port + Postgres adapter),
EventRecordenvelope, optimistic concurrency viaexpected_version. - Projection Store — version-monotonic projection + snapshot store (port + Postgres adapter), query interface for replay state.
- Outbox — transactional
outbox publisher (
python -m noetl.outbox) that drainsnoetl.outboxto NATS with at-least-once retry/backoff. - Projector —
out-of-process projection worker (
python -m noetl.projector), durable NATS pull consumer, shard-stable, Prometheus metrics, replay-state folding shared with the in-process replay API.
Repository model (ai-meta driven)
NoETL development is now coordinated through the ai-meta repository:
- Orchestration/meta: https://github.com/noetl/ai-meta
- Server: https://github.com/noetl/server
- Worker: https://github.com/noetl/worker
- Gateway: https://github.com/noetl/gateway
- CLI: https://github.com/noetl/cli
- Tools: https://github.com/noetl/tools
- End-to-end fixtures: https://github.com/noetl/e2e
- Ops/automation: https://github.com/noetl/ops
- Docs: https://github.com/noetl/docs
ai-meta tracks all component repositories as Git submodules and is the primary place to
coordinate cross-repo changes, pointer bumps, and release choreography.
End-to-end integration playbooks, fixture payloads, local credential templates, notebooks,
and Gateway UI test fixtures live in noetl/e2e.
Distribution channels
- PyPI
noetl(Python): https://pypi.org/project/noetl/
- Rust components
- CLI repo: https://github.com/noetl/cli
- Gateway repo: https://github.com/noetl/gateway
- Server repo: https://github.com/noetl/server
- Worker repo: https://github.com/noetl/worker
- Tools repo: https://github.com/noetl/tools
- APT (Debian/Ubuntu)
- Homebrew
License
Dual License — see LICENSE for details.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file noetl-2.98.0.tar.gz.
File metadata
- Download URL: noetl-2.98.0.tar.gz
- Upload date:
- Size: 799.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
858bcaf27aaa497f19eb8ec2fc576584edc7a0ed00af8cf2a8a3ea72bf488877
|
|
| MD5 |
781717abfabe353992bf8ef61503ec2b
|
|
| BLAKE2b-256 |
9c1879b38f141511cdb3088b0d19e03e024bef864ae7feabb81d7968ada13473
|
File details
Details for the file noetl-2.98.0-py3-none-any.whl.
File metadata
- Download URL: noetl-2.98.0-py3-none-any.whl
- Upload date:
- Size: 893.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.9.30 {"installer":{"name":"uv","version":"0.9.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d1cc7fbfb19b4b80df62c4ce63fdd43920a765f3195d57f31380134f8638447d
|
|
| MD5 |
4688c1a9c336e0f086cf6875275543d5
|
|
| BLAKE2b-256 |
ef088f5fbb3eca215d5a0e96976329277175609cc226694793332786d26c8300
|
