FlowTask
FlowTask is a plugin-based, component-driven task execution framework for create complex Tasks.
FlowTask runs Tasks defined in JSON, YAML or TOML files, any Task is a combination of Components, and every component in the Task run sequentially or depend of others, like a DAG.
Can create a Task combining Commands, Shell scripts and other specific Components (as TableInput: Open a Table using a datasource, DownloadFromIMAP: Download a File from a IMAP Folder, and so on), any Python Callable can be a Component inside a Task, or can extends UserComponent to build your own componets.
Every designed Task can run from CLI, programmatically, via RESTful API (using our aioHTTP-based Handler), called by WebHooks or even dispatched to a external Worker using our built-in Scheduler.
Quickstart
pip install flowtask
Tasks can organizated into directory structure like this:
tasks / ├── programs / ├── test / ├── tasks /
The main reason of this structure, is maintain organized several tasks by tenant/program, avoiding filling a directory with several task files.
FlowTask support "TaskStorage", a Task Storage is the main repository for tasks, main Task Storage is a directory in any filesystem path (optionally you can syncronize that path using git), but Tasks can be saved onto a Database or a S3 bucket.
Dependencies
- aiohttp (Asyncio Web Framework and Server) (required by navigator)
- AsyncDB
- QuerySource
- Navigator-api
- (Optional) Qworker (for distributing asyncio Tasks on distributed workers).
Features
- Component-based Task execution framework with several components covering several actions (download files, create pandas dataframes from files, mapping dataframe columns to a json-dictionary, etc)
- Built-in API for execution of Tasks.
How I run a Task?
Can run a Task from CLI:
task --program=test --task=example
on CLI, you can pass an ENV (enviroment) to change the environment file on task execution.
ENV=dev task --program=test --task=example
or Programmatically:
from flowtask import Task
import asyncio
task = Task(program='test', task='example')
results = asyncio.run(task.run())
# we can alternatively, using the execution mode of task object:
results = asyncio.run(task())
Requirements
- Python >= 3.9
- asyncio (https://pypi.python.org/pypi/asyncio/)
- aiohttp >= 3.6.2
Contribution guidelines
Please have a look at the Contribution Guide
- Writing tests
- Code review
- Other guidelines
Who do I talk to?
- Repo owner or admin
- Other community or team contact
License
Navigator is licensed under Apache 2.0 License. See the LICENSE file for more details.
2 maintainers
Release files for flowtask 5.12.18
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| flowtask-5.12.18.tar.gz | 1.6 MB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| flowtask-5.12.18-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.13 | CPython 3.13 | Linux glibc 2.17+ x86-64 | Details |
| flowtask-5.12.18-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.17+ x86-64 | Details |
| flowtask-5.12.18-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.17+ x86-64 | Details |
Total release size: 12.8 MB
Release files / flowtask-5.12.18.tar.gz
| Download URL | flowtask-5.12.18.tar.gz |
|---|---|
| Size | 1.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
312627feb0d557af918e99f92013cac2edd9107e85c14a58ae4279716f68b3a8
|
|
BLAKE2b-256 checksum How to use checksums |
027b40494383f396b4d749c3fee826f39c836a324e9903558b31f1322a441b72
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 25, 2026.
Transparency logRelease files / flowtask-5.12.18-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | flowtask-5.12.18-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 3.7 MB |
| Tags | CPython 3.13 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
1849b42cd4a43db07062ea51c06d31c3ad7773ad49a111f32b33eb29994ba101
|
|
BLAKE2b-256 checksum How to use checksums |
b5d342be2bc656634b2728436857f6b03c4904f791cd0d21e77bb28153cf975a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 25, 2026.
Transparency logRelease files / flowtask-5.12.18-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | flowtask-5.12.18-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.12 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
49affecda1a12cc6205b3df3f84e7f9094eaba5597fc3400cb07059182d4bb4f
|
|
BLAKE2b-256 checksum How to use checksums |
628eb036eef94229b29b173cfb94062668a3f273cc980596d722b740beea7976
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 25, 2026.
Transparency logRelease files / flowtask-5.12.18-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | flowtask-5.12.18-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 3.7 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
b495283863ab76ec1ecb3620d8fdc21258ddc2246443ec7db051e06b5f688b0d
|
|
BLAKE2b-256 checksum How to use checksums |
95964bd4a6ae1f4e26869015d54bf0302e6f12bfc875a079ab5d3e4eb9165b86
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 25, 2026.
Transparency log
