Python library for dataflow programming with Amazon SWF
Project description
Simpleflow
Simpleflow is a Python library that provides abstractions to write programs in the distributed dataflow paradigm. It coordinates the execution of distributed tasks with Amazon SWF.
It relies on futures to describe the dependencies between tasks. A Future object
models the asynchronous execution of a computation that may end. It tries to mimic
the interface of the Python concurrent.futures library.
Features
- Provides a
Futureabstraction to define dependencies between tasks. - Define asynchronous tasks from callables.
- Handle workflows with Amazon SWF.
- Implement replay behavior like the Amazon Flow framework.
- Handle retry of tasks that failed.
- Automatically register decorated tasks.
- Encodes/decodes large fields to S3 objects transparently (aka "jumbo fields").
- Handle the completion of a decision with more than 100 tasks.
- Provides a local executor to check a workflow without Amazon SWF (see
simpleflow --localcommand). - Provides decider and activity worker process for execution with Amazon SWF.
- Ships with the
simpleflowcommand.simpleflow --helpfor more information about the commands it supports.
You can read more in the Features section of the documentation.
Overview
Please read and even run the demo script to have a quick glance of
simpleflow commands. To run the demo you will need to start decider
and activity worker processes.
Start a decider with:
$ PYTHONPATH=$PWD simpleflow decider.start --domain TestDomain --task-list test examples.basic.BasicWorkflow
Start an activity worker with:
$ simpleflow worker.start --domain TestDomain --task-list quickstart
Then execute ./extras/demo.
More information
Read the main documentation at https://botify-labs.github.io/simpleflow/.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file simpleflow-0.35.0.tar.gz.
File metadata
- Download URL: simpleflow-0.35.0.tar.gz
- Upload date:
- Size: 113.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.11.21 {"installer":{"name":"uv","version":"0.11.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9e0a42ba2da88fcd5ef5a7c9f8170376045a3cf49460ac5a77fdd4f86218310f
|
|
| MD5 |
f2978396caa333049af41dd0467e81c9
|
|
| BLAKE2b-256 |
e8d457dd8e1f7803f37b9efd358d6b5eae2bde29632a083afb7281aca5bee383
|
File details
Details for the file simpleflow-0.35.0-py3-none-any.whl.
File metadata
- Download URL: simpleflow-0.35.0-py3-none-any.whl
- Upload date:
- Size: 158.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.11.21 {"installer":{"name":"uv","version":"0.11.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
787b5f2e05735edd5069e3b5635ab48b60d8e7dff00672bdf01f89fe699cf7ee
|
|
| MD5 |
da5d72ceeab4b95f9b2d6d663f5ff887
|
|
| BLAKE2b-256 |
8bc73cda010e86e0c9f788d0da2a879a33737cb51d553940aca3209c4c179a64
|
