Light python framework for AWS SWF
Project description
Light python framework for AWS SWF
About this Project
Caravan is being used at Ludia for projects like marketing campaign system and on-demand distributed processing systems (thanks to hundreds of Lambda functions).
Feedbacks, ideas and contributions are highly welcomed. (Just open a Github issue).
Doc: TODO
Focus of this project
Similar projects exists (like Simpleflow which seems mature). Here is where Caravan differs from existing projects:
Support of AWS Lambda tasks
Boto3
KISS
Bring your own workflow framework (standard implementations as contribs)
No coupling between Decider code and Activity code
Paster compatible config file
Features
Decider worker
Activity task worker TODO
Commands to start/signal/terminate an arbitrary workflow execution
Command to list open workflow execution
Command to register a domain / list domains
Configuration
Caravan uses Boto3 to connect to AWS. See Boto 3 configuration guide for the complete documentation.
Environment Variables
- AWS_ACCESS_KEY_ID
The access key for your AWS account.
- AWS_SECRET_ACCESS_KEY
The secret key for your AWS account.
- AWS_DEFAULT_REGION
The default region to use, e.g. us-east-1.
- AWS_PROFILE
The default credential and configuration profile to use, if any.
Configuration Files
The credentials file is located at ~/.aws/credentials:
[default] # The access key for your AWS account aws_access_key_id=<YOUR ACCESS KEY ID> # The secret key for your AWS account aws_secret_access_key=<YOUR SECRET KEY>
The settings file is located at ~/.aws/config:
[default] # The default region when making requests region=<REGION NAME>
It also supports profiles:
[profile dev-profile] # The default region when using the dev-profile account region=<REGION NAME>
Demo
Setup a SWF domain to run this example:
$ caravan-domain-register -n CaravanDemo --retention-days 1
Write a workflow type (see full demo)
from caravan import Workflow
class Demo(Workflow):
"""Noop workflow using the bare caravan API."""
name = 'Demo'
version = '0.1'
default_execution_start_to_close_timeout = '600'
default_task_start_to_close_timeout = '10'
def run(self):
self.task.print_events()
self.task.add_decision('CompleteWorkflowExecution')
Run the decider with the Demo workflow:
$ caravan-decider -d CaravanDemo -m caravan.examples.demo -t default --verbose
Start an execution of the Demo workflow:
$ caravan-start -d CaravanDemo -n Demo -v 0.1 -i 1 (The Demo workflow will wait for 5 minutes)
List the executions:
$ caravan-list -d CaravanDemo $ caravan-list -d CaravanDemo --oldest 2015-01-01
Send a signal to an execution:
$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Hello World!' $ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Lorem ipsum' $ caravan-signal -d CaravanDemo -i 1 -s STOP
Terminate an execution:
$ caravan-terminate -d CaravanDemo -i 1
Similar projects
Python:
Simpleflow: https://github.com/botify-labs/simpleflow
Ruby:
AWS Flow: https://github.com/aws/aws-flow-ruby
References
Development
Possibly use virtualenvwrapper to manage your virtualenvs.
$ mkvirtualenv caravan
or
$ virtualenv caravan
Clone and install development dependencies:
(caravan)$ git clone git@github.com:pior/caravan.git (caravan)$ cd caravan (caravan)$ pip install -e .[dev]
Run tests:
(caravan)$ nosetests
Release
The release process use zest.releaser:
$ fullrelease
License
MIT licensed. See the bundled LICENSE file for more details
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.