Skip to main content

Light python framework for AWS SWF

Project description

Light python framework for AWS SWF

About this Project

Caravan is being used at Ludia for projects like marketing campaign system and on-demand distributed processing systems (thanks to hundreds of Lambda functions).

Feedbacks, ideas and contributions are highly welcomed. (Just open a Github issue).

Focus of this project

Similar projects exists (like Simpleflow which seems mature). Here is where Caravan differs from existing projects:

  • Support of AWS Lambda tasks

  • Boto3

  • KISS

  • Bring your own workflow framework (standard implementations as contribs)

  • No coupling between Decider code and Activity code

  • Paster compatible config file

Features

  • Decider worker

  • Activity task worker TODO

  • Commands to start/signal/terminate an arbitrary workflow execution

  • Command to list open workflow execution

  • Command to register a domain / list domains

Configuration

Caravan uses Boto3 to connect to AWS. See Boto 3 configuration guide for the complete documentation.

Environment Variables

AWS_ACCESS_KEY_ID

The access key for your AWS account.

AWS_SECRET_ACCESS_KEY

The secret key for your AWS account.

AWS_DEFAULT_REGION

The default region to use, e.g. us-east-1.

AWS_PROFILE

The default credential and configuration profile to use, if any.

Configuration Files

The credentials file is located at ~/.aws/credentials:

[default]
# The access key for your AWS account
aws_access_key_id=<YOUR ACCESS KEY ID>

# The secret key for your AWS account
aws_secret_access_key=<YOUR SECRET KEY>

The settings file is located at ~/.aws/config:

[default]
# The default region when making requests
region=<REGION NAME>

It also supports profiles:

[profile dev-profile]
# The default region when using the dev-profile account
region=<REGION NAME>

Demo

Setup a SWF domain to run this example:

$ caravan-domain-register -n CaravanDemo --retention-days 1

Write a workflow type (see full demo)

from caravan import Workflow


class Demo(Workflow):

    """Noop workflow using the bare caravan API."""

    name = 'Demo'
    version = '0.1'
    default_execution_start_to_close_timeout = '600'
    default_task_start_to_close_timeout = '10'

    def run(self):
        self.task.print_events()
        self.task.add_decision('CompleteWorkflowExecution')

Run the decider with the Demo workflow:

$ caravan-decider -d CaravanDemo -m caravan.examples.demo -t default --verbose

Start an execution of the Demo workflow:

$ caravan-start -d CaravanDemo -n Demo -v 0.1 -i 1

(The Demo workflow will wait for 5 minutes)

List the executions:

$ caravan-list -d CaravanDemo
$ caravan-list -d CaravanDemo --oldest 2015-01-01

Send a signal to an execution:

$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Hello World!'
$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Lorem ipsum'
$ caravan-signal -d CaravanDemo -i 1 -s STOP

Terminate an execution:

$ caravan-terminate -d CaravanDemo -i 1

Similar projects

Python:

Ruby:

References

Development

Possibly use virtualenvwrapper to manage your virtualenvs.

$ mkvirtualenv caravan

or

$ virtualenv caravan

Clone and install development dependencies:

(caravan)$ git clone git@github.com:pior/caravan.git
(caravan)$ cd caravan
(caravan)$ pip install -e .[dev]

Run tests:

(caravan)$ nosetests

Release

The release process use zest.releaser:

$ fullrelease

License

MIT licensed. See the bundled LICENSE file for more details

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

caravan-0.2.2.tar.gz (19.7 kB view details)

Uploaded Source

Built Distribution

caravan-0.2.2-py2-none-any.whl (32.1 kB view details)

Uploaded Python 2

File details

Details for the file caravan-0.2.2.tar.gz.

File metadata

  • Download URL: caravan-0.2.2.tar.gz
  • Upload date:
  • Size: 19.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for caravan-0.2.2.tar.gz
Algorithm Hash digest
SHA256 a2fe28892ba5657328527c2952f64deb639147bf117e756077f99c7d69a9a12b
MD5 23999c0fb1b91a59b3745e499fc66a6e
BLAKE2b-256 6956cc47a1089782f2241379710f39f7ba88577fc701a623a3cae82f77e8d792

See more details on using hashes here.

File details

Details for the file caravan-0.2.2-py2-none-any.whl.

File metadata

File hashes

Hashes for caravan-0.2.2-py2-none-any.whl
Algorithm Hash digest
SHA256 8748a069a437751b2c7ddd0ce80db959107c47eb714257474fb51c22aa14b9e4
MD5 27ca1e40c13a89093d1a59eb70292d9a
BLAKE2b-256 b081a2f1dd81ec9273a93aadd2647b2e95f44b01234cd789f745a9dc286b3cf6

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page