jldaws 0.4.16

Collection of Amazon AWS related scripts

Overview

This package contains a collection of Amazon Web Service related scripts e.g.

• jldsqsjob : wait for stdin trigger, dequeue msg(s) from SQS Q#1, send to stdout, requeue to SQS Q#2, delete msg(s) from Q#1

• jlds3upload : automated file upload to S3

• jlds3download : automated file download from S3

• jldrxsqs : receive from an SQS queue to JSON/string stdout (with optional stdin trigger)

• jldtxsqs : transmit on SQS JSON/string from stdin

• jldleader: distributed leader election protocol manager

• jlds3up : simple file upload to s3 with functionality to delete ‘old’ files

• jlds3list : listing files in an S3 bucket

• jldsdb: list, create and delete operations on SDB

• jldsdbget: retrieve entries in SDB

• jldsdbinsert: insert records in SDB

• jldsdbbatchinsert: batch insert records in SDB from file

The philosophy behind these scripts is:

• simplicity : each script only does 1 thing

• reporting : status & error reporting through stderr and (optionally) JSON stdout

• data driven : object in from src ==> object out to dst

• data flow : piping through stdin / stdout

Scripts can easily be “composed” through standard Linux piping :

jldrxsqs @config1.txt | some_other_command | jldtxsqs @config2.txt

The data format used is JSON.

Configuration

Can be performed through options on the command line or using a file (use a leading @).

History

Changes history can be found at the project page: https://github.com/jldupont/jldaws