Skip to main content

Tools to work with clustered applications (PostgreSQL, Redis) on EC2

Project description

Generic clustering package for EC2, suitable for redis/postgresql etc.

This package makes it easier to deploy clustered applications such as PostgreSQL and Redis on EC2 by handling generic logic, including:

* Deciding which role the instance should assume
* Creating and updating DNS records
* Preparing the instance for its role (e.g. writing a recovery.conf file for postgres)

Basic Usage:

The default ec2cluster classes assume your EC2 instances have JSON-encoded user data containing some specific attributes. The following attributes are required:

* cluster - the name of the cluster, e.g. maindb

Install ec2cluster with pip:

pip install ec2cluster

Create a configuration file:

MASTER_CNAME = 'master.%(cluster)s.example.com'
SLAVE_CNAME = 'slave.%(cluster)s.example.com'
INIT_MASTER_SCRIPT = '/path/to/some_script.py'
INIT_SLAVE_SCRIPT = '/path/to/another_script.py'

Run ec2cluster, specifying the path to the config file:

ec2cluster init # initialise the cluster service
ec2cluster promote # promote a slave to the master role

PostgreSQL cluster:

When starting a postgres read-slave, a file named recovery.conf must be written to the postgres configuration directory. A template file is used to make it easy to customise your recovery options.

Config file:

MASTER_CNAME = 'master.%(cluster)s.example.com'
SLAVE_CNAME = 'slave.%(cluster)s.example.com'
RECOVERY_TEMPLATE = '/path/to/template.conf'
RECOVERY_FILENAME = '/var/lib/postgresql/9.1/main/recovery.conf'

In the recovery template file, specify the options required for your read-slaves. Instance metadata and userdata can be used for string replacement. For example:

standby_mode = on
recovery_target_timeline = latest
pause_at_recovery_target = false
restore_command = '/usr/bin/s3cmd --config=/var/lib/postgresql/.s3cfg get s3://%(cluster)s/archive/wal/%%f %%p'
primary_conninfo = 'host=%(master_cname) port=5432 user=postgres password=secret sslmode=disable

Note the use of “%%f” - because we are using string formatting we need to escape the percentage sign in order to end up with “%f” as required by postgres.

Project details


Release history Release notifications

History Node

0.28

History Node

0.27

History Node

0.26

History Node

0.25

History Node

0.24

History Node

0.23

This version
History Node

0.22

History Node

0.21

History Node

0.20

History Node

0.19

History Node

0.18

History Node

0.17

History Node

0.16

History Node

0.15

History Node

0.14

History Node

0.13

History Node

0.12

History Node

0.11

History Node

0.10

History Node

0.9

History Node

0.8

History Node

0.7

History Node

0.6

History Node

0.5

History Node

0.4

History Node

0.3

History Node

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
ec2cluster-0.22.tar.gz (8.4 kB) Copy SHA256 hash SHA256 Source None Nov 14, 2012

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page