Skip to main content

Fabric tasks for Django and Amazon Web Services

Project description

Fabric tasks for Django and Amazon Web Services
***********************************************

Required arrangements
---------------------

.. note:: `django_fabfile` package should be listed in your path. It's already
in place if your are installed it using `pip install django-fabfile`. But if
you are working within repository, then::

export PYTHONPATH=<path-to-folder-with-django_fabfile-package>

in your shell.

Pre-run configuration
---------------------

Django settings will be checked first if environment variable
`DJANGO_SETTINGS_MODULE` configured properly. If option was not configured
within Django settings, then its value will be taken from ``./fabfile.cfg``
file - copy-paste there rows that should be overriden from defaults
:download:`django_fabfile/fabfile.cfg.def <../django_fabfile/fabfile.cfg.def>`.

Django settings should be configured in following form (with preserving
sections case)::

FABFILE = {
'DEFAULT': {
'LOGGING_FOLDER': '.',
},
'Credentials': {
'AWS_ACCESS_KEY_ID': 'AKIAABCDEFGHIJKLMNOP',
'AWS_SECRET_ACCESS_KEY': 'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLM',
},
}

.. note::
Boto config files http://code.google.com/p/boto/wiki/BotoConfig will be
examined at least - if ``[Credentials]`` or some other options was filled
neither in Django settings nor in ``./fabfile.cfg`` or :download:`django_fabfile/fabfile.cfg.def <../django_fabfile/fabfile.cfg.def>`.

Example of required AWS IAM Policy
----------------------------------

For using Fabric tasks you need AWS account with following permissions:

* create-snapshot::

{
"Statement": [
{
"Sid": "Stmt1304003463574",
"Action": [
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:DescribeInstances",
"ec2:DescribeRegions",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes"
],
"Effect": "Allow",
"Resource": "*"
}
]
}

* delete-snapshot::

{
"Statement": [
{
"Sid": "Stmt1306410750989",
"Action": [
"ec2:DescribeRegions",
"ec2:DescribeSnapshots",
"ec2:DeleteSnapshot"
],
"Effect": "Allow",
"Resource": "*"
}
]
}

* reboot-instance::

{
"Statement": [
{
"Sid": "Stmt1312204628195",
"Action": [
"ec2:RebootInstances"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Sid": "Stmt1312276311113",
"Action": [
"ec2:DetachVolume"
],
"Effect": "Allow",
"Resource": "*"
}
]
}

* rsync-snapshot::

{
"Statement": [
{
"Effect": "Allow",
"Action": [
"EC2:AttachVolume",
"EC2:CreateKeyPair",
"EC2:CreateVolume",
"EC2:DeleteKeyPair",
"EC2:DeleteVolume",
"EC2:DescribeAvailabilityZones",
"EC2:DescribeImages",
"EC2:DescribeKeyPairs",
"EC2:RunInstances",
"EC2:TerminateInstances"
],
"Resource": "*"
}
]
}

Backup script
-------------

.. note:: Instances and snaphots are filtered by tag "Earmarking" value
"production". This can be configured in settings - see "Pre-run
configuration" section above.

Following backup commands may be configured with crontab to run daily::

#!/bin/sh

VE=/home/backup-agent/.virtualenvs/backup

. $VE/bin/activate && \
fab --keepalive=60 -f $VE/lib/python2.6/site-packages/django_fabfile/backup.py \
backup_instances_by_tag \
trim_snapshots \
rsync_region:src_region_name=us-east-1,dst_region_name=eu-west-1 \
rsync_region:src_region_name=us-west-1,dst_region_name=eu-west-1 \
rsync_region:src_region_name=eu-west-1,dst_region_name=us-east-1

With 30 production instances it tooks around 6 hours to replicate
snapshots updates. First replication tooks much more time obviously.

Running tests
-------------

The package has test cases to make sure, that all methods, being tested,
keep on the entire program logic. The test cases are using packages ``fudge``
and ``unittest``. The test files (``'test_*.py'``) are located in folder
``'/django_fabfile/tests/'``. To run tests from command line, implement::

python -m unittest discover django_fabfile

Recent changes
--------------

See :doc:`changelog` for recent updates.

Licensing
---------

The product is licensed by oDesk http://www.odesk.com/ under
:download:`GNU GENERAL PUBLIC LICENSE Version 3<license.txt>` except
portions with attached inline license information like
:func:`django_fabfile.backup._trim_snapshots`.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-fabfile-2013.08.29.2.tar.gz (70.6 kB view details)

Uploaded Source

File details

Details for the file django-fabfile-2013.08.29.2.tar.gz.

File metadata

File hashes

Hashes for django-fabfile-2013.08.29.2.tar.gz
Algorithm Hash digest
SHA256 10024f376ae691a3f5e5d1ea1283c40cc2189e65e7331859f7493e87c22fc7bf
MD5 1e61a616379c295f9ac4b453c7d62a15
BLAKE2b-256 4c7139b6725c8edbb26d054322e2cf4cd8ee93ec77ea03ed3c8504ee135fba8f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page