Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Django cache backend for Amazon ElastiCache (memcached)

Project Description

Simple Django cache backend for Amazon ElastiCache (memcached based). It uses pylibmc and sets up a connection to each node in the cluster using auto discovery.

Requirements

  • pylibmc
  • Django 1.5+.

It was written and tested on Python 2.7 and 3.4.

Installation

Get it from pypi:

pip install django-elasticache

or github:

pip install -e git://github.com/gusdan/django-elasticache.git#egg=django-elasticache

Usage

Your cache backend should look something like this:

CACHES = {
    'default': {
        'BACKEND': 'django_elasticache.memcached.ElastiCache',
        'LOCATION': 'cache-c.draaaf.cfg.use1.cache.amazonaws.com:11211',
        'OPTIONS' {
            'IGNORE_CLUSTER_ERRORS': [True,False],
        },
    }
}

By the first call to cache it connects to cluster (using LOCATION param), gets list of all nodes and setup pylibmc client using full list of nodes. As result your cache will work with all nodes in cluster and automatically detect new nodes in cluster. List of nodes are stored in class-level cached, so any changes in cluster take affect only after restart of working process. But if you’re using gunicorn or mod_wsgi you usually have max_request settings which restart process after some count of processed requests, so auto discovery will work fine.

The IGNORE_CLUSTER_ERRORS option is useful when LOCATION doesn’t have support for config get cluster. When set to True, and config get cluster fails, it returns a list of a single node with the same endpoint supplied to LOCATION.

Django-elasticache changes default pylibmc params to increase performance.

Another solutions

ElastiCache provides memcached interface so there are three solution of using it:

1. Memcached configured with location = Configuration Endpoint

In this case your application will randomly connect to nodes in cluster and cache will be used with not optimal way. At some moment you will be connected to first node and set item. Minute later you will be connected to another node and will not able to get this item.

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
        'LOCATION': 'cache.gasdbp.cfg.use1.cache.amazonaws.com:11211',
    }
}

2. Memcached configured with all nodes

It will work fine, memcache client will separate items between all nodes and will balance loading on client side. You will have problems only after adding new nodes or delete old nodes. In this case you should add new nodes manually and don’t forget update your app after all changes on AWS.

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
        'LOCATION': [
            'cache.gqasdbp.0001.use1.cache.amazonaws.com:11211',
            'cache.gqasdbp.0002.use1.cache.amazonaws.com:11211',
        ]
    }
}

3. Use django-elasticache

It will connect to cluster and retrieve ip addresses of all nodes and configure memcached to use all nodes.

CACHES = {
    'default': {
        'BACKEND': 'django_elasticache.memcached.ElastiCache',
        'LOCATION': 'cache-c.draaaf.cfg.use1.cache.amazonaws.com:11211',
    }
}

Difference between setup with nodes list (django-elasticache) and connection to only one configuration Endpoint (using dns routing) you can see on this graph:

Testing

Run the tests like this:

nosetests
Release History

Release History

This version
History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

History Node

0.0.4

History Node

0.0.3

History Node

0.0.2

History Node

0.0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
django-elasticache-1.0.2.tar.gz (5.1 kB) Copy SHA256 Checksum SHA256 Source May 2, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting