Skip to main content

a Python interface to a Cluster of Redis key-value store

Project description

a Python interface to a Cluster of Redis key-value stores.

Project Goals

The goal is to be a drop in replacement of redis-py when you would like to shard your data among different Redis instances in a transparent, fast, and fault tolerant way. rediscluster-py is based on the awesome redis-py StrictRedis Api, thus the original api commands would work without problems within the context of a cluster of redis servers

Travis CI

Currently, rediscluster-py is being tested via travis ci for python version 2.6, 2.7 and 3.2: Build Status


$ sudo pip install rediscluster

or alternatively (you really should be using pip though):

$ sudo easy_install rediscluster

From source:

$ sudo python install

Running Tests

$ git clone
$ cd rediscluster-py
$ vi tests/
$ ./run_tests

Getting Started

>>> import rediscluster
>>> cluster = {
...          # node names
...          'nodes' : { # masters
...                      'node_1' : {'host' : '', 'port' : 63791},
...                      'node_2' : {'host' : '', 'port' : 63792},
...                      # slaves
...                      'node_3' : {'host' : '', 'port' : 63793},
...                      'node_4' : {'host' : '', 'port' : 63794}
...                    },
...          # replication information
...          'master_of' : {
...                          'node_1' : 'node_4', # node_4 slaveof node_1 in redis4.conf
...                          'node_2' : 'node_3'  # node_3 slaveof node_2 in redis3.conf
...                        },
...          'default_node' : 'node_1'
...     }
>>> r = rediscluster.StrictRedisCluster(cluster=cluster, db=0)
>>> r.set('foo', 'bar')
>>> r.get('foo')

Cluster Configuration

The cluster configuration is a hash that is mostly based on the idea of a node, which is simply a host:port pair that points to a single redis-server instance. This is to make sure it doesn’t get tied it to a specific host (or port). The advantage of this is that it is easy to add or remove nodes from the system to adjust the capacity while the system is running.

Read Slaves & Write Masters

rediscluster uses master/slave mappings stored in the cluster hash passed during instantiation to transparently relay read redis commands to slaves and writes commands to masters.

Partitioning Algorithm

rediscluster doesn’t used a consistent hashing like some other libraries. In order to map every given key to the appropriate Redis node, the algorithm used, based on crc32 and modulo, is :

(abs(binascii.crc32(<key>) & 0xffffffff) % <number of masters>) + 1

this is used to ensure some compatibility with other languages, php in particular. A function getnodefor is provided to get the node a particular key will be/has been stored to.

>>> r.getnodefor('foo')
{'node_2': {'host': '', 'port': 63792}}

Tagged keys

In order to specify your own hash key (so that related keys can all land on a given node), rediscluster allows you to pass a list where you’d normally pass a scalar. The first element of the list is the key to use for the hash and the second is the real key that should be fetched/modify:

>>> r.get(["userinfo", "foo"])

In that case “userinfo” is the hash key but “foo” is still the name of the key that is fetched from the redis node that “userinfo” hashes to.

Multiple Keys Redis Commands

In the context of storing an application data accross many redis servers, commands taking multiple keys as arguments are harder to use since, if the two keys will hash to two different instances, the operation can not be performed. Fortunately, rediscluster is a little fault tolerant in that it still fetches the right result for those multi keys operations as far as the client is concerned. To do so it processes the related involved redis servers at interface level.

>>> r.sadd('foo', *['a1', 'a2', 'a3'])
>>> r.sadd('bar', *['b1', 'a2', 'b3'])
>>> r.sdiffstore('foobar', 'foo', 'bar')
>>> r.smembers('foobar')
set(['a1', 'a3'])
>>> r.getnodefor('foo')
{'node_2': {'host': '', 'port': 63792}}
>>> r.getnodefor('bar')
{'node_1': {'host': '', 'port': 63791}}
>>> r.getnodefor('foobar')
{'node_2': {'host': '', 'port': 63792}}

Redis-Sharding & Redis-Copy

In order to help with moving an application with a single redis server to a cluster of redis servers that could take advantage of rediscluster, i wrote redis-sharding and redis-copy



rediscluster-py is developed and maintained by Salimane Adjao Moustapha ( It can be found here:

Project details

Release history Release notifications

History Node


History Node


History Node


History Node


History Node


History Node


This version
History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
rediscluster-0.2.7-py2.7.egg (13.6 kB) Copy SHA256 hash SHA256 Egg 2.7 Sep 27, 2012
rediscluster-0.2.7-py3.2.egg (13.8 kB) Copy SHA256 hash SHA256 Egg 3.2 Sep 27, 2012
rediscluster-0.2.7.tar.gz (10.0 kB) Copy SHA256 hash SHA256 Source None Sep 27, 2012
rediscluster-0.2.7.win32-py2.7.exe (76.4 kB) Copy SHA256 hash SHA256 Windows Installer 2.7 Sep 27, 2012
rediscluster-0.2.7.win32-py3.2.exe (76.4 kB) Copy SHA256 hash SHA256 Windows Installer 3.2 Sep 27, 2012

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page