This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description
Redis Shard
###########

.. image:: https://img.shields.io/travis/zhihu/redis-shard.svg?style=flat
:target: https://travis-ci.org/zhihu/redis-shard
:alt: Build Status

.. image:: https://pypip.in/version/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: Latest Version

.. image:: https://pypip.in/py_versions/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: Supported Python versions

.. image:: https://pypip.in/license/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: License

A Redis sharding implementation.

Redis is great. It's fast, lightweight and easy to use. But when we want to store
a mass of data into one single instance, we may encounter some problems such as performance
degradation and slow recovery and we need to scale it.

Usage
=====

First, Create an RedisShardAPI instance with multiple nodes, node ``name`` **must be unique**::

from redis_shard.shard import RedisShardAPI

servers = [
{'name': 'server1', 'host': '127.0.0.1', 'port': 10000, 'db': 0},
{'name': 'server2', 'host': '127.0.0.1', 'port': 11000, 'db': 0},
{'name': 'server3', 'host': '127.0.0.1', 'port': 12000, 'db': 0},
]

client = RedisShardAPI(servers, hash_method='md5')

Then, you can access the Redis cluster as you use `redis-py <https: github.com="" andymccurdy="" redis-py="">`_::

client.set('test', 1)
client.get('test') # 1
client.zadd('testset', 'first', 1)
client.zadd('testset', 'second', 2)
client.zrange('testset', 0, -1) # [first, second]


Hash Tags
---------

If you want to store specific keys on one node for some reason (such as you prefer single instance pipeline, or
you want to use multi-keys command such as ``sinter``), you should use Hash Tags::

client.set('foo', 2)
client.set('a{foo}', 5)
client.set('b{foo}', 5)
client.set('{foo}d', 5)
client.set('d{foo}e', 5)

client.get_server_name('foo') == client.get_server_name('a{foo}') == client.get_server_name('{foo}d') \
== client.get_server_name('d{foo}e') # True

The string in a braces of a key is the Hash Tag of the key. The hash of a Hash Tag will be treated the hash of the key.
So, keys ``foo``, ``bar{foo}`` and ``b{foo}ar`` will be sotred in the same node.

Tag method
~~~~~~~~~~~

Just add ``tag_`` prefix, you can use any of the normal redis method on the same hash tag::

client.tag_mget("{user:1}question1", "{user:1}question2")
client.tag_delete("{user:1}question1", "{user:1}question2")


Multi-keys method
~~~~~~~~~~~~~~~~~~
Only support ``mget``, ``mset`` and ``flushdb``.


Config Details
--------------
There are three parameters ``servers``, ``hash_method`` and ``sentinel`` in the :class:`redis_shard.shard.RedisShardAPI`.

``servers`` is a list. Each element in it should be a dict or a URL schema.

- dict::

[
{'name': 'server1', 'host': '127.0.0.1', 'port': 10000, 'db': 0},
{'name': 'server2', 'host': '127.0.0.1', 'port': 11000, 'db': 0, 'max_connections': 50},
{'name': 'server3', 'host': '127.0.0.1', 'port': 12000, 'db': 0},
]

- URL schema::

[
'redis://127.0.0.1:10000/0?name=node1',
'redis://127.0.0.1:11000/0?name=node2&max_connections=50',
'redis://127.0.0.1:12000/0?name=node3'
]

If the following parameter ``sentinel`` is enabled, only **name** is needed for the ``servers`` config.

``hash_method`` is a string which indicate the method of generating the hash key of the consistent hash ring.
The default value is **crc32**. It also supports **md5** and **sha1**.


``sentinel`` is the config for `Redis Sentinel <http: redis.io="" topics="" sentinel="">`_. With the sentinel support, redis-shard
will do read/write splitting. Config is like this::

{"hosts": [('localhost', 26379)], "socket_timeout": 0.1}



Limitations
===========

* Redis Shard dose not support all Redis commands.
* As mentioned above, Redis Shard does not support all multi-keys commands crossing different nodes,
you have to use Hash Tag to work with those commands.
* Redis Shard does not have any replication mechanism.


How it Works
============

Redis Shard is basically inspired by `this article <http: oldblog.antirez.com="" post="" redis-presharding.html="">`_.


Change History
==============

0.3.4 (2016-07-19)
------------------
- fix md5 hash_method occus NameError on python3

0.3.3 (2016-03-18)
-------------------
- max_connections can now be set when instantiating client instance, and has a default value 100.

0.3.2 (2015-06-02)
--------------------
- add strict redis support

0.3.1 (2015-02-02)
------------------
- transaction support

0.3.0 (2014-11-20)
------------------
- shard key generate function now support **md5** and **sha1** .
- add Redis Sentinel support

0.2.4 (2014-06-26)
------------------
- remove gevent dependency

0.2.3 (2014-06-02)
------------------
- better pipeline support

0.2.1 (2013-08-07)
------------------
- add evel method

0.2.0 (2013-06-02)
------------------
- add python3 support

0.1.11 (2013-05-06)
-------------------
- add mset support

0.1.10 (2013-04-13)
-------------------
- add mget support ,thks to @Yuekui

0.1.9 (2013-03-04)
------------------
- add an reshard example
- tidy the pipeline code
- add more shard methods

0.1.8 (2013-01-21)
------------------
- add `append` and `getrange` method, thks @simon-liu

0.1.7 (2012-11-19)
------------------
- use new redis url config instead of dict

0.1.5 (2012-07-16)
------------------
- Add many new methods, support socket_timeout and password

0.1.4 (2011-07-20)
------------------
- modify hash key algor, support suffix match, thks to dkong
- support more redis methods, include `keys`.

0.1.3 (2011-06-20)
------------------
- support 2.4.X version of redis-py

0.1.2 (2011-06-01)
------------------
- add MANIFEST.in file

0.1.1 (2011-05-29)
------------------
- create hashring use server's name config.

0.1 (2011-05-28)
----------------
- first version
Release History

Release History

0.3.4

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.11

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.10

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.9

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.8

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.7

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
redis-shard-0.3.4.tar.gz (12.7 kB) Copy SHA256 Checksum SHA256 Source Jul 19, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting