Redis Sharding API
Project description
Redis Shard
==============
A redis sharding api. Sharding is done based on the CRC32 checksum of a key or key tag ("key{key_tag}"),
according to this article http://antirez.com/post/redis-presharding.html .
The source code is locate at `github <https://github.com/youngking/redis-shard>`_ .
.. image:: https://travis-ci.org/youngking/redis-shard.png?branch=master
:alt: Build Status
Usage
==============
Creating a hash ring with multiple servers,By default the hash ring uses a crc32
hashing algorithm on the server's ``name`` config.You can define the name anything
as you like,but it must be unique.
I don't want to bind the hashring with ipaddress,because if I do some master/slave switches,
I can only change the ipaddress related config. The ``name`` is kept,so the hashring's order
is kept.
>>> from redis_shard.shard import RedisShardAPI
>>> servers = [
... {'name':'server1','host':'127.0.0.1','port':10000,'db':0},
... {'name':'server2','host':'127.0.0.1','port':11000,'db':0},
... {'name':'server3','host':'127.0.0.1','port':12000,'db':0},
... {'name':'127.0.0.1:13000','host':'127.0.0.1','port':13000,'db':0},
... ]
>>>
>>> client = RedisShardAPI(servers)
>>> client.set('test',1)
>>> print client.get('test')
>>> client.zadd('testset','first',1)
>>> client.zadd('testset','second',2)
>>> print client.zrange('testset',0,-1)
Hash tags
----------------
see article `http://antirez.com/post/redis-presharding.html` for detail.
>>> client.set('foo',2)
>>> client.set('a{foo}',5)
>>> client.set('b{foo}',5)
>>> client.set('{foo}d',5)
>>> client.set('d{foo}e',5)
>>> print client.get_server_name('foo') == client.get_server_name('a{foo}') == client.get_server_name('{foo}d') \
... == client.get_server_name('d{foo}e')
I also added an `tag_keys` method,which is more quickly than default `keys` method,because it only look
one machine.
>>> client.tag_keys('*{foo}*') == client.keys('*{foo}*')
Config Format
-------------------
There's three config formats
- list::
servers = [
{'name':'node1','host':'127.0.0.1','port':10000,'db':0},
{'name':'node2','host':'127.0.0.1','port':11000,'db':0},
{'name':'node3','host':'127.0.0.1','port':12000,'db':0},
]
- dict::
servers =
{ 'node1': {'host':'127.0.0.1','port':10000,'db':0},
'node2': {'host':'127.0.0.1','port':11000,'db':0},
'node3': {'host':'127.0.0.1','port':12000,'db':0},
}
- url_schema::
servers = ['redis://127.0.0.1:10000/0?name=node1',
'redis://127.0.0.1:11000/0?name=node2',
'redis://127.0.0.1:12000/0?name=node3'
]
Change History
==============
0.1.9 (2013-03-04)
-------------------
- add an reshard example
- tidy the pipeline code
- add more shard methods
0.1.8 (2013-01-21)
-------------------
add `append` and `getrange` method, thks @simon-liu
0.1.7 (2012-11-19)
-------------------
use new redis url config instead of dict
0.1.5 (2012-07-16)
-------------------
Add many new methods, support socket_timeout and password
0.1.4 (2011-07-20)
------------------
modify hash key algor,support suffix match,thks to dkong
support more redis methods,include `keys`.
0.1.3 (2011-06-20)
------------------
support 2.4.X version of redis-py
0.1.2 (2011-06-01)
------------------
add MANIFEST.in file
0.1.1 (2011-05-29)
------------------
create hashring use server's name config.
0.1 (2011-05-28)
------------------
first version
==============
A redis sharding api. Sharding is done based on the CRC32 checksum of a key or key tag ("key{key_tag}"),
according to this article http://antirez.com/post/redis-presharding.html .
The source code is locate at `github <https://github.com/youngking/redis-shard>`_ .
.. image:: https://travis-ci.org/youngking/redis-shard.png?branch=master
:alt: Build Status
Usage
==============
Creating a hash ring with multiple servers,By default the hash ring uses a crc32
hashing algorithm on the server's ``name`` config.You can define the name anything
as you like,but it must be unique.
I don't want to bind the hashring with ipaddress,because if I do some master/slave switches,
I can only change the ipaddress related config. The ``name`` is kept,so the hashring's order
is kept.
>>> from redis_shard.shard import RedisShardAPI
>>> servers = [
... {'name':'server1','host':'127.0.0.1','port':10000,'db':0},
... {'name':'server2','host':'127.0.0.1','port':11000,'db':0},
... {'name':'server3','host':'127.0.0.1','port':12000,'db':0},
... {'name':'127.0.0.1:13000','host':'127.0.0.1','port':13000,'db':0},
... ]
>>>
>>> client = RedisShardAPI(servers)
>>> client.set('test',1)
>>> print client.get('test')
>>> client.zadd('testset','first',1)
>>> client.zadd('testset','second',2)
>>> print client.zrange('testset',0,-1)
Hash tags
----------------
see article `http://antirez.com/post/redis-presharding.html` for detail.
>>> client.set('foo',2)
>>> client.set('a{foo}',5)
>>> client.set('b{foo}',5)
>>> client.set('{foo}d',5)
>>> client.set('d{foo}e',5)
>>> print client.get_server_name('foo') == client.get_server_name('a{foo}') == client.get_server_name('{foo}d') \
... == client.get_server_name('d{foo}e')
I also added an `tag_keys` method,which is more quickly than default `keys` method,because it only look
one machine.
>>> client.tag_keys('*{foo}*') == client.keys('*{foo}*')
Config Format
-------------------
There's three config formats
- list::
servers = [
{'name':'node1','host':'127.0.0.1','port':10000,'db':0},
{'name':'node2','host':'127.0.0.1','port':11000,'db':0},
{'name':'node3','host':'127.0.0.1','port':12000,'db':0},
]
- dict::
servers =
{ 'node1': {'host':'127.0.0.1','port':10000,'db':0},
'node2': {'host':'127.0.0.1','port':11000,'db':0},
'node3': {'host':'127.0.0.1','port':12000,'db':0},
}
- url_schema::
servers = ['redis://127.0.0.1:10000/0?name=node1',
'redis://127.0.0.1:11000/0?name=node2',
'redis://127.0.0.1:12000/0?name=node3'
]
Change History
==============
0.1.9 (2013-03-04)
-------------------
- add an reshard example
- tidy the pipeline code
- add more shard methods
0.1.8 (2013-01-21)
-------------------
add `append` and `getrange` method, thks @simon-liu
0.1.7 (2012-11-19)
-------------------
use new redis url config instead of dict
0.1.5 (2012-07-16)
-------------------
Add many new methods, support socket_timeout and password
0.1.4 (2011-07-20)
------------------
modify hash key algor,support suffix match,thks to dkong
support more redis methods,include `keys`.
0.1.3 (2011-06-20)
------------------
support 2.4.X version of redis-py
0.1.2 (2011-06-01)
------------------
add MANIFEST.in file
0.1.1 (2011-05-29)
------------------
create hashring use server's name config.
0.1 (2011-05-28)
------------------
first version
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
redis-shard-0.1.9.tar.gz
(6.7 kB
view hashes)