aioredis-cluster 1.6.0b1

Redis Cluster support extension for aioredis

aioredis_cluster

Redis Cluster support for aioredis.

Many implementation features were inspired by go-redis project.

Requirements

• Python 3.6.5+
• aioredis 1.1+
• attrs 18.2+
• async_timeout

Features

• commands execute failover (retry command on other node in cluster)
• support resharding replies ASK/MOVED
• restore cluster state from alive nodes
• one node is enough to know the topology and initialize client
• cluster state auto reload

Limitations

Commands with limitations

• Keys in mget/mset must provide one key slot.

  # works
  await redis.mget("key1:{foo}", "key2:{foo}")
  
  # throw RedisClusterError
  await redis.mget("key1", "key2")
  

Commands are not supported

Redis methods below do not works and not supported in cluster mode implementation.

cluster_add_slots
cluster_count_failure_reports
cluster_count_key_in_slots
cluster_del_slots
cluster_failover
cluster_forget
cluster_get_keys_in_slots
cluster_meet
cluster_replicate
cluster_reset
cluster_save_config
cluster_set_config_epoch
cluster_setslot
cluster_readonly
cluster_readwrite
client_setname
shutdown
slaveof
script_kill
move
select
flushall
flushdb
script_load
script_flush
script_exists
scan
iscan
quit
swapdb
migrate
migrate_keys
wait
bgrewriteaof
bgsave
config_rewrite
config_set
config_resetstat
save
sync
pipeline
multi_exec

But you can always execute command you need on concrete node on cluster with usual aioredis.RedisConnection, aioredis.ConnectionsPool or high-level aioredis.Redis interfaces.

Installation

pip install aioredis-cluster

Usage

import aioredis_cluster

redis = await aioredis_cluster.create_redis_cluster([
    "redis://redis-cluster-node1",
])

# or
redis = await aioredis_cluster.create_redis_cluster([
    "redis://redis-cluster-node1",
    "redis://redis-cluster-node2",
    "redis://redis-cluster-node3",
])

# or
redis = await aioredis_cluster.create_redis_cluster([
    ("redis-cluster-node1", 6379),
])

await redis.set("key", "value", expire=180)

redis.close()
await redis.wait_closed()

License

The aioredis_cluster is offered under MIT license.

Changes

1.6.0 (2021-11-xx)

• make public Address, ClusterNode and ClusterState structs. Available by import from aioredis_cluster import
• Cluster provides some new helpful methods^
  • get_master_node_by_keys(*keys) - return master ClusterNode which contains keys keys
  • create_pool_by_addr(addr, **kwargs) - create connection pool by addr and return pool wrapped by commands_factory from Cluster constructor. By default is aioredis_cluster.RedisCluster instance.
  • get_cluster_state() - return ClusterState instance with recent known cluster state received from Redis cluster
• drop pytest-aiohttp plugin for tests
• add pytest-asyncio dependency for tests
• switch asynctest -> mock library for aio tests

1.5.2 (2020-12-14)

• README update

1.5.1 (2020-12-11)

• speedup crc16. Use implementation from python stdlib

1.5.0 (2020-12-10)

• remove state_reload_frequency from ClusterManager. state_reload_interval now is one relevant option for state auto reload
• default state_reload_interval increased and now is 300 seconds (5 minutes)
• commands registry loads only once, on cluster state initialize
• improve failover. First connection problem cause retry to random slot replica
• improve python3.9 support
• default idle_connection_timeout now is 10 minutes

1.4.0 (2020-09-08)

• fix aioredis.locks.Lock issue (https://github.com/aio-libs/aioredis/pull/802, bpo32734)
• now aioredis_cluster.Cluster do not acquire dedicate connection for every execute
• aioredis_cluster now requires python>=3.6.5

1.3.0 (2019-10-23)

• improve compatible with Python 3.8
• improve failover logic while command timed out
• read-only commands now retries if attempt_timeout is reached
• add required dependeny async_timeout
• aioredis dependency bound now is aioredis >=1.1.0, <2.0.0

1.2.0 (2019-09-10)

• add timeout for command execution (per execution try)
• add Cluster option attempt_timeout for configure command execution timeout, default timeout is 5 seconds
• Cluster.execute_pubsub() fixes

1.1.1 (2019-06-07)

• CHANGES fix

1.1.0 (2019-06-06)

• Cluster state auto reload
• new state_reload_frequency option to configure state reload frequency
• new state_reload_interval option to configure state auto reload interval
• follow_cluster option enable load cluster state from previous cluster state nodes
• establish connection only for master nodes after cluster state load
• change default commands_factory to aioredis_cluster.RedisCluster instead aioredis.Redis
• all cluster info commands always returns structs with str, not bytes
• keys_master and all_masters methods now try to ensure cluster state instead simply raise exception if connection lost to cluster node, for example
• max_attempts always defaults fix

1.0.0 (2019-05-29)

• Library full rewrite
• Cluster state auto reload
• Command failover if cluster node is down or key slot resharded

0.2.0 (2018-12-27)

• Pipeline and MULTI/EXEC cluster implementation with keys distribution limitation (because cluster)

0.1.1 (2018-12-26)

• Python 3.6+ only

0.1.0 (2018-12-24)

• Initial release based on aioredis PR (https://github.com/aio-libs/aioredis/pull/119)

Contributors

• Anton Ilyushenkov