Skip to main content

Django Redis sdk

Project description

Django Redis SDK

0.1.1.0

General Bug Fixes

0.1.0.0

Django Redis SDK: A SDK for connecting to Redis server from Django.

  • Supports both TCP socket connection and Unix socket connection
  • Plug and play architecture.
  • Everything is configurable.
  • Supports base client connection in master/slave configuration (Out of box with DjangoRedisSDKCache backend class).
  • Supports sharded client connection configuration (Out of box with DjangoRedisSDKShrededCache backend class).
  • You have Dummy Cache backend for just testing or monkey patching (Achieved using DjangoRedisSDKDummyCache backend class).
  • A wrapper around redis package.
  • Pluggable CLIENT, REDIS_CLIENT, PARSER, COMPRESSOR, SERIALIZER,
  • Out of the box support for Master-Slave and Shard client.
  • supports all default apis.
  • supports redis cluster; requires redis-py-cluster

Dependancies

QuickStart

Installation and Basic Configuration

  1. Install Django Redis SDK by running pip install django-redis-sdk.
  2. Make changes in you settings file to accommodate the cache settings.
# DjangoRedisSDKCache -> Master - slave connection
CACHES = {
    'default': {
        'BACKEND': 'django_redis_sdk.DjangoRedisSDKCache',
        'KEY_PREFIX': 'production',
        'LOCATION': [
            '[<scheme>://][:password@]<host>:<port>',  # Master
            '[<scheme>://][:password@]<host>:<port>',  # slave 1
            '[<scheme>://][:password@]<host>:<port>',  # slave 2
            # ...
        ],
        'OPTIONS': {
            'DB': 0,
            'PASSWORD': 'passwd',
            'CLIENT_CLASS': 'django_redis_sdk.clients.BaseClient',
            'PARSER_CLASS': 'redis.connection.DefaultParser',
            'CONNECTION_POOL_CLASS': 'redis.connection.ConnectionPool',
            'CONNECTION_POOL_CLASS_KWARGS': {
                'max_connections': 14,
            },
            'SERIALIZER_CLASS': 'django_redis_sdk.serializers.PickleSerializer',
            'PICKLE_VERSION': -1,
            'SERIALIZER_CLASS_KWARGS': {
                'PICKLE_VERSION': -1,
            },
            'COMPRESSOR_CLASS': 'django_redis_sdk.compressors.DummyCompressor',
            'COMPRESS_LEVEL': 5,
            'COMPRESSOR_CLASS_KWARGS': {
                'COMPRESS_LEVEL': 5,
            },
            'REDIS_CLIENT_CLASS': 'redis.client.StrictRedis',
            'REDIS_CLIENT_KWARGS': {},
            'SOCKET_CONNECT_TIMEOUT': 5,  # in seconds; 5000 milliseconds,
            'SOCKET_TIMEOUT': 1,  # in seconds; 1000 milliseconds,
            'HANDLE_EXCEPTIONS': True,
            'LOG_EXCEPTIONS': True,
            'LOGGER_NAME': 'django_redis_sdk',
        }
    }
}

# DjangoRedisSDKShrededCache -> Shard
CACHES = {
    'default': {
        'BACKEND': 'django_redis_sdk.DjangoRedisSDKShrededCache',
        'KEY_PREFIX': 'production',
        'LOCATION': [
            '[<scheme>://][:password@]<host>:<port>[/db]',  # read-write 1
            '[<scheme>://][:password@]<host>:<port>[/db]',  # read-write 2
            '[<scheme>://][:password@]<host>:<port>[/db]',  # read-write 3
            # ...
        ],
        'OPTIONS': {
            'DB': 0,
            'PASSWORD': 'passwd',
            'CLIENT_CLASS': 'django_redis_sdk.clients.BaseClient',
            'PARSER_CLASS': 'redis.connection.DefaultParser',
            'CONNECTION_POOL_CLASS': 'redis.connection.ConnectionPool',
            'CONNECTION_POOL_CLASS_KWARGS': {
                'max_connections': 14,
            },
            'SERIALIZER_CLASS': 'django_redis_sdk.serializers.PickleSerializer',
            'PICKLE_VERSION': -1,
            'SERIALIZER_CLASS_KWARGS': {
                'PICKLE_VERSION': -1,
            },
            'COMPRESSOR_CLASS': 'django_redis_sdk.compressors.DummyCompressor',
            'COMPRESS_LEVEL': 5,
            'COMPRESSOR_CLASS_KWARGS': {
                'COMPRESS_LEVEL': 5,
            },
            'REDIS_CLIENT_CLASS': 'redis.client.StrictRedis',
            'REDIS_CLIENT_KWARGS': {},
            'SOCKET_CONNECT_TIMEOUT': 5,  # in seconds; 5000 milliseconds,
            'SOCKET_TIMEOUT': 1,  # in seconds; 1000 milliseconds,
            'HANDLE_EXCEPTIONS': True,
            'LOG_EXCEPTIONS': True,
            'LOGGER_NAME': 'django_redis_sdk',
        }
    }
}

# Cluster -> cluster
CACHES = {
    'default': {
        'BACKEND': 'django_redis_sdk.DjangoRedisSDKCache',
        'KEY_PREFIX': 'production',
        'LOCATION': [
            '[<scheme>://]<host>:<port>[/db]',  # cluster 1
            '[<scheme>://]<host>:<port>[/db]',  # cluster 2
            '[<scheme>://]<host>:<port>[/db]',  # cluster 3
            # ...
        ],
        'OPTIONS': {
            'PARSER_CLASS': 'redis.connection.DefaultParser',
            'CONNECTION_POOL_CLASS': 'rediscluster.connection.ClusterConnectionPool',
            'CONNECTION_POOL_CLASS_KWARGS': {
                'max_connections': 14,
                'skip_full_coverage_check': True,  # some redis implementation has disabled the CONFIG
            },
            'SERIALIZER_CLASS': 'django_redis_sdk.serializers.PickleSerializer',
            'PICKLE_VERSION': -1,
            'SERIALIZER_CLASS_KWARGS': {
                'PICKLE_VERSION': -1,
            },
            'COMPRESSOR_CLASS': 'django_redis_sdk.compressors.DummyCompressor',
            'COMPRESS_LEVEL': 5,
            'COMPRESSOR_CLASS_KWARGS': {
                'COMPRESS_LEVEL': 5,
            },
            'REDIS_CLIENT_CLASS': 'rediscluster.RedisCluster',
            'REDIS_CLIENT_KWARGS': {},
            'SOCKET_CONNECT_TIMEOUT': 5,  # in seconds; 5000 milliseconds,
            'SOCKET_TIMEOUT': 1,  # in seconds; 1000 milliseconds,
            'HANDLE_EXCEPTIONS': True,
            'LOG_EXCEPTIONS': True,
            'LOGGER_NAME': 'django_redis_sdk',
        }
    }
}

Basic Usage

Django Redis SDK has same backend apis as Django with some additions

example:

>>>from django.core.cache import cache
>>>cache.set('foo', 'bar', timeout=5)  # key = foo, value = 'bar' and valid for 5 seconds
>>>cache.ttl('foo')
5
>>>cache.get('foo')
bar
>>>cache.set_many({'foo':'boo', 'bar': 'baz'})
>>>cache.get_many(['foo', 'baz'])
OrderedDict([('foo', 'boo'), ('bar', 'baz')])
>>>cache.get_or_set('faz', 'baz', timeout=5)  # key = 'faz', dafault='baz' and valid for 5 seconds
baz  # since a key is not the value will be first set and then returned.
>>>import random
>>>cache.get_or_set('fazo', random.random, timeout=5)  # the default can take a callable also as input.
0.32685093104745067  # since the key is not set the random.random is called and the value is set and returned.

BACKEND values

  • django_redis_sdk.DjangoRedisSDKCache # for single namespace
  • django_redis_sdk.DjangoRedisSDKShrededCache # for sharded namespace

LOCATION values

  • String: comma seperated string for multiple server, single string value for single server.

  • List: single server in list for single server, multiple list value for multiple servers.

  • scheme:

    host:port -> 127.0.0.1:6379 -> db defaults to DB option value or ‘0’. host:port/db -> 127.0.0.1:6379/1 -> db is taken from url. :password@host:port -> :myPasswd@127.0.0.1:6379/1 -> password taken from url as opposed to None or value in options. /path/to/the/unix/socket -> /etc/redis/connection.sock -> uses unix socket for communication url_scheme://<combinations of above values>

    • redis://:passwd@127.0.0.0:6379/1
    • rediss://localhost:6379/1 –> ssl connection
    • unix://path/to/the/unix/socket –> unix socket connection

    NOTE: if url_scheme is not specified, we try to best assume the url_scheme; however it is best to provide the scheme.

OPTIONS

DB

Default: 0

The URL specified db has precedence over this one. If you with to see the cached values through redis-cli please select the db you assigned before querying by SELECT <db>

PASSWORD

Default: None

The URL specified password has precedence over this one. Ideally the REDIS server will be deployed inside a secure network with no access from outside; So, there wouldn’t be a password set in that case. But if you have password set, Please configure it here.

CLIENT_CLASS

Default: According to the BACKEND.

  • django_redis_sdk.clients.BaseClient # used as default in DjangoRedisSDKCache backend
  • django_redis_sdk.clients.ShardedClient # used as default in DjangoRedisSDKShrededCache backend

PARSER_CLASS

Default: redis.connection.DefaultParser

  • redis.connection.PythonParser
  • redis.connection.HiredisParser # requires hiredis pip install hiredis
  • redis.connection.DefaultParser # automatically chooses between python or hiredis (if hiredis available then hiredis else python)

CONNECTION_POOL_CLASS

Default: redis.connection.ConnectionPool

Apply kwargs if any through CONNECTION_POOL_CLASS_KWARGS options for this class.

  • redis.connection.ConnectionPool # takes additional kwargs max_connections
  • redis.connection.BlockingConnectionPool # takes additional kwargs max_connections, timeout
  • rediscluster.connection.ClusterConnectionPool # requires redis-py-cluster pip install redis-py-cluster

SERIALIZER_CLASS

Default: django_redis_sdk.serializers.PickleSerializer

Apply kwargs if any through SERIALIZER_CLASS_KWARGS options for this class.

  • django_redis_sdk.serializers.PickleSerializer # python pickle, takes PICKLE_VERSION options; defaults to -1
  • django_redis_sdk.serializers.DummySerializer # no serialization
  • django_redis_sdk.serializers.JsonSerializer # json.loads and json.dumbs

PICKLE_VERSION

Default: -1 # for custom SERIALIZER_CLASS you should configure the default value.

Used along with SERIALIZER_CLASS=django_redis_sdk.serializers.PickleSerializer; Otherwise no effect.

COMPRESSOR_CLASS

Default: django_redis_sdk.compressors.DummyCompressor

Apply kwargs if any through COMPRESSOR_CLASS_KWARGS options for this class.

  • django_redis_sdk.compressors.DummyCompressor # no compression
  • django_redis_sdk.compressors.ZlibCompressor # requires zlib to compress and decompress, takes COMPRESS_LEVEL

COMPRESS_LEVEL

Default: 5 # for custom COMPRESSOR_CLASS you should configure the default value.

  • Allowed values 0 to 9
  • 0 no compression.
  • 9 full compression.

REDIS_CLIENT_CLASS

Default: redis.client.Redis

Apply kwargs if any through REDIS_CLIENT_KWARGS option for this class.

  • redis.client.Redis
  • redis.client.StrictRedis # in redis>=3.4.1 this is same as redis.client.Redis
  • rediscluster.RedisCluster # requires redis-py-cluster ; install using pip install redis-py-cluster. For cluster support.

SOCKET_CONNECT_TIMEOUT

Default: None # means wait infinitely

The maximum allowed time to wait to make a connection.

SOCKET_TIMEOUT

Default: None # means wait infinitely

The maximum allowed time to wait for an operation to wait (wait for response once after the connection is made).

HANDLE_EXCEPTIONS

Default: False

Whether to handle exceptions gracefully or propagate it? The exceptions defined in django_redis_sdk.utils.EXCEPTIONS_TO_HANDLE are caught and handled gracefully

LOG_EXCEPTIONS

Default: False

Whether to log the exceptions While handling the exceptions. Used along with HANDLE_EXCEPTIONS.

LOGGER_NAME

Default: __name__

Defines which python logger to send the logs to while logging the exceptions. Used along with LOG_EXCEPTIONS and HANDLE_EXCEPTIONS.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for django-redis-sdk, version 0.1.1.0
Filename, size File type Python version Upload date Hashes
Filename, size django_redis_sdk-0.1.1.0-py3-none-any.whl (25.7 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size django-redis-sdk-0.1.1.0.tar.gz (16.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page