Skip to main content

Cachetools Utilities

Project description

cachetools-utils

Classes to add key prefix and stats to cachetools classes and use redis and memcached as storage backends, and other cache-related utils.

Thoughts about Caching

Caching is a key component of any significant Web or REST backend so as to avoid performance issues when accessing the storage tier, in term of latency, throughput and resource usage.

  • Shared Cache

    A convenient setup is to have one shared cache storage tier at the application level, which is accessed through wrappers to avoid collisions between cache functions, basically by prepending keys with some prefix.

    Depending on the access pattern, it may or may not be useful to put a multiple-level caching strategy in place, with a local in-process cache and a higher-level inter-process and inter-host cache.

    When using a global shared cache, it should be clear that the cache may hold sensitive data and its manipulation may allow to change the behavior for the application, including working around security by tampering with the application authentication and authorizations guards.

  • Latency

    In order to reduce latency, most time should be spent in network accesses, so reducing the number of trips is a key strategy. This suggests combining data transfers where possible through higher-level queries, both at the HTTP level and at the database level.

    Denormalizing the data model may help. Having an application-oriented view of the model (eg JSON objects rather than attributes and tables) can help performance, at the price of losing some of the consistency warranties provided by a database. The best of both word may be achieved, to some extent, by storing JSON data into a database such as Postgres.

    Invalidating data from the cache requires a detailed knowledge of internal cache operations and are not very easy to manage at the application level, so devops should want to avoid this path if possible.

  • Throughput

    Write operations need to be sent to storage. Depending on transaction requirements, i.e. whether rare some data loss is admissible, various strategies can be applied, such as updating in parallel the cache and the final storage.

    Read operations can be cached, at the price of possibly having inconsistency data shown to users. LFU/LRU cache strategies mean that inconsistent data can be kept in cache for indefinite time, which is annoying. A TLL expiration on top of that makes such discrepancies bounded in time, so that after some time the data shown are eventually up to date.

Basically the application should aim at maximizing throughput for the available resources whilst keeping the latency under control, eg 90% of queries under some limit.

Module Documentation

This module provide the following cache wrappers suitable to use with cachetools:

  • Some classes provide actual storage or API to actual storage. For this purpose a cache is basically a key-value store, aka a dictionary, possibly with some constraints on keys (type, size) and values (size, serialization).

  • Other classes add features on top of these, such as using a prefix so that a storage can be shared without collisions or keeping usage and efficiency statistics.

PrefixedCache

Add a key prefix to an underlying cache to avoid key collisions.

import CacheToolsUtils as ctu

ct_base = cachetools.TTLCache(maxsize=1048576, ttl=600)
foo_cache = ctu.PrefixedCache(ct_base, "foo.")
bla_cache = ctu.PrefixedCache(ct_base, "bla.")

@cachetools.cached(cache=foo_cache)
def foo():
    return 

@cachetools.cached(cache=bla_cache)
def bla():
    return 

StatsCache

Keep stats, cache hit rate shown with hits().

scache = ctu.StatsCache(cache)

TwoLevelCache

Two-level cache, for instance a local in-memory cachetools cache for the first level, and a larger shared redis or memcached distributed cache for the second level. Whether such setting can bring performance benefits is an open question.

cache = ctu.TwoLevelCache(TTLCache(), RedisCache())

There should be some consistency between the two level configurations so that it make sense. For instance, having two TTL-ed stores would suggest that the secondary has a longer TTL than the primary.

MemCached

Basic wrapper, possibly with JSON key encoding thanks to the JsonSerde class.

import pymemcache as pmc

mc_base = pmc.Client(server="localhost", serde=ctu.JsonSerde())
cache = ctu.MemCached(mc_base)

@cachetools.cached(cache=cache)
def poc():

PrefixedMemCached

Wrapper with a prefix. A specific class is needed because of necessary key encoding.

pcache = ctu.PrefixedMemCached(mc_base, prefix="pic.")

StatsMemCached

Wrapper with stats actually taken from the MemCached server.

scache = ctu.StatsMemCached(pcache)

RedisCache

TTL'ed Redis wrapper, default ttl is 10 minutes.

import redis

rd_base = redis.Redis(host="localhost")
cache = ctu.RedisCache(rd_base, ttl=60)

PrefixedRedisCache

Wrapper with a prefix and a ttl. A specific class is needed because of key encoding and value serialization and deserialization.

pcache = ctu.PrefixedRedisCache(rd_base, "pac.", ttl=3600)

StatsRedisCache

Wrapper with stats (call hits()) and a ttl. Stats are actually taken from the Redis server.

scache = ctu.StatsRedisCache(pcache)

cacheMethods and cacheFunctions

This utility function create a prefixed cache around methods of an object or functions in the global scope. First parameter is the actual cache, second parameter is the object or scope, and finally a keyword mapping from function names to prefixes.

ctu.cacheMethods(cache, obj, get_data="1.", get_some="2.")
ctu.cacheFunctions(cache, globals(), some_func="f.")

Install

Install with pip:

pip install CacheToolsUtils

See above for example usage.

License

This code is public domain.

Versions

3.0 on 2022-03-06

Use simpler kwargs approach for caching methods and functions. Add a gen parameter for caching methods and functions. Improve documentation.

2.0 on 2022-02-24

Add cacheMethods and cacheFunctions. Improve documentation. 100% coverage test.

1.1.0 on 2022-01-30

Improve documentation. Add TwoLevelCache. Add 100% coverage test.

1.0.0 on 2022-01-29

Add set, get and delete forwarding to RedisCache, so that redis classes can be stacked.

0.9.0 on 2022-01-29

Initial version extracted from another project.

TODO

  • add a close?
  • rename hits hit_rate?
  • add other efficiency statistics?

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

CacheToolsUtils-3.0.tar.gz (10.4 kB view details)

Uploaded Source

Built Distribution

CacheToolsUtils-3.0-py3-none-any.whl (6.5 kB view details)

Uploaded Python 3

File details

Details for the file CacheToolsUtils-3.0.tar.gz.

File metadata

  • Download URL: CacheToolsUtils-3.0.tar.gz
  • Upload date:
  • Size: 10.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.8.10

File hashes

Hashes for CacheToolsUtils-3.0.tar.gz
Algorithm Hash digest
SHA256 055ea554ae2ec34aac1ffc646d1dc777afeba57d9d1552f36c18811e1c810c92
MD5 46ba40c9d43265e3a31a3d9499b8eeb7
BLAKE2b-256 9a9b05cd2805644ec26a25417fb6b6eeddf7646928507644c439ecf315d65562

See more details on using hashes here.

File details

Details for the file CacheToolsUtils-3.0-py3-none-any.whl.

File metadata

  • Download URL: CacheToolsUtils-3.0-py3-none-any.whl
  • Upload date:
  • Size: 6.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/3.8.10

File hashes

Hashes for CacheToolsUtils-3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8a2295a47158127b053f5973d50c6e8141b4c4e50447f22a6285f85180efeee5
MD5 281823406e1ea8040751ab2bedd79107
BLAKE2b-256 fcb39e36add9ffb0dc60754534b480c4a5420f171053ebc67e4ff2da8096d0cb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page