Clustered Caching Library
- numpy (for optimizations in FilesCacheClient)
- python-memcached (for MemcachedClient)
- pyzmq < 14 (for coherence support)
- dnspython (for dynamic dns-based load-balancing of MemcachedClient)
- cython (for optimizations in InprocCacheClient)
When declaring dependencies in your project, you can reference the following optional features:
- mq (everything under chorde.mq, mostly pulls pyzmq)
- shmem (shared memory utils, needed for optimized files cache clients)
The simplest way to use is to create one of the supported cache clients, and use it directly, like
from chorde.clients.inproc import InprocCacheClient from chorde import CacheMissError c = InprocCacheClient(200) c.put(3, 10, 300) # put value 10 on key 3, TTL 5min assert 10 == c.get(3) try: c.get(5) except CacheMissError: print "miss"
This creates an In-process LRU cache. The in-process part indicates that it is process-private, and not shared with other processes.
There are two implementations of the LRU, with different performance characteristics. The InprocCacheClient can take alternative store implementations as an argument, see the module for details.
The default LRUCache, accessible as chorde.clients.inproc.Cache, is a regular LRU implemented with a priority queue and a hash table in tandem, so it has O(log n) writes and O(1) reads, but by default all reads entail a write (to update the LRU). That can be disabled by specifying custom options, see the module’s documentation for more details.
There’s an alternative approximate LRU, accessible in chorde.clients.inproc.CuckooCache, that implements a lazy version of a cuckoo hash table, and has O(1) reads and amortized O(1) writes. It’s also quite more space-efficient than the regular LRU, so it’s better suited for very large caches, but its eviction strategy will be approximate, and thus not guaranteed to always evict the actual least-recently-used item.
A common approach when dealing with remote caches, like the above example using memcached, is to have at least two levels: the memcached itself, and an in-process small cache to avoid having to talk to the memcached all the time.
This can be done straightforwardly with the tiered clients:
from chorde.clients.memcached import MemcachedClient from chorde.clients.inproc import InprocCacheClient from chorde.clients.tiered import TieredInclusiveClient from chorde import CacheMissError l1 = InprocCacheClient(10) l2 = MemcachedClient(["localhost:11211"], checksum_key="test") c = TieredInclusiveClient(l1,l2) c.put(3, 10, 300) assert 10 == c.get(3) try: c.get(5) except CacheMissError: print "miss"
Here we build an inclusive tiered client, in which elements on higher levels are promoted into the lower levels by copying, rather than swapping. This means there is duplication among them, but this is usually best in cases like these, where the upper levels are shared among processes.
An exclusive client isn’t provided at this moment, since there is seldom any use for the exclusive pattern on these types of caches.
A more natural way to think about caching, is in that it’s a decorator of plain functions.
Rather than explicitly putting and getting from caches, one can simply consider caching as an optimization on an otherwise expensive function.
Decorators in chorde.decorators provide a huge amount of functionality and flexibility, these examples cover only the most basic usage:
Assuming c is the client we want to use for caching,
from chorde.decorators import cached import random @cached(c, ttl=300, async_ttl=-60) def expensive_func(x): return x * random.random() print expensive_func(1) print expensive_func(1) # Should return the same print expensive_func.async()(1) # will refresh asynchronously every minute print expensive_func.future()(1).result() # same as before, but using the futures interface print expensive_func.peek(1) # just check the cache print expensive_func.put(1, _cache_put=5) # write an explicit value print expensive_func.async().lazy(1) # don't wait, raise CacheMissError if not available, compute in background print expensive_func.future().lazy(1).result() # same as before, but using the futures interface
There, the async_ttl means the minimum TTL that triggers an asynchronous recomputation (you can use it to avoid ever having to wait on a recomputation). The negative value makes it relative to the total TTL, so -60 always means recompute every minute (60 seconds). The plain ttl is an absolute limit, no result older than that will ever be returned.
The documentation on chorde.decorators.cached will have more to say about the ways of invoking cached functions.
In general, the terms are:
- lazy: don’t wait for computation, return a cached result or raise CacheMissError. When combined with async, it will compute in the background.
- peek: don’t compute. Similar to lazy, but it will never trigger a computation
- async: expensive things (computation) happen on a background threadpool.
- future: return futures rather than results, use the future to get notified of results when they’re available. Actual cache access happens on a threadpool. A non-blocking way of calling.
- refresh: immediately recompute the value.
Integration with other libraries
The decorators’ future() interface is especially suited for integration with other libraries that can talk to futures. Chorde’s futures, however, are not directly compatible with other libraries’, but they can easily be wrapped like so:
import tornado.web import tornado.gen from chorde.clients.async import makeFutureWrapper WF = makeFutureWrapper(tornado.web.Future) ... @tornado.gen.coroutine def get(self): some_result = yield WF(some_func.future()(some_args))
There is a better way to integrate with tornado >= 4.0
from chorde.external_integration import monkey_patch_tornado monkey_patch_tornado() import tornado.web import tornado.gen ... @tornado.gen.coroutine def get(self): some_result = yield some_func.future()(some_args)