Skip to main content

A dependent cache manager

Project description

[![Build Status](https://travis-ci.org/OpenGov/py_cache_manager.svg?branch=master)](https://travis-ci.org/OpenGov/py_cache_manager)

# CacheMan
A Python interface for managing dependent caches.

'Ba-Bop-Ba-Dop-Bop'

## Description
This module acts as a dependency manager for caches and is ideal for instances
where a program has many repeated computations that could be safely persisted.
This usually entails a DB layer to house key value pairs. However, such a layer
is sometimes overkill and managing a DB along with a project can be more effort
than it's worth. That's where CacheMan comes in and provides an interface
through which you can define savers, loaders, builders, and dependencies with
disk-based defaults.

By default all caches will auto save when 10k changes occur over 60 seconds, 10
changes occur over 300 seconds (but after 60 seconds), or 1 change occurs within
900 seconds (after 300 seconds). This behavior can be changed by instantiating
an AutoSyncCache from the autosync submodule.

## Dependencies
psutil -- for asynchronous cache saving

## Features
* Drop in replacement for local memory dictionaries
* Default persistent pickle caches
* Non-persistent caching
* Cache load/save/delete hooks w/ defaults
* Cache validation hooks
* Cache builder hooks
* Dependent invalidation
* Auto-Syncing caches

## How to use
Below are some simple examples for how to use the repository.

### Setting up a simple persistent cache
from cacheman import cacher

manager = cacher.get_cache_manager() # Optional manager name argument can be used here
cache = manager.register_cache('my_simple_cache') # You now have a cache!
print cache.get('my_key') # `None` first run, 'my_value' if this code was executed earlier
cache['my_key'] = 'my_value'
cache.save() # Changes are now persisted to disk
manager.save_cache_contents('my_simple_cache') # Alternative way to save a cache

### Non-persistent caches
from cacheman import cacher

manager = cacher.get_cache_manager()
cache = manager.register_custom_cache('my_simple_cache', persistent=False) # You cache won't save to disk
cache.save() # This is a no-op

### Registering hooks
from cacheman import cacher
from cacheman import cachewrap

def my_saver(cache_name, contents):
print("Save requested on {} cache content: {}".format(cache_name, contents))

def my_loader(cache_name):
return { 'load': 'faked' }

manager = cacher.get_cache_manager()

cache = cachewrap.PersistentCache('my_cache', saver=my_saver, loader=my_loader)
# Can also use manager to set savers/loaders
#manager.retrieve_cache('my_cache')
#manager.register_saver('my_cache', my_saver)
#manager.register_loader('my_cache', my_loader)

cache.save() # Will print 'Save ... : { 'load': 'faked' }'
cache['new'] = 'real' # Add something to the cache
cache.save() # Will print 'Save ... : { 'load': 'faked', 'new': 'real' }'


### Dependent caches
from cacheman import cacher

manager = cacher.get_cache_manager()
edge_cache = manager.retrieve_cache('edge_cache')
root_cache = manager.register_cache('root_cache')
manager.register_dependent_cache('root_cache', 'edge_cache')

def set_processed_value():
# Computes and caches 'processed' from root's 'raw' value
processed = edge_cache.get('processed')
if processed is None:
processed = (root_cache.get('raw') or 0) * 5
edge_cache['processed'] = processed
return processed

# A common problem with caching computed or dependent values:
print set_processed_value() # 0 without raw value
root_cache['raw'] = 1
print set_processed_value() # still 0 because it's cache in edge

# Now we use cache invalidation to tell downstream caches they're no longer valid
root_cache.invalidate() # Invalidates dependent caches
print edge_cache # Prints {} even though we only invalidated the root_cache
root_cache['raw'] = 1
print set_processed_value() # Now 5 because the edge was cleared before the request
print edge_cache # Can see {'processed': 5} propogated

### Setting cache directory
from cacheman import cacher

# Default cache directory is '/tmp/general_cacher' or 'user\appadata\local\temp\general_cache'
# All pickle caches now save to namespaced directories within the base_cache_directory directory
manager = cacher.get_cache_manager(base_cache_directory='secret/cache/location')

cache = manager.register_cache('my_cache')
cache['new'] = 'real' # Add something to the cache
cache.save('my_cache') # Will save contents to 'secret/cache/location/general_cache/my_cache.pkl'

## Navigating the Repo
### cacheman
Package wrapper for the repo.

### tests
All unit tests for the repo.

## Language Preferences
* Google Style Guide
* Object Oriented (with a few exceptions)

## TODO
V2.1
* cPickle with fallback to pickle by default
* Better argument checks

## Author
Author(s): Matthew Seal

© Copyright 2013, [OpenGov](http://opengov.com)

Project details


Release history Release notifications

This version
History Node

2.0.8

History Node

2.0.6

History Node

2.0.5

History Node

2.0.4

History Node

2.0.2

History Node

2.0.1

History Node

2.0.0

History Node

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
CacheMan-2.0.8.tar.gz (11.7 kB) Copy SHA256 hash SHA256 Source None Jan 23, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page