tempcache 0.0.5

Python library to cache data and function results in temporary files

Python library to cache data and function results in temporary files

This library offers a basic method to cache data and functions to temporary files. By default it will use the pickle module to hash key values and to serialize data in temporary files. It is meant to work with long running functions whose results can be re-used. To avoid possible collisions make sure to use a unique name when instantiating TempCache.

Note For more advanced use cases you may want to look at the Memory class in joblib.

Basic Usage

An instance of the TempCache class be used as a decorator to automatically cache the results of a function.

from tempcache import TempCache

CACHE_MAX_AGE = 24 * 60 * 60 * 2    # two days
temp_cache = TempCache(__name__, max_age=CACHE_MAX_AGE)

@temp_cache
def long_running(...):
    ...

result = long_running(...)

Caching results at the call site

You can also use a TempCache object to cache a result at the call site with the cache_result method.

from tempcache import TempCache

CACHE_MAX_AGE = 24 * 60 * 60 * 2    # two days
temp_cache = TempCache(__name__, max_age=CACHE_MAX_AGE)

def long_running(...):
    ...

result = temp_cache.cache_result(long_running, ...)

Advanced usage

In cases where the function or some of its arguments are defined in the __main__ namespace or in a jupyter notebook and cannot be pickled by pickle you may want to use a different pickle module like cloupickle.

import cloudpickle

from tempcache import TempCache

CACHE_MAX_AGE = 24 * 60 * 60 * 2    # two days
temp_cache = TempCache("tempcache-foo",
                       pickler=cloudpickle,
                       max_age=CACHE_MAX_AGE)

Examples

Examples notebooks are in the extras folder.

Installation

You can install the latest version of this module with pip.

python -mpip install git+https://github.com/furechan/tempcache.git

Related Projects

• joblib Computing with Python functions
• cloudpickle Extended pickling support for Python objects