Provides a decorator for caching a function and an equivalent command-line util.
Project description
Provides a decorator for caching a function and an equivalent command-line util.
It wraps an ordinary function. Whenever the function is called with the same arguments, the result is loaded from the cache instead of computed.
Quickstart
$ pip install charmonium.cache
>>> import charmonium.cache as ch_cache
>>> @ch_cache.decor(ch_cache.MemoryStore.create())
... def square(x):
... print('computing')
... return x**2
...
>>> square(4)
computing
16
>>> square(4) # square is not called again; answer is just looked up
16
>>> with square.disabled():
... # disable caching; always recomptue
... square(4)
...
computing
16
Customization
cache_decor is flexible because it supports multiple backends.
MemoryStore: backed in RAM for the duration of the program (see example above).
FileStore: backed in a file which is loaded on first call.
>>> import charmonium.cache as ch_cache
>>> @ch_cache.decor(ch_cache.FileStore.create("tmp/1"))
... def square(x):
... return x**2
...
>>> # Now that we cache in a file, this is persistent between runs
>>> # So I must clear it here.
>>> square.clear()
>>> list(map(square, range(5)))
[0, 1, 4, 9, 16]
>>> import os
>>> os.listdir("tmp/1")
['__main__.square_cache.pickle']
DirectoryStore: backed in a directory. Results are stored as individual files in that directory, and they are loaded lazily. Use this for functions that return large objects.
>>> import charmonium.cache as ch_cache
>>> @ch_cache.decor(ch_cache.DirectoryStore.create("tmp/2"))
... def square(x):
... return x**2
...
>>> # Now that we cache in a file, this is persistent between runs
>>> # So I must clear it here.
>>> square.clear()
>>> list(map(square, range(5)))
[0, 1, 4, 9, 16]
>>> import os
>>> sorted(os.listdir("tmp/2/__main__.square"))
['(0).pickle', '(1).pickle', '(2).pickle', '(3).pickle', '(4).pickle']
Custom stores: to create a custom store, just extend ObjectStore and implement a dict-like interface.
FileStore and DirectoryStore can both themselves be customized by:
Providing a cache_path (conforming to the PathLike interface), e.g. one can transparently cache in AWS S3 with an S3Path object.
Providing a serializer (conforming to the Serializer interface), e.g. pickle (default), cloudpickle, dill, or messagepack.
cache_decor also takes a “state function” which computes the value of some external state that this computation should depend on. Unlike the arguments (which the cache explicitly depends on), values computed with a different state are evicted out, so this is appropriate when you never expect to revisit a prior state (e.g. modtime of a file could be a state, as in make_file_state_fn).
With verbose=True, this will output to a logger.
>>> import charmonium.cache as ch_cache
>>> @ch_cache.decor(ch_cache.MemoryStore.create(), verbose=True)
... def square(x):
... print('computing')
... return x**2
...
>>> square(4) # doctest:+SKIP
2020-06-19 11:31:40,197 - __main__.square: miss with args: (4,), {}
computing
16
>>> square(4) # doctest:+SKIP
2020-06-19 11:31:40,197 - __main__.square: hit with args: (4,), {}
16
CLI
# cache a commandline function based on its args $ cache --verbose -- compute_square 6 miss for square(["6"]) 36 $ cache -- compute_square 6 hit for square(["6"]) 36
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for charmonium.cache-0.5.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 506561cb4705c6b9004f68d2ebc96f88ceb46147cec24c597b28da5df1067dee |
|
MD5 | efea9151a1313cd6e6ec55cdb98e3c64 |
|
BLAKE2b-256 | 25a8710e51f16624757f7d9de0780076314af2e57556dc91623e35237785e201 |