Skip to main content

A package that lets you cache your function outputs between runs

Project description

picklecache

This program allows you to stop programs in the middle and pick up where you left off with pickle module.

Documentation

Class picklecache.Picache:

  • directory: The directory to store the cache files in.

Usage:

  • Creation: mycache = picklecache.Picache("./path/to/my/files")
  • Cache a function: @mycache(subgroup='', picache_key_gen=lambda args, kwargs: (tuple(args), tuple(kwargs.items())))
  • This will decorate any function. By default, it uses the args and kwargs as keys.
  • When the function is decorated, you can pass picache_key=... to add your own caching key
  • This is useful if you have something like epochs in a Machine Learning model, and can't pass in a network as a key to the output

What are subgroups?

  • Subgroups are added as a prefix to the key. They're just so you can use the same caching folder for multiple functions without key collisions.
  • Keys are stored as a tuple, (subgroup, picache_key), so keys only have to be distinct for a certain function.
mycache = picklecache.Picache("./path/to/my/files")
@mycache("MyFunction1")
def costly_function():
    time.sleep(1000)
    return 100


costly_function() # takes a long time

Reload the program.

costly_function() # instant

This time, the run is instant. Key is stored as ("MyFunction1", ((), ()))

You could also specify a custom picache_key if you wanted to.

Author: Michael Fatemi

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for picklecacher-myfatemi04, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size picklecacher_myfatemi04-0.0.1-py3-none-any.whl (4.9 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size picklecacher-myfatemi04-0.0.1.tar.gz (3.4 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page