Skip to main content

A package that lets you cache your function outputs between runs

Project description

picklecache

This program allows you to stop programs in the middle and pick up where you left off with pickle module.

Documentation

Class picklecache.Picache:

  • directory: The directory to store the cache files in.

Usage:

  • Creation: mycache = picklecache.Picache("./path/to/my/files")
  • Cache a function: @mycache(subgroup='', picache_key_gen=lambda args, kwargs: (tuple(args), tuple(kwargs.items())))
  • This will decorate any function. By default, it uses the args and kwargs as keys.
  • When the function is decorated, you can pass picache_key=... to add your own caching key
  • This is useful if you have something like epochs in a Machine Learning model, and can't pass in a network as a key to the output

What are subgroups?

  • Subgroups are added as a prefix to the key. They're just so you can use the same caching folder for multiple functions without key collisions.
  • Keys are stored as a tuple, (subgroup, picache_key), so keys only have to be distinct for a certain function.
mycache = picklecache.Picache("./path/to/my/files")
@mycache("MyFunction1")
def costly_function():
    time.sleep(1000)
    return 100


costly_function() # takes a long time

Reload the program.

costly_function() # instant

This time, the run is instant. Key is stored as ("MyFunction1", ((), ()))

You could also specify a custom picache_key if you wanted to.

Author: Michael Fatemi

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

picklecacher-myfatemi04-0.0.1.tar.gz (3.4 kB view details)

Uploaded Source

Built Distribution

picklecacher_myfatemi04-0.0.1-py3-none-any.whl (4.9 kB view details)

Uploaded Python 3

File details

Details for the file picklecacher-myfatemi04-0.0.1.tar.gz.

File metadata

  • Download URL: picklecacher-myfatemi04-0.0.1.tar.gz
  • Upload date:
  • Size: 3.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.24.0 CPython/3.7.7

File hashes

Hashes for picklecacher-myfatemi04-0.0.1.tar.gz
Algorithm Hash digest
SHA256 0433e4fb0141d57f230b156bb279ff2f094be98c9c794301f6c657d87c5c9e97
MD5 620e90577d95cdf49646bbbcd579fe4f
BLAKE2b-256 fd74c9743a8bb9755293a2f1cf18b19c4aeb4e135a45504869378a7e5a329b77

See more details on using hashes here.

File details

Details for the file picklecacher_myfatemi04-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: picklecacher_myfatemi04-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 4.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.24.0 CPython/3.7.7

File hashes

Hashes for picklecacher_myfatemi04-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 7360c72435b52400283c7b8a308e59939822444a80f007875e2e813495940947
MD5 2dee29ddfe57812fe026eaca76581d1e
BLAKE2b-256 47e97dcd0f68bd24b4b63cee1568941350fa2221f00215dc4e13ff607c6fa1c1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page