Skip to main content

Cache any function to disk

Project description

What is this?

The smart way to cache outputs to cold storage.

  • auto rebuilds cache when you edit function source code
  • uses mutltiprocessing to keep main thread running fast while saving to disk
  • excellent change-tracking of arguments thanks to super_hash
  • can watch change-tracking of external vars and method attributes
  • uses python pickle for saving function outputs, if dill is available it will use that instead

How do I use this?

pip install cool_cache

from cool_cache import cache, settings

settings.default_folder = None # disable caching to cold-storage, and instead cache to ram
# this is the default, but you can change it
settings.default_folder = "cache.ignore/"

# 
# 
# simple usage (updates whenever function is edited (excluding comments) or when args change)
# 
# 
@cache()
def things_with_args(a,b,c):
    
    from time import sleep; sleep(1) # <- simulating a long-running process
    
    return a + b + c

things_with_args(1,2,3) # not yet cached
things_with_args(1,2,3) # uses cache
things_with_args(9,9,9) # not yet cached
things_with_args(9,9,9) # uses cache


# 
# 
# external vars
# 
# 
external_counter = 0
@cache(depends_on=lambda:[external_counter])
def things_with_external(a,b,c):
    global external_counter
    
    from time import sleep; sleep(1) # <- simulating a long-running process
    
    return external_counter + a + b + c


# 
# behavior
# 
things_with_external(4,5,6) # not yet cached
things_with_external(4,5,6) # uses cache
external_counter = 1
things_with_external(4,5,6) # not yet cached (because external_counter changed)
things_with_external(4,5,6) # uses cache

# 
# 
# filepath arguments
# 
# 
@cache(watch_filepaths=lambda arg1, arg2, arg3: [ arg1, arg2 ]) # because first two args are filepaths
def things_with_files(filepath1, filepath2, c):
    with open(filepath1, 'r') as in_file1:
        with open(filepath2, 'r') as in_file2:
            return in_file1.readlines() + c + in_file2.readlines()

# 
# behavior
# 
things_with_files("./file1.txt", "./file2.txt", "hello")  # not yet cached
things_with_files("./file1.txt", "./file2.txt", "hello")  # cached
with open("./file2.txt",'w') as f: f.write(str(" world")) # <-- modify the file
things_with_files("./file1.txt", "./file2.txt", "hello")  # not yet cached, because file change is detected
things_with_files("./file1.txt", "./file2.txt", "hello")  # cached

# 
# 
# class methods (e.g. self)
# 
# 
class MyThing:
    def __init__(self, path, other_stuff):
        self.path = path
        self.other_stuff = other_stuff
    
    # for example: self.path changing will affect the cache, but self.other_stuff wont affect the cache
    @cache(watch_attributes=lambda self:[ self.path, ])
    def do_some_stuff(self, arg1):
        from time import sleep; sleep(1)
        return self.path + arg1

# 
# bust=True wipes out all cached values for this function on the next run
# 
@cache(bust=True)
def things(a,b,c):
    return 10

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cool_cache-0.3.9.tar.gz (2.1 MB view details)

Uploaded Source

Built Distribution

cool_cache-0.3.9-py3-none-any.whl (2.2 MB view details)

Uploaded Python 3

File details

Details for the file cool_cache-0.3.9.tar.gz.

File metadata

  • Download URL: cool_cache-0.3.9.tar.gz
  • Upload date:
  • Size: 2.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/6.6.0 pkginfo/1.9.6 requests/2.30.0 requests-toolbelt/1.0.0 tqdm/4.65.0 CPython/3.8.13

File hashes

Hashes for cool_cache-0.3.9.tar.gz
Algorithm Hash digest
SHA256 d742ebce22d634d3bb1e6aa99bf02acadd9dc24c6cfeb06b1df683c05d64f421
MD5 56116fd597eb51ed493fa32748089ae7
BLAKE2b-256 91391c0ec6b5209ff78252c6afe202ad305cefbd10562f65683a663843ff59df

See more details on using hashes here.

File details

Details for the file cool_cache-0.3.9-py3-none-any.whl.

File metadata

  • Download URL: cool_cache-0.3.9-py3-none-any.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/6.6.0 pkginfo/1.9.6 requests/2.30.0 requests-toolbelt/1.0.0 tqdm/4.65.0 CPython/3.8.13

File hashes

Hashes for cool_cache-0.3.9-py3-none-any.whl
Algorithm Hash digest
SHA256 feae84488efea8f2b8f0e0bc565b62ca4772265fbac1123eef7986fe3db04735
MD5 965ce65c77eb8588ab115808d561dddc
BLAKE2b-256 7316a15ebc2fe5375b91166f56e745cb92126876bf8816006b39ec8960ca0e59

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page