Skip to main content

A simple numpy caching library

Project description

Pyache

Python numpy caching library

This library caches numpy data that is generated from files and saves them in chunks to the disk. This is useful any time a computationally expensive task is done to files to transform them into a form needed in memory.

Usage

Create a Pycache object and call load with your filenames.

import numpy as np
from time import sleep
from pyache import Pyache


def load_file(filename) -> np.ndarray:
    print('Processing {}...'.format(filename))
    sleep(0.5)
    return np.ones([100])


pyache = Pyache('.cache', load_file, 'ones-processor')
data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png'],
    on_gen=lambda x: print('Just reprocessed', x),
    on_loop=lambda: print('Loaded one more...')
)  # Takes 1.5 seconds

# ... Run a second time (or program re-run):
data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png']
)  # Takes 0.0 seconds

data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png', 'thing-4.png']
)  # Takes 0.5 seconds

Installation

pip install pyache

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyache-0.2.0.tar.gz (3.3 kB view hashes)

Uploaded source

Built Distribution

pyache-0.2.0-py3-none-any.whl (7.6 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page