Skip to main content

A simple numpy caching library

Project description

Pyache

Python numpy caching library

This library caches numpy data that is generated from files and saves them in chunks to the disk. This is useful any time a computationally expensive task is done to files to transform them into a form needed in memory.

Usage

Create a Pycache object and call load with your filenames.

import numpy as np
from time import sleep
from pyache import Pyache


def load_file(filename) -> np.ndarray:
    print('Processing {}...'.format(filename))
    sleep(0.5)
    return np.ones([100])


pyache = Pyache('.cache', load_file, 'ones-processor')
data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png'],
    on_gen=lambda x: print('Just reprocessed', x),
    on_loop=lambda: print('Loaded one more...')
)  # Takes 1.5 seconds

# ... Run a second time (or program re-run):
data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png']
)  # Takes 0.0 seconds

data = pyache.load(
    ['thing-1.png', 'thing-2.png', 'thing-3.png', 'thing-4.png']
)  # Takes 0.5 seconds

Installation

pip install pyache

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyache, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size pyache-0.2.0-py3-none-any.whl (7.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size pyache-0.2.0.tar.gz (3.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page