Variety of object caching utilities for OWPHydroTools.
Project description
OWPHydroTools :: Caches
This subpackage implements different methods to cache objects generated by hydrotools
methods. See the Caches Documentation for a complete list and description of the currently available caches. To report bugs or request additional features, submit an issue through the OWPHydroTools Issue Tracker on GitHub.
Installation
In accordance with the python community, we support and advise the usage of virtual environments in any workflow using python. In the following installation guide, we use python's built-in venv
module to create a virtual environment in which the tool will be installed. Note this is just personal preference, any python virtual environment manager should work just fine (conda
, pipenv
, etc. ).
# Create and activate python environment, requires python >= 3.8
$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install --upgrade pip
# Install caches
$ python3 -m pip install hydrotools.caches
Usage
The following example demonstrates how one might use hydrotools.caches.hdf
to cache a pandas.dataframe
generated by a long running process.
Code
from hydrotools.caches.hdf import HDFCache
import pandas as pd
from time import sleep
# Some long running process that returns a pandas.DataFrame
def long_process(cols, rows):
sleep(1.0)
data = {f'col_{i}' : [j for j in range(rows)] for i in range(cols)}
return pd.DataFrame(data)
# Setup the cache with a context manager
# Similar to setting up a pandas.HDFStore
with HDFCache(
path='mycache.h5',
complevel=1,
complib='zlib',
fletcher32=True
) as cache:
# The first call runs long_process and stores the result
df = cache.get(long_process, 'data/results', cols=10, rows=1000000)
# The second call retrieves the result from cache without
# running long_process
df = cache.get(long_process, 'data/results', cols=10, rows=1000000)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hydrotools.caches-0.1.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0945ba15bd71b0e15b865f770fff90b65ce3f1eff81ceada763fe7c13c0c9224 |
|
MD5 | 134f670b5546da05cebbc9c581904f46 |
|
BLAKE2b-256 | f721fcdd73b80ce433f2121e61d5f18c8aabc3a9b0e0afafc547fd5694a1527d |