Skip to main content

persistent caching to memory, disk, or database

Project description

About Klepto

klepto extends python’s lru_cache to utilize different keymaps and alternate caching algorithms, such as lfu_cache and mru_cache. While caching is meant for fast access to saved results, klepto also has archiving capabilities, for longer-term storage. klepto uses a simple dictionary-sytle interface for all caches and archives, and all caches can be applied to any python function as a decorator. Keymaps are algorithms for converting a function’s input signature to a unique dictionary, where the function’s results are the dictionary value. Thus for y = f(x), y will be stored in cache[x] (e.g. {x:y}).

klepto provides both standard and “safe” caching, where “safe” caches are slower but can recover from hashing errors. klepto is intended to be used for distributed and parallel computing, where several of the keymaps serialize the stored objects. Caches and archives are intended to be read/write accessible from different threads and processes. klepto enables a user to decorate a function, save the results to a file or database archive, close the interpreter, start a new session, and reload the function and it’s cache.

klepto is part of pathos, a python framework for heterogeneous computing. klepto is in active development, so any user feedback, bug reports, comments, or suggestions are highly appreciated. A list of issues is located at, with a legacy list maintained at

Major Features

klepto has standard and “safe” variants of the following:

  • lfu_cache - the least-frequently-used caching algorithm
  • lru_cache - the least-recently-used caching algorithm
  • mru_cache - the most-recently-used caching algorithm
  • rr_cache - the random-replacement caching algorithm
  • no_cache - a dummy caching interface to archiving
  • inf_cache - an infinitely-growing cache

klepto has the following archive types:

  • file_archive - a dictionary-style interface to a file
  • dir_archive - a dictionary-style interface to a folder of files
  • sqltable_archive - a dictionary-style interface to a sql database table
  • sql_archive - a dictionary-style interface to a sql database
  • hdfdir_archive - a dictionary-style interface to a folder of hdf5 files
  • hdf_archive - a dictionary-style interface to a hdf5 file
  • dict_archive - a dictionary with an archive interface
  • null_archive - a dictionary-style interface to a dummy archive

klepto provides the following keymaps:

  • keymap - keys are raw python objects
  • hashmap - keys are a hash for the python object
  • stringmap - keys are the python object cast as a string
  • picklemap - keys are the serialized python object

klepto also includes a few useful decorators providing:

  • simple, shallow, or deep rounding of function arguments
  • cryptographic key generation, with masking of selected arguments

Current Release

This documentation is for version klepto-0.2.1.

The latest released version of klepto is available from:

klepto is distributed under a 3-clause BSD license.

>>> import klepto
>>> klepto.license()

Development Version

You can get the latest development version with all the shiny new features at:

If you have a new contribution, please submit a pull request.


klepto is packaged to install from source, so you must download the tarball, unzip, and run the installer:

$ tar -xvzf klepto-0.2.1.tar.gz
$ cd klepto-0.2.1
$ python setup py build
$ python setup py install

You will be warned of any missing dependencies and/or settings after you run the “build” step above.

Alternately, klepto can be installed with pip or easy_install:

$ pip install klepto


klepto requires:

  • python, version == 2.7 or version >= 3.6, or pypy
  • dill, version >= 0.3.4
  • pox, version >= 0.3.0

Optional requirements:

  • h5py, version >= 2.8.0
  • pandas, version >= 0.17.0
  • sqlalchemy, version >= 0.8.4
  • jsonpickle, version >= 0.9.6
  • cloudpickle, version >= 0.5.2
  • setuptools, version >= 0.6

More Information

Probably the best way to get started is to look at the documentation at Also see klepto.tests for a set of scripts that test the caching and archiving functionalities in klepto. You can run the test suite with python -m klepto.tests. The source code is also generally well documented, so further questions may be resolved by inspecting the code itself. Please feel free to submit a ticket on github, or ask a question on stackoverflow (@Mike McKerns). If you would like to share how you use klepto in your work, please send an email (to mmckerns at uqfoundation dot org).


If you use klepto to do research that leads to publication, we ask that you acknowledge use of klepto by citing the following in your publication:

Michael McKerns and Michael Aivazis,
"pathos: a framework for heterogeneous computing", 2010- ;

Please see or further information.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for klepto, version 0.2.1
Filename, size File type Python version Upload date Hashes
Filename, size klepto-0.2.1-py2.py3-none-any.whl (85.5 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size (180.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page