Skip to main content

Versatile cache line magic for jupyter notebooks, based on

Project description


This package adds %cache line-magic to Jupyter notebooks.


The author of this Python package makes no commitment to maintain it. It was forked from this project and you’re probably better off using that! That said, if you like the tweaks I made (added compression, bug fixes, style preferences, etc.), feel free to use it how you see fit. Just be sure to respect the original author’s license (see LICENSE copied here for your convenience).


  • The pip-package is called cache-magic
  • The python module is called cache_magic
  • The magic is called %cache

So you can run the magic by entering this into an Jupyter cell:

!pip install cache-magic
import cache_magic
%cache a = 1+1


install directly from notebook

  1. open jupyter notebook
  2. create new cell
  3. enter !pip install cache-magic
  4. execute

install into conda-environment

conda create -n test
source activate test
conda install -c pyython cache-magic
jupyter notebook


Activate the magic by loading the module like any other module. Write into a cell import cache_magic and excecute it.

When you want to apply the magic to a line, just prepend the line with %cache


%cache myVar = someSlowCalculation(some, "parameters")

This will calculate someSlowCalculation(some, "parameters") once. And in subsequent calls it restores myVar from storage.

The magic turns this example into something like this (if there was no ipython-kernel and no versioning):

  with open("myVar.pkl.gz", 'rb') as fp:
    myVar = pickle.loads(zlib.decompress(
  myVar = someSlowCalculation(some, "parameters")
  with open("myVar.pkl.gz", 'wb') as fp:

general form

%cache <variable> = <expression>

Variable: This Variable’s value will be fetched from cache.

Expression: This will only be excecuted once and the result will be stored to disk.

full form

%cache [--version <version>] [--reset] [--debug] variable [= <expression>]

-v or –version: either a variable name or an integer. Whenever this changes, a new value is calculated (instead of returning an old value from the cache).

if version is ‘*’ or omitted, the hashed expression is used as version, so whenever the expression changes, a new value is cached.

-r or –reset: delete the cached value for this variable. Forces recalculation, if <expression> is present

-d or –debug: additional logging

show cache


shows all variables in cache as html-table

full reset

%cache -r
%cache --reset

deletes all cached values for all variables

where is the cache stored?

In the directory where the kernel was started (usually where the notebook is located) in a subfolder called .cache

developer Notes

push to pypi

prepare environment:

gedit ~/.pypirc
chmod 600 ~/.pypirc
sudo apt install pandoc

upload changes to test and production:

pandoc -o README.rst
restview --pypi-strict README.rst
# update version in
rm -r dist
python sdist
twine upload dist/* -r testpypi
twine upload dist/*

test install from testpypi

pip install --index-url --extra-index-url cache-magic --no-cache-dir --user

test installation

sudo pip install cache-magic --no-cache-dir --user

editable import

Install into environment with -e:

!pip install -e .

reload after each change:

import cache_magic
from imp import reload

Alternatively (if you don’t want to install python, jupyter & co), you can use the docker-compose.yml for development:

cd cache-magic
docker-compose up

create Conda Packet

requires the bash with latest anaconda on path

mkdir test && cd test
conda skeleton pypi cache-magic
conda config --set anaconda_upload yes
conda-build cache-magic -c conda-forge

running tests

conda remove --name test --all
conda env create -f test/environment.yml
source activate test
conda remove cache-magic
pip uninstall cache-magic
pip install -e .

If there is any error, it will be printed to stderr and the script fails.

the output can be found in “test/temp”.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cache-magic-1.0.4.tar.gz (7.0 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page